←Research
Researchsecurity8 min read

Marking your own homework at machine speed

Anthropic's Claude Code Security found 500 zero-days in open-source code. The industry's reaction revealed more about the state of software security than the tool itself.

S
Sebastion
Marking your own homework at machine speed

Anthropic released Claude Code Security on 20 February 2026 - a tool that reads code the way a human security researcher would, scanning for vulnerabilities that pattern-matching tools miss and suggesting patches for human review. It arrived backed by a striking claim: the underlying model, Claude Opus 4.6, had found over 500 previously unknown high-severity vulnerabilities in production open-source codebases. Bugs that survived decades of expert review. Bugs that thousands of hours of automated fuzzing never triggered.

Within days, cybersecurity stocks dropped. Industry analysts scrambled. Application security vendors rushed to explain why they weren't obsolete.

The technology is impressive. But the reaction revealed something more interesting than the tool itself: a fundamental tension about whether the same AI that writes an increasing share of the world's code should also be the one auditing it.

How it works

Claude Code Security doesn't operate like a traditional static analysis tool. Where conventional scanners match code against known vulnerability patterns - looking for hardcoded credentials, outdated encryption or unparameterised queries - Claude reads the codebase holistically. It traces data flows across files, reasons about how components interact and identifies vulnerabilities in business logic that no signature database would catch.

Every finding goes through a multi-stage verification loop. Claude re-examines each result, attempting to prove or disprove its own findings before anything reaches an analyst. Validated issues appear in a dashboard with suggested patches, confidence ratings and severity scores. Nothing gets applied without human approval.

The tool launched as a limited research preview for Enterprise and Team customers. Open-source maintainers can apply for free expedited access - a significant detail, given that many of the highest-impact codebases are maintained by small teams without dedicated security resources.

The 500 zero-days

The headline number deserves scrutiny because the methodology behind it is novel.

Anthropic's Frontier Red Team placed Claude Opus 4.6 inside a virtual machine with access to the latest versions of open-source projects, standard utilities and vulnerability analysis tools. No custom scaffolding. No specialised prompts. They were testing whether the model could find real bugs out of the box.

The GhostScript vulnerability is illustrative. Claude initially tried fuzzing - the standard automated approach - and got nowhere. Then it tried manual analysis, also without success. It changed strategy entirely: reading the Git commit history, finding a security-relevant commit that added stack bounds checking and reasoning that similar unpatched vulnerabilities might exist in other call paths. It found one in gdevpsfx.c and built a working proof-of-concept crash.

In CGIF - a GIF processing library - Claude identified a heap buffer overflow that required a conceptual understanding of the LZW compression algorithm. The vulnerability only triggers when compressed output exceeds uncompressed size, something that traditional coverage-guided fuzzers struggle to reach even with complete branch coverage. Claude understood the algorithm well enough to construct exact input that would overflow the buffer.

These aren't theoretical findings. Maintainers are patching them. The responsible disclosure process is ongoing as of late February 2026.

The pushback

The industry reaction was swift and pointed.

In one test posted to LinkedIn and cited by Dark Reading, an analyst found Claude Code Security took 17 minutes to review a code sample. It flagged three vulnerabilities, two of which were false positives. OpenGrep - a conventional SAST tool - found the same real issue in 30 seconds. Neatsun Ziv, CEO of OX Security, confirmed similar results in his own testing: more than 15 minutes and roughly $4 in token costs for a single scan that a purpose-built scanner handles for less than a cent.

The speed and cost gap matters at enterprise scale. An organisation with hundreds of repositories cannot afford to spend 17 minutes and $4 per scan when the same detection is available near-instantly from existing tools. The value proposition has to come from finding things those tools miss - and in early testing, the false positive rate undermined that pitch.

Randall Degges, VP of AI engineering at Snyk, made a sharper structural point. "Finding vulnerabilities has never been the hard part," he wrote. "The hard part - the part that keeps AppSec teams up at night - is fixing them. At scale. Across hundreds of repositories. Without breaking anything." Claude Code Security identifies problems and suggests patches. It doesn't integrate with CI/CD pipelines, triage across an organisation's full attack surface or track remediation over time.

Marking your own homework

The most pointed critique comes from OX Security, and the logic is hard to dismiss.

Claude writes code. Claude reviews code. Claude suggests fixes. At no point in this loop does an independent system validate the findings. As OX Security puts it: "When the same system writes, evaluates, filters and patches, you don't have independent validation. You have self-approval. And self-approval is not security."

This is not a theoretical concern. In OX's recent testing of AI application builders, AI-generated internal security scans cleared code that contained basic exploitable XSS vulnerabilities. The model passed its own work. Security has always depended on separating implementation from verification - having different eyes on the code than the ones that wrote it. Collapsing that separation into a single model collapses the governance structure that makes modern software survivable.

OX raises one more uncomfortable question: if Claude can detect a vulnerability during scanning, why did it - or a model like it - generate that vulnerability in the first place? Either it didn't understand secure implementation while writing, or it only recognised the flaw during review. Neither answer inspires confidence if you're delegating both writing and auditing to the same technology.

Anyone who has worked in security knows this instinctively. You don't let the developer who wrote the feature sign off the security review. You bring in someone else - someone with different assumptions, different blind spots and a mandate to find problems rather than ship features. The agentic AI security landscape is already struggling with tools that promise to do everything and end up creating the same category of risk they claim to eliminate.

The numbers that should worry everyone

The irony runs deeper than one product announcement.

ETH Zurich, UC Berkeley and INSAIT developed BaxBench - a benchmark that evaluates LLMs on security-critical backend coding tasks. As of early 2026, 62% of solutions generated by even the best-performing models are either incorrect or contain security vulnerabilities. Claude Opus 4.5 - Anthropic's previous top performer - produced secure and correct code only 56% of the time without specific security prompting. The same family of models now being positioned as your security auditor is simultaneously one of the most prolific sources of insecure code the industry has ever seen.

Veracode's 2026 State of Software Security report paints a broader picture. Security debt - vulnerabilities left unresolved for more than a year - now affects 82% of organisations, up from 74% in 2025 and 71% in 2024. Critical security debt impacts 60% of organisations, a 20% relative increase. High-risk vulnerabilities are up 36% year-over-year. The report attributes the acceleration to AI-assisted development: more code ships faster, but the mechanisms for fixing flaws haven't kept pace with the mechanisms for creating them.

Ben Livshits formalised the theoretical case in a February 2026 arXiv paper titled "LLMs + Security = Trouble." He argues that the prevailing "fighting fire with fire" approach - using probabilistic AI-based checkers to secure probabilistically generated code - fails to address the long tail of security bugs. The model might catch common patterns. It will miss the edge cases that matter most. And those edge cases are precisely what sophisticated attackers exploit. Livshits proposes enforcing security constraints during code generation rather than relying on post-hoc detection - a fundamentally different architecture to the scan-and-patch model.

Where this actually helps

This isn't a story about a bad product. Claude Code Security is doing something no rule-based scanner can - reasoning about code at a level that catches bugs a decade of fuzzing missed. The GhostScript and CGIF examples demonstrate capability that traditional tooling simply cannot replicate. The 500 zero-days are real.

The question is where it fits.

Snyk's Degges probably has the most balanced take: "AI belongs in the remediation loop, not just the detection loop. The breakthrough isn't 'AI can find vulnerabilities.' The breakthrough is 'AI can reason about code well enough to fix vulnerabilities.'"

The right architecture layers AI reasoning where it excels - novel vulnerability discovery, cross-file logic bugs and complex business logic flaws - with deterministic analysis where it excels: known patterns, supply chain risks and compliance-critical findings with auditable traces. Use AI to discover. Use traditional tools to verify. Use AI again to remediate. Use traditional tools to re-verify. Neither approach alone is sufficient.

The OpenClaw saga showed what happens when AI tools ship without a security model. Claude Code Security is far more considered than that. But the architectural question persists: who watches the watcher when the watcher is also the builder?

Anthropic is clearly aware of this tension. They've released the tool as a limited preview, explicitly inviting collaboration. They've built in human approval gates. They've published their methodology. They're running responsible disclosure. This is not a reckless launch.

But the industry is generating security debt faster than it can pay it down. AI-written code is accelerating that trend. An AI that can also find some of the resulting bugs is welcome. The real question is whether organisations are building the layered defences needed for a world where AI is simultaneously the best code reviewer anyone has ever seen and the most prolific source of insecure code.

Newsletter

One email a week. Security research, engineering deep-dives and AI security insights - written for practitioners. No noise.