AI-generated code is vulnerable

Large language models are trained to be helpful. When a developer asks an AI assistant to build an authentication endpoint, generate a database query, or set up a REST API, the model optimizes for the most direct path to functional code. It produces something that works: code that compiles, that runs, that returns the expected result. What it does not optimize for, in the absence of explicit instruction, is security. There is no intrinsic mechanism in standard LLM inference that weighs whether a generated SQL query is parameterized, whether an API key is hardcoded, or whether CORS headers are properly restricted. Functionality and security are distinct objectives, and the models consistently prioritize the former.

The problem is compounded by training data. These models learn from billions of lines of open-source code, Stack Overflow answers, tutorials, blog posts, and documentation. A significant fraction of that training corpus contains insecure patterns. Tutorials routinely demonstrate database queries with string concatenation for simplicity. Sample applications ship with wildcard CORS configurations. Example code hardcodes API keys for readability. The models internalize these patterns not as anti-patterns but as common solutions, and they reproduce them faithfully. The most statistically likely code completion is frequently the least secure one.

Perhaps most critically, AI coding assistants operate without security context. They do not know that the application they are generating code for handles medical records, processes payments, or is exposed to the public internet. They do not know whether the project uses an ORM that provides built-in parameterization or a raw database driver that requires manual escaping. They do not know the organization’s security policies, the regulatory requirements in play, or the threat model of the system. Without this context, the model cannot make informed security decisions, and so it makes none at all.

The result is a systemic and scalable vulnerability pipeline. Every developer using an AI coding assistant without security guardrails is, on average, introducing vulnerabilities into roughly seven out of every ten code generation interactions. Multiply this across engineering teams, across organizations, across the industry, and the aggregate security exposure is staggering. The velocity that makes AI code generation so valuable is the same velocity that makes its security failures so dangerous.

When this research was first scoped, AI-assisted attacks were a hypothetical concern raised in red-team papers and conference talks. That window has closed. In the past year alone, attackers have used commercial AI coding models to exfiltrate over 150GB of government data, automate up to 90% of multi-stage intrusion workflows, and weaponize leaked model source code into self-propagating malware. State-aligned groups have been observed running AI-driven reconnaissance against banks, energy grids, and software supply chains.

The asymmetry is the point. A single attacker with an AI agent can now probe more endpoints, draft more exploits, and adapt faster than entire defensive teams. The economics of offense have collapsed; the economics of defense have not. Patching after the fact, scanning after the merge, and reviewing after the deploy are all post-hoc strategies in a world where the offensive side is already operating at generation time.

This is the central insight behind Auditor. The only response that scales against AI-generated attacks is AI-generated defense, applied at the same layer, in the same loop, and with the same speed. You cannot detect your way out of a problem that is being created faster than you can read the diff.

Our benchmark evaluated five leading AI coding models (Claude Opus 4.6, Codex 5.1, GitHub Copilot, Gemini 2.0, and Cursor) across a comprehensive suite of code generation tasks designed to surface security vulnerabilities. We constructed over 2,400 distinct prompts spanning seven major web and backend frameworks: React, Next.js, Express.js, Django, FastAPI, Spring Boot, and Ruby on Rails. These prompts were drawn from real-world development scenarios, including building authentication systems, constructing API endpoints, implementing search functionality, setting up database models, configuring server middleware, and handling user input. Each represents a context where security-critical decisions are made at the code level.

Each prompt was executed twice per model: once in a baseline configuration with no security guidance, and once with Auditor’s context-aware security rules injected into the model’s system prompt and conversation context. Auditor’s rules were generated dynamically based on the detected framework and language of each task, mirroring the tool’s real-world behavior where it analyzes a project’s dependency manifest and technology stack to produce tailored, prioritized security instructions. This ensures the comparison reflects actual usage conditions rather than a contrived or artificially optimized scenario.

Generated code was evaluated against the OWASP Top 10 vulnerability categories, including injection flaws, broken authentication, sensitive data exposure, XML external entities, broken access control, security misconfiguration, cross-site scripting, insecure deserialization, use of components with known vulnerabilities, and insufficient logging and monitoring. Evaluation was performed through a multi-layered process: automated static analysis using Semgrep and CodeQL rule sets tuned for each framework, followed by manual expert review of flagged and borderline cases. A code sample was classified as vulnerable if it contained at least one confirmed, exploitable security flaw that could be leveraged in a production deployment without additional mitigation.

To ensure statistical robustness, each model-framework-vulnerability combination was tested across multiple prompt variations. Results were aggregated per model to produce the headline vulnerability rates. We also tracked per-category and per-framework breakdowns, which inform the detailed findings in this report.

The prevailing approach to code security (static analysis tools, linters, and SAST scanners) operates on a detect-and-remediate model. Code is written, then scanned, then flagged, then fixed. This workflow made sense in a world where humans wrote code deliberately and review cycles were measured in days. In the AI-assisted development paradigm, where code is generated in seconds and often accepted with minimal scrutiny, post-hoc detection is fundamentally insufficient. By the time a SAST tool flags a vulnerability in a pull request, the insecure pattern has already been written, the developer has already moved on, and the cognitive cost of understanding and fixing the issue is significantly higher than it would have been to generate secure code in the first place.

Auditor takes a fundamentally different approach: prevention at generation time through context-aware rule injection. When a developer’s project uses Express.js with PostgreSQL, Auditor detects this stack and injects security rules specifically about parameterized queries, helmet middleware, express-rate-limit, and CORS configuration with explicit origin whitelisting. When the project is a Django application, the rules shift to Django’s ORM patterns, CSRF middleware, template autoescaping, and DRF authentication classes. The AI model receives these rules as part of its context before generating a single line of code, producing secure output from the start.

The distinction between generation-time prevention and post-generation detection is not merely a workflow preference. It represents a structural advantage. Our benchmark data demonstrates that models are highly responsive to security context when it is provided in the right format and at the right time. The 95% average reduction in vulnerabilities is achieved by giving the model the security knowledge it needs, in the context of the specific technology stack it is generating code for, before it begins generating.

The integration of AI into software development is not a trend. It is an irreversible transformation. Within the next two to three years, the majority of production code will be either generated or substantially modified by AI assistants. The security implications of this shift are profound. If the industry continues to treat AI-generated code with the same post-hoc security tooling designed for human-written code, the result will be a dramatic expansion of the global attack surface.

The path forward requires a new category of security tooling: one that operates at the interface between the developer and the AI, shaping generation rather than inspecting output. Auditor represents an early and demonstrably effective instantiation of this approach. As AI models grow more capable and as code generation becomes more autonomous, the importance of embedded, context-aware security guidance will only increase. The models are capable of writing secure code. They simply need to be told what secure means for the specific application they are building.