As AI moved from chat interfaces to coding tools to autonomous agents, its reach into real systems grew. So did the attack surface, and the security models built for each prior era didn't carry over.
The attack shapes are familiar: injection, impersonation, privilege escalation. Prompt injection and SQL injection are structurally the same thing: attacker-controlled input that changes execution intent. What changed is how the attack is expressed.
In deterministic systems, defenders can enumerate valid states and block invalid ones. A WAF can reject '; DROP TABLE-- because SQL is a formal grammar with a finite valid space. In an LLM system, there is no such boundary. Every string of natural language is potentially legitimate input. Defense at the syntactic layer fails because the attack lives at the semantic layer, and semantics cannot be pattern-matched.
Consider the compounding: we've spent decades building defenses for deterministic systems, and those systems still get breached regularly. AI adds a non-deterministic layer on top of that existing struggle. Same attack shapes, but the input space is now infinite natural language, execution is probabilistic, and "correct" output isn't formally defined. The difficulty doesn't just increase. It changes category.
| Dimension | Deterministic Layer (App / API) | Non-Deterministic Layer (AI / LLM) |
|---|---|---|
| Attack vector | HTTP requests, SQL strings, binary payloads | Natural language prompts |
| Target | Application logic, database, OS | Model behaviour, guardrails, system prompts |
| Payload format | Structured: code, bytes, query syntax | Unstructured: plain text, infinite variations |
| Detection | WAF rules, regex, IDS signatures | No reliable signature: semantic rewrites evade every filter |
| Blast radius | Data exfil, RCE, privilege escalation | Policy bypass, prompt leak (bounded by model having no real-world reach) |
| Defender's lever | Block the malformed input | Can't reliably block natural language; must constrain what the model can do |
That last row is the crux. In the app layer, defenders block bad inputs. In the AI layer, the "bad input" is text, and text cannot be reliably blocked. Two years of prompt filter research confirmed this. None of them held.
Defenders were still working on that problem when the capability curve jumped. GPT-4 didn't just make jailbreaks harder to patch. It sent generative AI in two directions at once: into developer tooling as a code contributor, and into professional workflows as an information source. The attack surface didn't move domains. It forked.
GitHub Copilot wired AI into developer workflows in 2023, making AI a direct contributor to code that ships. Two problems entered together: the supply side feeding into development was compromised, and the development process itself became sloppier.
The downstream effect is the compound: code that may have been designed to be vulnerable, built through a process that can't audit itself at scale. That was one fork. Running in parallel, the same capability was entering professional workflows: not as a code contributor, but as the information source organisations were reading, citing, and acting on.
The same models that generated code began handling professional research: contract review, financial due diligence, legal analysis, threat triage. The capability wasn't new. What changed was organisational trust. Lawyers, auditors, and analysts started treating AI summaries as authoritative input to decisions they no longer had time to verify at source.
What's telling is that no attacker was needed. The model's own hallucinations caused the damage, generated with the same confidence as accurate output, and trusted the same way. That reliability gap is also an adversarial gap: any organisation that has demonstrated it won't verify AI output is equally exposed to content deliberately crafted to steer the AI's conclusions. The gap hallucination exposes is the same gap an attacker would exploit.
All three incidents happened at the information layer, where a human was still nominally in the loop and could have verified. They didn't. Consider what happens at the actuation layer, where there is no checkpoint to miss.
OpenClaw runs on your own machine using Anthropic's Computer Use API, always on, with access to your file system, browser, calendar, and terminal. When it gained traction, developers bought Mac Minis to host it as a 24/7 AI colleague. Jensen Huang called agentic AI the next era of computing at CES 2025. Pilots were announced before architectures were designed. Nobody was asking what the safety model was for an agent that acts as you, with your credentials, around the clock.
At the same time, fierce model competition collapsed the cost of attack. Chinese open-weight models (DeepSeek, Qwen) run locally, require no API key, and carry few content restrictions. What once required expensive commercial APIs or specialised criminal tooling now runs on commodity hardware at near-zero marginal cost. Generating thousands of convincing, personalised injection payloads is a reusable prompt, not a deep competency.
These two curves meet in indirect prompt injection. Your agent reads a malicious README. The embedded instruction takes over: exfiltrate secrets, redirect output, run a secondary payload. It executes with your credentials, your session, your authorised tools. Audit log says you did it. Traditional access control sees an authenticated session taking authorised actions. Every check passes. There is no protocol in any existing model that distinguishes agent acting on your intent from agent acting on an injected instruction. That is not a misconfiguration. It is structural.
AI didn't add one new attack surface. It forked: one path into the development pipeline where AI became a direct contributor to code, one path into the information layer where AI became the source of answers. Then both converged into an autonomous actuation layer where the rules changed again.
The fork maps to two attack paths that look nothing alike in practice. A direct attack comes through the interface you built. An indirect attack comes through content your agent was told to trust.
| Direct Attack | Indirect Injection | |
|---|---|---|
| How attacker reaches AI | Types directly into the chat interface | Plants instructions in content the agent reads: a README, a web page, a model card |
| Real example | Jailbreaking ChatGPT to reveal confidential system prompt | Malicious text in a HuggingFace model card hijacks a developer's AI coding agent mid-task |
| What the AI does | Produces harmful text output: leaked data, bypassed policy | Takes real action: calls APIs, writes files, exfiltrates data, using your credentials |
| Blast radius | Bounded by what the model can say | Bounded only by what the agent can do |
| What the logs show | Attack visible in conversation history | Audit log says you did it. Attacker is invisible. |
| Defense lever | Input validation, output filtering, guardrails | Constrain what agents can reach and act on |
The first era: attacker manipulates what AI says. The second: attacker hijacks what AI does with your access.
The problem splits cleanly into two tracks. Not because they use different frameworks, but because the attack path is different: one comes at you directly, the other rides the agent you already trust.
Prompt injection, jailbreaks, goal hijacking via the input channel. The app is the surface. Defense means trust boundaries, I/O validation, and session isolation inside the LLM application itself.
Securing the Application Layer →Injected instructions in content the agent reads, phantom packages in generated code, poisoned training data. The attack never touches your app. It lands on your host when the agent executes. Defense moves to host and network: Reachability constraints, Capability curtailment, tool permission scoping.
Host + Network Defense Framework →