AI agent threat modeling is the practice of systematically identifying, prioritizing, and mitigating security risks specific to systems where large language models (LLMs) pursue goals, call tools, and take autonomous actions. Unlike traditional application threat modeling, which focuses on data flows between deterministic components, agentic threat modeling must account for probabilistic behavior, natural-language attack surfaces, tool abuse, and multi-step chains of failure that no single component fully controls. As of August 2026, the field has matured enough that several structured approaches exist: Microsoft's guidance on threat modeling AI applications, STRIDE adaptations for LLMs, OWASP's Top 10 for LLM Applications, MITRE ATLAS for adversarial tactics against ML systems, and emerging automated approaches such as code-driven threat modeling tools like TITO. This article walks through what these techniques are, why they differ from classical methods, how to apply them step by step, and where teams most often go wrong.
Why AI Agents Break Traditional Threat Models
Also worth reading: What are the most effective prompt injection defense techniques in 2026 for securing AI applications? · How can I achieve the most professional results using AI headshot generation optimization techniques? · How can I create images using innovative techniques or tools?
Classical threat modeling assumes components behave deterministically: given input X, function Y produces output Z. An AI agent violates this assumption at every layer. The same prompt can yield different tool calls depending on retrieved context, model version, temperature settings, or injected content buried in a web page the agent just scraped. Unit 42 documented web-based indirect prompt injection observed in the wild, confirming that attackers no longer need direct access to your chat box — they can plant instructions in any content your agent ingests, from a support ticket to a GitHub README. This means the trust boundary is no longer the network perimeter; it is effectively every token the model reads.
A second structural difference is agency itself. A traditional web app cannot decide to email a customer, delete a database row, or transfer funds. An agent with tool access can. Anthropic's mapping of a year's worth of AI-enabled cyber threats found that adversaries increasingly use models not just as targets but as operators — automating reconnaissance, vulnerability triage, and even exploit validation. Cisco Talos reached similar conclusions when analyzing how adversaries weaponize AI, noting that the marginal cost of running a competent attack campaign has dropped dramatically. When you threat model an agent, you must therefore ask not only "what data can leak?" but "what actions can this system be manipulated into taking, and who bears the blast radius?"
Finally, the supply chain has changed. Model weights, embeddings, vector databases, RAG corpora, plugins, and MCP-style tool servers are all part of the attack surface. Morphisec's analysis of prompt injection, model poisoning, and AI supply chain attacks highlights that a poisoned document in a knowledge base functions like a persistent XSS payload inside your reasoning pipeline. Any credible threat modeling technique for agents must treat third-party content as untrusted executable logic — because to an LLM, instructions and data are the same medium.
Technique 1: Adapted STRIDE for Agentic Architectures
STRIDE (Spoofing, Tampering, Repudiation, Information Disclosure, Denial of Service, Elevation of Privilege) remains the backbone of most professional threat modeling, and Microsoft's published guidance on threat modeling AI applications extends it explicitly to machine learning components. Applied to agents, each category maps to concrete agentic risks. Spoofing becomes impersonation via forged context: an attacker crafts content that makes the model believe it is talking to a trusted administrator. Tampering covers both memory manipulation (rewriting conversation history or scratchpads) and RAG corpus poisoning. Repudiation is acute for agents because probabilistic outputs make audit trails ambiguous — if an agent takes a destructive action, can you reconstruct which input caused it?
Information disclosure in agentic systems includes training-data regurgitation, cross-session leakage through shared caches, and exfiltration via tool calls. A well-documented pattern is the "confused deputy" attack: an agent with read access to secrets and write access to an external channel (email, webhook, pasted URL) can be instructed by injected text to copy one into the other. Denial of service shifts from bandwidth exhaustion to economic exhaustion — an attacker who triggers expensive tool loops or long-context reprocessing can run up inference bills measured in thousands of dollars per day. Elevation of privilege is the most dangerous category: prompt injection that causes an agent to invoke admin-level tools, bypass approval workflows, or modify its own system prompt.
The practical adaptation is to draw your data flow diagram exactly as you would for a normal app, then add three new element types: the model itself (treat as a black box that can be lied to), every retrieval source (all untrusted), and every tool (each one a potential privilege escalation endpoint). For each trust boundary crossing, ask what happens when the content crossing it contains adversarial instructions rather than benign data. Teams that skip this step routinely discover, during incident response, that their agent had far more effective permissions than anyone intended.
Technique 2: OWASP LLM Top 10 as a Risk Checklist
The OWASP Top 10 for LLM Applications has become the de facto industry checklist, and its 2025 revision reflects lessons from real deployments. The entries most relevant to agents include prompt injection (both direct and indirect), insecure output handling (treating model output as trusted markup or code), excessive agency (granting more tool permissions or autonomy than needed), sensitive information disclosure, supply chain vulnerabilities (plugins, fine-tunes, embedding models), and unbounded consumption. Wiz.io's breakdown of serious AI security risks aligns closely with these categories, emphasizing that misconfigured AI infrastructure — exposed vector databases, permissive IAM roles on inference endpoints — causes as many breaches as novel model attacks.
Using OWASP as a threat modeling technique works best as a coverage check after architecture-specific analysis. Walk each entry against your design and record whether it applies, what controls exist, and what residual risk remains. For example, under "excessive agency," a coding agent that can execute shell commands should be scored against questions like: does it run sandboxed, does it require human approval for network access, can it modify its own configuration files? Under "insecure output handling," trace every place model output flows into HTML rendering, SQL queries, shell execution, or file writes. BankInfoSecurity reporting on how AI agents validate software vulnerabilities illustrates both sides: agents genuinely accelerate triage, but an agent that reads attacker-controlled issue reports and then executes suggested reproduction steps is one injection away from becoming an attack primitive.
The honest limitation of checklist approaches is that they encode yesterday's attacks. They will not surface a novel multi-agent collusion scenario or a timing-based side channel in your caching layer. Use them as a floor, not a ceiling — a minimum standard below which you should not ship, while reserving deeper analysis for the highest-risk agent capabilities.
Technique 3: MITRE ATLAS and Adversary-Centric Modeling
MITRE ATLAS (Adversarial Threat Landscape for Artificial-Intelligence Systems) adapts the ATT&CK framework to ML systems, cataloguing real-world adversary tactics and techniques from poisoning and evasion to model extraction. Where STRIDE asks "what can go wrong structurally?" and OWASP asks "what categories of flaw exist?", ATLAS asks "what would a competent attacker actually do, in what order?" This adversary-centric framing is valuable because it forces you to think in kill chains rather than isolated vulnerabilities.
For an agentic deployment, an ATLAS-informed exercise might proceed as follows. Reconnaissance: the attacker probes your public-facing assistant to map its tools, refusal behaviors, and system-prompt structure — research consistently shows system prompts are extractable within minutes. Initial access: indirect prompt injection through any ingested content channel, or direct jailbreaking of a user-facing endpoint. Execution: the injected instruction directs tool use — fetching a malicious URL, writing a file, calling an internal API. Persistence: the agent writes attacker-controlled content into a memory store or knowledge base, so the payload survives across sessions. Impact: data exfiltration, fraudulent transactions, or reputational damage through the agent acting in your brand voice. Each stage maps to specific mitigations: content provenance labeling, tool allowlists, egress filtering, memory isolation, and rate limits.
Anthropic's threat intelligence work and Microsoft's agentic security research both stress that defenders should assume injection attempts will succeed sometimes and design so that a successful injection is survivable. That principle — containment over prevention — is arguably the single most important shift in mindset for agentic threat modeling. You cannot patch a language model the way you patch a buffer overflow; you architect so that even a fully compromised model cannot cause unacceptable harm.
Comparing the Major Techniques Side by Side
No single technique covers everything, and mature teams typically layer two or three. The table below compares the four dominant approaches as practiced in mid-2026.
| Dimension | STRIDE Adaptation | OWASP LLM Top 10 | MITRE ATLAS | Automated Code-Based Modeling |
|---|---|---|---|---|
| Primary lens | Structural data flows | Known flaw categories | Adversary kill chains | Source-code analysis |
| Best stage | Design time | Pre-release review | Red team planning | CI/CD continuous |
| Coverage of novel attacks | Moderate | Low | High | Low–moderate |
| Effort required | High (workshops) | Medium | High (expertise) | Low after setup |
| Tooling maturity | Mature (Microsoft templates) | Mature (checklists) | Growing | Emerging (e.g., TITO) |
| Main blind spot | Probabilistic behavior | Zero-day patterns | Requires red-team skill | Misses runtime/context risks |
Practical Steps: Running an Agent Threat Model in Two Weeks
A realistic engagement for a team deploying its first production agent fits into roughly ten working days. Days one and two: inventory. Enumerate every model, every tool, every data source, every identity the agent uses, and every human approval gate. Draw the data flow diagram including the model as a node. Most teams discover surprises here — an agent quietly inheriting a service account with broad permissions, or a RAG pipeline ingesting a wiki that anyone in the company can edit.
Days three to five: apply adapted STRIDE across each trust boundary, scoring findings by likelihood and impact. Pay particular attention to the confused-deputy pattern: list every pair of (readable secret, writable external channel) and treat each pairing as a candidate exfiltration path. Days six and seven: map findings to OWASP LLM categories and assign controls — input/output filtering, tool allowlisting, least-privilege credentials, human-in-the-loop thresholds, sandboxing, and full audit logging of prompts, tool calls, and outputs. Days eight and nine: red-team the design using ATLAS kill chains, attempting actual injections against a staging instance. Day ten: document residual risk, set monitoring thresholds, and schedule quarterly reviews. Defense-in-depth principles from enterprise platforms like Workday's published guidance apply directly: no single control should be load-bearing.
Two quantitative anchors help prioritize. First, cap effective autonomy: a common threshold is requiring explicit human approval for any action that is irreversible, touches money, or sends communications outside the organization. Second, budget for abuse: model worst-case inference spend under adversarial load and set hard daily ceilings — teams have reported surprise bills in the five-figure range from runaway agent loops before cost guardrails existed.
Common Mistakes and How to Avoid Them
The most frequent error is treating the system prompt as a security control. Prompt-level instructions are suggestions to a sufficiently motivated attacker; every serious publication on the topic, from Unit 42's field observations to vendor guidance, treats system-prompt leakage as inevitable. Put real enforcement in code: permission checks, allowlists, and validators that run outside the model. The second mistake is scoping the model to the threat model's edge — forgetting that the browser extension, the Slack integration, the cron job that feeds documents into the vector store, and the third-party plugin are all part of the system. Supply chain analysis of AI attacks repeatedly shows the initial compromise arriving through an adjacent component, not the model API itself.
Third, teams over-index on preventing injection and under-invest in containment. Assume the injection lands; ask what the agent could still do. If the answer is "delete production data," your permission model is wrong regardless of filter quality. Fourth, neglecting repudiation and forensics: log every prompt, completion, tool invocation, and retrieved chunk with timestamps and session IDs, or you will be unable to reconstruct incidents. Fifth, treating threat modeling as a one-time gate. Agent behavior changes with every model update — a silent provider-side upgrade can alter refusal rates and tool-calling tendencies overnight, which is why quarterly (or per-deployment) re-review is now standard practice among mature adopters. Finally, do not ignore the human layer: social engineering of the people who approve agent actions remains cheaper and more reliable than any technical bypass.
When to Act and What It Costs
Threat modeling should happen before the first production deployment, not after the first incident — retrofitting permissions and logging onto a live agent is dramatically harder than designing them in. Trigger a fresh modeling cycle whenever you add a new tool, expand data access, change model providers, enable autonomous multi-step execution, or expose the agent to untrusted users. Organizations in regulated sectors should note that auditors in 2026 increasingly expect documented AI risk assessments alongside traditional SOC 2 and ISO evidence, so the artifact itself has compliance value beyond security.
Costs vary widely. The DIY route — a trained engineer applying Microsoft's free AI threat modeling templates, the OWASP checklist, and MITRE ATLAS — costs primarily time: roughly 40 to 80 person-hours for a first pass on a moderately complex agent. Commercial platforms for AI security posture management and automated threat modeling typically run from tens of thousands of dollars annually for small deployments to six figures for enterprise portfolios. Dedicated red-team engagements against agentic systems commonly price between $30,000 and $150,000 depending on scope. Against those figures, weigh the cost of a single incident: an exfiltrated customer database, a fraudulent transaction series executed by a compromised agent, or regulatory exposure under expanding AI governance rules routinely exceeds seven figures. Even modest investment in structured threat modeling pays for itself the first time it prevents one confused-deputy exfiltration.
The bottom line: effective AI agent threat modeling in 2026 combines adapted STRIDE for structure, OWASP LLM Top 10 for coverage, MITRE ATLAS for adversary realism, and automation for keeping the model current — all built on the assumption that injections will succeed and the design goal is limiting blast radius rather than achieving perfect prevention.