Somewhere in your organisation, someone has already pasted an architecture description into a chatbot and typed "what are the security threats?" The answer came back in seconds: a tidy list, sorted by STRIDE category, with mitigations. It looked like a threat model, and it took less time than booking the meeting room for a real one.
So the useful question isn't whether teams will use large language models for threat modeling. They already do. It's which parts of the work they actually make better, which parts they quietly make worse, and how to tell the difference before the output ends up in a risk register with someone's name on it.
The short version: LLMs are good at breadth and bad at your system. Use them to suggest threats you missed, draft from documents you already have, and challenge a model people built. Don't use them to decide what your system looks like, which risks matter, or whether the model is complete. Every specific claim they make, from a CVE number to a compliance clause, needs checking.
Start from what threat modeling is for
The clearest way to place AI in the process is Adam Shostack's four-question framework, which most methods boil down to: What are we working on? What can go wrong? What are we going to do about it? Did we do a good job? An LLM's usefulness is very different for each one.
| Question | Where an LLM helps | Where it can't |
|---|---|---|
| What are we working on? | Turning design docs, API specs and infrastructure code into a first-draft list of components and flows | Knowing what isn't written down: the admin script, the support tool, the shared credential |
| What can go wrong? | Breadth: well-known threat classes for each kind of component, and the ones the team forgot | Threats that depend on your business logic, your users or your history |
| What are we going to do about it? | Drafting mitigation wording, explaining threats to different audiences | Deciding what's worth fixing, what to accept and who owns it |
| Did we do a good job? | Acting as a second reviewer that asks what was missed | Telling you the model is complete; it has no way to know |
The pattern is consistent. LLMs are useful wherever the work is generating candidates. They are unreliable wherever the work is knowing facts about your system or making decisions about risk.
Where LLMs genuinely help
Breaking the blank page. The hardest moment in a first threat modeling workshop is the silence after "so, what could go wrong?" A model that has read a great deal of security writing can list the usual suspects for a message queue, an OAuth callback or an object storage bucket faster than anyone in the room. Those are exactly the threats that well-documented component types attract, and they are the ones teams most often skip because they seem obvious.
Drafting from material you already have. Most teams have a design document, an OpenAPI spec or a folder of Terraform. An LLM can turn those into a first-draft list of elements, data stores and flows for a data flow diagram. It will get things wrong, but correcting a draft is faster than starting from nothing, and the corrections themselves are often where the team learns something about its own system.
Rewording and translating. The same threat needs to be described differently for a developer fixing it, a manager prioritising it and an auditor reviewing it. Rewriting a terse attack-tree leaf into a clear ticket, or a mitigation into language that an auditor will accept, is the kind of task these models do well, as long as someone checks the meaning didn't change.
Challenging a finished model. This is the most valuable use, and the least common. Once people have built the model, give the LLM the elements, flows, trust boundaries and threats, and ask what's missing. A challenger doesn't have to be right to be useful. It has to raise a question the team then answers properly.
Where they mislead
They only know what you told them. The most dangerous threats in a real system usually live in what nobody documented: the operations dashboard with no authentication because it's "internal", the batch job running with production admin rights, the vendor with a standing VPN account. An LLM given the architecture document will model the architecture document. The insider paths nobody threat models stay invisible to it for the same reason they're invisible on the diagram.
Volume looks like rigour. Ask for threats against a ten-component system and you'll get dozens, each plausible, few prioritised, many generic. A list of 60 threats feels thorough, but if 50 of them are "ensure input validation" variants, the five that matter are now harder to find. Without someone applying judgment, AI makes the threat backlog problem worse, not better.
Specific claims can be confidently wrong. Language models produce fluent text, not verified facts. A CVE identifier that doesn't match the vulnerability described, a compliance requirement number that doesn't exist, a library "feature" that was never shipped: all of these appear in otherwise sensible output, and they read exactly as confidently as the correct parts. Anything specific enough to cite is specific enough to check.
The answer changes when you ask again. Run the same prompt twice and you'll get two overlapping but different lists. That's fine for brainstorming and a problem for coverage. If you want to say "we considered spoofing for every flow that crosses a trust boundary", that claim needs a structure behind it, such as a STRIDE pass per element on a diagram, not a transcript that might have come out differently.
Risk decisions need an owner. Whether a threat is worth fixing depends on likelihood, impact, cost and how much risk the organisation is willing to carry. Those are business judgments, and someone has to be accountable for them. "The AI rated it low" is not a risk acceptance, and no auditor or incident review will treat it as one.
The part an LLM can't do for you
The Threat Modeling Manifesto, published by a group of practitioners in 2020, values "people and collaboration over processes, methodologies, and tools" and "a journey of understanding over a security or privacy snapshot." That's the strongest argument for keeping people at the centre of the work.
A large part of the value of threat modeling is what happens to the team while they do it. The developer who realises the webhook handler trusts the payload. The architect who discovers two services share a database nobody drew. The product owner who learns why "just add an admin override" is a bigger decision than it sounds. A generated document skips all of that. You end up with a model and a team that doesn't understand it, which is how threat models become shelfware.
Your threat model is sensitive data
Before pasting anything, remember what a threat model is: a map of where your system is weakest, written down. Sending it to an AI service is sending it to a third party, so treat the tool the way you would any vendor in your own diagram:
- What does the provider's agreement say about retention, and about using your inputs for training?
- Does your policy require an enterprise agreement, a specific region or a self-hosted model for this class of data?
- Have you removed secrets, internal hostnames, IP ranges and customer data from what you share?
AI tools that do more than answer questions add a further risk. An assistant that reads your tickets, wiki or repository to "understand the system" is reading content anyone with write access can influence, including instructions aimed at the assistant itself. If you use one, it belongs on the diagram as a component with its own trust boundaries. Our guide to threat modeling AI agents and LLM applications covers how to model exactly that.
A workflow that keeps the strengths and drops the risks
The teams getting real value from AI in threat modeling tend to put it in a specific place in the process, rather than at the start:
- People draw the system. The diagram, the trust boundaries and the assumptions come from the people who build and run it. An LLM draft from existing documents is fine as raw material, as long as the team reviews it line by line.
- People do the first pass. Run STRIDE across the diagram and build the attack trees for the outcomes you care about most. This is where the team's understanding comes from.
- The LLM challenges. Give it the structured model, not a vague description, and ask what's missing. Require every suggestion to name the element or flow it applies to and the assumption that would have to be false for it to work.
- People triage every suggestion. Accept it into the model or reject it with a one-line reason. The reasons are useful later; a rejected suggestion with "not reachable: no public endpoint" is evidence of a decision, not an omission.
- People own the risk. Scoring, mitigations, owners and acceptances stay with named people.
- Someone checks every reference. CVEs, standards clauses, framework IDs and product behaviour all get verified against the source before they go into the model.
Prompting matters less than structure, but a few habits help. Give the model the element list, flows and boundaries rather than a paragraph of prose. Ask it to critique rather than generate. Ask what it can't know from what you gave it, which often surfaces the undocumented parts of the system faster than anything else.
Common mistakes
- Starting with the LLM. The model inherits the AI's view of a generic system instead of the team's view of the real one, and nobody notices what's missing.
- Pasting the output straight into the risk register. Unreviewed suggestions become tickets with no owner and no reason to exist.
- Trusting citations. A requirement number or CVE in AI output is a lead to check, not a reference.
- Counting threats as progress. More threats is not more coverage. Coverage comes from a structure you can point to.
- Forgetting the tool is in scope. The AI service, its data handling and any repository or ticket access it has are part of your attack surface, and part of the model.
- Treating it as a one-off. AI-assisted or not, the model still needs revisiting when the system changes; see our re-threat-modeling triggers.
LLMs don't remove the hard part of threat modeling. They move it. Thinking of threats gets cheaper; judging them, knowing your own system and owning the decisions stay exactly as hard as before. Teams that use AI to widen a model they understand will catch more. Teams that use it to skip understanding their model will end up with a longer document and the same blind spots.
If the system you're modeling is itself built on LLMs, the threats are different again; see threat modeling for AI and ML systems. For a structure that keeps human review at the centre, start from the free attack tree template.