AI shows up in a threat model in two ways. One is the AI in your own product: models, agents, retrieval and tools, which can be attacked. We've covered that in threat modeling for AI agents and MCP servers. The other is the AI your attackers use against everything else you run: your helpdesk, your login page, your email domain, your patch cycle. Systems with no AI in them at all are now attacked with it.
This post is about the second kind. It takes the Invoicely model from our threat model example, a small invoicing SaaS with ten scored threats, and updates it for an attacker who uses AI. Four existing risks move. Five new ones appear, and one of them tops the register. A handful of controls that looked fine turn out to have depended on attacks being expensive.
The short version: you don't need a new method or a separate "AI threat model". AI rarely invents an attack. It makes the ones you already model cheaper per target, faster and better at imitating people. So do three things to the model you have. Re-score the threats those three changes touch, mostly by raising likelihood. Add the few leaves that are new: cloned voices and deepfakes, brand impersonation that reads perfectly, and coding assistants that install packages that don't exist. Replace controls that only worked because attacks took effort, such as recognising a voice, spotting clumsy wording, CAPTCHAs, monthly patching and next-day alert review. Controls that don't care how much effort the attacker spent still hold: phishing-resistant MFA, call-backs to a known number, server-side authorization checks, signed webhooks and pinned dependencies.
What attackers are actually doing with AI
It helps to separate what has been seen from what is speculated. These are public cases and measurements, as of 28 October 2026:
- Deepfake video call, $25 million (2024). A finance employee at the engineering firm Arup in Hong Kong joined a video call with what looked like the CFO and several colleagues. All of them were deepfakes. He made 15 transfers totalling about US$25 million. CNN via WRAL
- AI-generated voice messages impersonating officials (2025). The FBI warned that attackers were sending text and AI-generated voice messages claiming to come from senior US officials, to build trust before asking for access to personal accounts. FBI IC3
- Malware that asks a model to rewrite itself (2025). Google's Threat Intelligence Group described PROMPTFLUX, a dropper that queries the Gemini API for new obfuscated versions of its own code. Google assessed it as experimental, but it was the first malware seen calling an LLM while it runs. Google GTIG
- An intrusion campaign largely run by an AI agent (2025). Anthropic reported a state-sponsored group that used Claude Code against about 30 organisations, succeeding against a few. The AI did an estimated 80 to 90% of the work, including reconnaissance, exploitation and data collection, with people stepping in at a handful of decision points. Anthropic
- Less time to patch. VulnCheck found that 32.1% of the vulnerabilities exploited in the first half of 2025 were exploited on or before the day their CVE was published. VulnCheck
- Packages that don't exist. A USENIX Security 2025 study of 16 code-generating models found that about one in five recommended packages didn't exist, and many of the invented names came back again and again. Attackers can register those names. This has been called "slopsquatting". Spracklen et al.
Not every exploit in those statistics was written with AI. The pattern across all of them is still the same: things that used to take skill, time or a native speaker now take much less. That's a change in likelihood, which is exactly what a threat model is built to absorb.
Step 1: Update the attacker, not the method
Every threat model assumes an attacker, even if it never writes one down. Write it down now, because that assumption is what has changed. For Invoicely, the realistic attacker is a financially motivated group, not a nation state. Here's what AI gives that group:
| Capability | Before | With AI | Threats it touches |
|---|---|---|---|
| Writing | Generic phishing, often with tell-tale mistakes. Targeted messages cost time per victim. | Fluent, personal messages in any language, for almost nothing per target. | Phishing of staff and customers, brand impersonation |
| Imitation | Impersonating someone by phone meant finding a convincing person. | A voice cloned from a short recording, and real-time video deepfakes. | Helpdesk resets, payment approvals, identity checks |
| Reading code and advisories | Turning a patch into an exploit took a skilled person days or weeks. | Models help diff patches, review code and draft exploits. | Unpatched dependencies, leaked source, app binaries |
| Scale and patience | Testing every endpoint or ID by hand was tedious, so people didn't. | Agents try thousands of variations and vary their traffic like people do. | Authorization flaws, credential stuffing, scraping |
Two things are deliberately missing. AI doesn't give the attacker your signing keys, and it doesn't break cryptography that's correctly used. Threats that depend on a secret the attacker doesn't have barely move. That's what keeps the re-scoring honest.
Step 2: Re-score what you already have
Go through the existing risk register and ask, for each threat: does one of those four capabilities make it cheaper, faster or more convincing? If so, raise the likelihood. If not, leave it alone and say why. Invoicely's register uses the same 5×5 risk matrix as the original model:
| # | Threat | Before | With AI | Why |
|---|---|---|---|---|
| T2 | IDOR on invoice endpoints | 4 × 5 = 20 | 5 × 5 = 25 | An agent reads the web app's JavaScript, finds every endpoint and tries other customers' IDs on all of them in an afternoon. |
| T1 | Credential stuffing on login | 4 × 4 = 16 | 5 × 4 = 20 | The login's CAPTCHA is solved automatically, and the traffic no longer looks like a bot to an IP-based limit. |
| T5 | Admin console's shared database role | 3 × 5 = 15 | 4 × 5 = 20 | Support agents are the easiest people to phish or call, and a compromised agent account reaches every customer. |
| T7 | Stored XSS in invoice notes | 2 × 4 = 8 | 3 × 4 = 12 | Automated review of the public app finds unescaped output faster than before. |
| T4 | Forged payment webhook | 2 × 5 = 10 | 2 × 5 = 10 | Unchanged: forging one still needs Stripe's signing secret. |
| T3, T10 | Open PDF links, mistyped email | 10, 6 | 10, 6 | Unchanged: these depend on a leaked link or a typo, not on attacker effort. |
| T8, T9, T6 | Session token, render queue, audit log | 12, 9, 6 | 12, 9, 6 | Unchanged. T8 rises only if T7 does, and the fix (an httpOnly cookie) already breaks that path. |
Four of ten move, and not one of them because of a new kind of attack. The IDOR (T2) was already Invoicely's top risk, and now it scores the maximum. That's typical: AI widens the gap between what's fixed and what isn't, and it punishes the flaws you knew about but deferred most.
Step 3: Add the leaves that are genuinely new
A few threats weren't realistic for a small company's attackers before, and now are. For Invoicely, the outcome that matters is money: a client paying an invoice to the wrong bank account. The attack tree for that outcome has three ways in, and the root is an OR gate because any one of them is enough:
Impersonating Invoicely is the leaf that tops the tree, and it's the one Invoicely hadn't modelled at all. Fake invoice emails are an old fraud. What's new is that a fake can now copy Invoicely's real emails exactly, word for word, in the client's language, with the right customer name and amount. If Invoicely's sending domain doesn't enforce DMARC, the fake can even come from it.
Taking over the account has two new routes in. One is the CAPTCHA that no longer stops credential stuffing. The other is a cloned voice on a support call, asking to have 2FA removed because "I've lost my phone". Notice the AND gate: either route only matters if the attacker can then change the payout bank details without anyone noticing. That leaf is where Invoicely can break the whole branch.
Impersonating the customer to their own client's finance team, with a deepfake call asking them to pay a new account, is real and serious. It's also entirely outside Invoicely's systems. The model records it as an assumption ("clients may be targeted directly") and as a reason to make Invoicely's own payment links the obvious, trusted way to pay.
Two more threats sit outside this tree but belong in the register. One is a coding assistant in Invoicely's own development that suggests an npm package which doesn't exist, under a name an attacker has registered. The other is an internet-facing dependency exploited before the monthly patch window closes.
Step 4: Find the controls that relied on effort
This is the step most teams skip, and it's where the real work is. Some controls never stopped an attack outright. They made it expensive enough that attackers went elsewhere. Look for them in your model:
| Control that relied on effort | Why it weakens | Replace or back it with |
|---|---|---|
| "We know our customers' voices" on support calls | Voices can be cloned from a short recording | No account or 2FA change by phone alone. Use recovery codes, a confirmation in the product or a call-back to a number on file, then notify the owner. |
| Phishing training based on spotting mistakes | AI-written messages don't have them | Phishing-resistant MFA (passkeys or security keys) for staff, and out-of-band checks on any request to move money or data. |
| CAPTCHA on login and sign-up | Solved automatically | Limits per account and device, breached-password checks, 2FA and risk-based step-up. |
| Rate limits keyed only to IP | Traffic spread and varied like real users | Limits per account and per business action, plus alerts on unusual volume. |
| Monthly patching for internet-facing services | Exploits can exist on disclosure day | An emergency path to patch within 48 hours, or a WAF rule or feature switch-off as a stopgap. |
| Next-business-day alert review | An automated intrusion moves in hours | Automatic containment for high-confidence alerts, and time to respond as a tracked measure. |
| "Obscure" endpoints and logic in the client | Code and binaries are explained in minutes | Treat everything shipped to a browser or phone as public, and enforce checks on the server. |
Note what's not on the list. Server-side authorization, signed webhooks, short-lived links, pinned dependencies and phishing-resistant MFA don't care how clever or patient the attacker is. Effort-independent controls like those are the ones to invest in now.
Step 5: The updated register
Here are the new rows, with IDs A1 to A5 following on from T1 to T10. The re-scored T1, T2, T5 and T7 keep their original treatments, which are now more urgent.
| # | Threat | L × I | Treatment | Owner |
|---|---|---|---|---|
| A1 | AI-written fake invoices impersonate Invoicely to clients | 4 × 5 = 20 | Mitigate: DMARC at p=reject on every sending domain. Invoice emails never contain bank details, only a link to pay through Invoicely. Tell clients on every invoice that bank details never change by email. Monitor look-alike domains. | Platform + customer success |
| A2 | Payout bank details changed quietly after an account takeover | 3 × 5 = 15 | Mitigate: changing payout details needs re-authentication and 2FA, sends an email to every account admin, and holds payouts to the new account for 48 hours. This breaks the middle branch whatever the way in. | Engineering lead |
| A3 | Cloned-voice caller gets support to remove 2FA | 3 × 5 = 15 | Mitigate: support can't remove 2FA. Recovery is self-service with recovery codes, or confirmed by another admin of the account in the product, with a 24-hour delay and a notification. | Head of support |
| A4 | Internet-facing dependency exploited before the monthly patch | 3 × 5 = 15 | Mitigate: an emergency patch path within 48 hours for critical, internet-facing vulnerabilities, with a WAF rule as a stopgap. Dependency alerts go to the on-call engineer, not a weekly digest. | Platform |
| A5 | A coding assistant installs a hallucinated npm package | 2 × 5 = 10 | Mitigate: lockfiles and npm ci in builds, installs through a registry proxy, and a review of any new dependency before it's merged. | Engineering lead |
Read together, the new top of the register is T2 (25), then A1, T1 and T5 (20 each). The honest note belongs on A1: even with DMARC, a look-alike domain can still send a convincing fake. Invoicely accepts that residual risk, and the controls that make it acceptable are pay-by-link and the warning on every invoice. The register entry should say that, and name the trigger to revisit it: an increase in look-alike registrations, or a client reporting a fake.
Map it so the next review can find it
When the next person reviews this model, they should be able to tell which leaves assume an AI-assisted attacker. MITRE ATT&CK gives you a tag for that: T1588.007, Obtain Capabilities: Artificial Intelligence. Put it on the leaf next to the technique the AI speeds up, for example T1566.004 (Spearphishing Voice) and T1656 (Impersonation) on A3, or T1190 (Exploit Public-Facing Application) on A4. Don't reach for MITRE ATLAS here. ATLAS covers attacks on AI systems, not attackers who use AI.
What threat modeling can't do here
- It can't detect a deepfake. It tells you which decisions rely on recognising a person, so you can stop relying on it.
- It can't predict the next capability. The scores above reflect what's public as of October 2026. Re-score when a credible report shows something new working against systems like yours.
- It can't make your clients careful. The deepfake call to a client's finance team stays outside Invoicely's control. The model can only make the safe way to pay the easy way.
- It doesn't replace testing. If attackers test your authorization with agents, so should you. The model tells you where to point the tests.
A checklist for your own model
- Write down the attacker you assume, and what AI gives them: writing, imitation, reading code, scale.
- Go through every threat in the register and ask whether one of those capabilities makes it cheaper, faster or more convincing.
- Raise likelihood where it does, and record why the rest don't move.
- Find every decision that relies on recognising a person by voice, face or writing style.
- Add leaves for impersonation of your brand, your staff and your executives.
- Check whether any account-takeover branch ends in a change that's quiet and instant, and add a step-up or a hold.
- List controls that only add effort (CAPTCHAs, IP limits, obscurity, monthly patching) and back each with one that doesn't.
- Add the build-side threats: hallucinated packages and leaked code.
- Tag AI-assisted leaves with ATT&CK T1588.007 so the next review can find them.
- Set the trigger to re-score: a new public capability, or a changed control.
Sources, as of 28 October 2026: the incident reports and studies linked above, and MITRE ATT&CK's T1588.007 and T1566.004. Statistics describe attacks seen in the wild generally, not attacks proven to use AI.
Frequently asked questions
Do AI-assisted attackers need a new threat modeling method?
No. STRIDE, attack trees and a scored risk register work as before. What changes is the attacker you assume: one who can write fluent messages for free, imitate a voice or a face, read code and advisories at machine speed, and run many attempts at once. Update the attacker assumptions, re-score the threats those capabilities touch, add the few leaves that are genuinely new and replace controls that only worked because attacks took effort.
Which threats does AI change the most?
Impersonation and social engineering (phishing, helpdesk calls, executive deepfakes), the time between a vulnerability being disclosed and being exploited, automated abuse that used to be stopped by CAPTCHAs or IP rate limits, and authorization flaws that an agent can test across thousands of requests. Threats that need a secret, a signing key or physical access move much less.
How do I defend a helpdesk against cloned voices?
Stop treating the caller's voice, caller ID or personal details as proof of identity. Route MFA and password resets through something the attacker can't fake on a call: recovery codes, a reset approved by the person's manager or an existing admin in the product, a call-back to a number already on file, or an ID check against records. Then notify the account owner and add a delay before the reset takes effect for privileged accounts.
Is MITRE ATLAS the framework for AI-assisted attackers?
No. MITRE ATLAS catalogues attacks on AI systems, such as prompt injection or model poisoning. Attackers using AI against ordinary systems are covered by MITRE ATT&CK, under technique T1588.007 (Obtain Capabilities: Artificial Intelligence), alongside the technique the AI helps with, such as T1566 Phishing or T1190 Exploit Public-Facing Application.
How often should I re-score for AI-assisted attackers?
Re-score once now, then at your normal review cadence, and again when a public report shows a new AI capability being used against systems like yours, such as working real-time voice deepfakes or agents running intrusions. Those are threat-landscape changes, one of the standard triggers to re-threat model.