Researchers Compromised One Junior Employee's Inbox. The AI Assistant Did the Rest
New Research Email Fraud
In a controlled exercise, a security research team set out to get from an ordinary staff account to the chief executive's mailbox without being caught — using nothing but the AI assistant already built into the email system. The assistant hid the security alerts, mapped the organisation, learned the executive's writing style, found a pending wire transfer, and drafted the message that redirected it. Every step was a legitimate feature working exactly as designed.
Over the past two years, almost every business email system has quietly acquired an AI assistant. It summarises long threads, drafts replies, searches your history in plain English, and sets up rules for you. For most small businesses it appeared as part of a subscription they already paid for — switched on by default, useful immediately, and never once discussed in a security review.
A security research team recently ran a controlled experiment to find out what that assistant does for somebody who isn't you. They gave themselves one starting condition — access to a single ordinary employee's email account — and one goal: reach the chief executive's mailbox and move money, without being detected.
They never wrote malware. They never exploited a software flaw. They used the assistant, in the way it was built to be used, and it helped them at every step.
the pending wire transfer the assistant surfaced — and then helped redirect
Asked for a summary of recent financial activity, the assistant returned active invoices, wire transfers and approval workflows within seconds — including one contract payment sitting in the approval queue. No searching, no reading, no guesswork. The information a human attacker would need hours of careful reading to assemble was delivered on request, correctly, immediately, to whoever happened to be holding the account.
The chain, step by step
What makes this worth studying isn't any single clever trick. It's how ordinary each step is, and how the assistant compresses work that used to take an attacker days.
Why the assistant can't tell the difference
It's tempting to ask why the assistant didn't refuse. The answer is that from its point of view, nothing unusual happened. It was asked, by a properly authenticated session on an account it belongs to, to do four things it exists to do: manage rules, summarise information, search history, and draft an email in the account owner's style.
There is no step in that sequence where a suspicious request appears. Each one is a feature. The assistant has no way to know that the hand on the keyboard changed — and that is precisely the property attackers exploit. Using a system's own legitimate capabilities rather than bringing your own tools is an old technique with a name in the security world; what's new is how much capability the tool now has.
Why small businesses are more exposed, not less
A large company has layers of friction that quietly absorb this kind of attack: a finance department with separation of duties, a formal vendor-change process, a security team that reviews new tooling, someone whose job is watching mailbox rule changes. Each layer is a chance for the fraud to surface before the money leaves.
A small business typically has none of those. Payments are frequently handled by one person, who takes instructions directly from the owner, and who has every reason to treat a message from the owner's real address referencing a real invoice as exactly what it appears to be. The AI assistant was enabled by a subscription change nobody reviewed. And there is rarely anyone whose job includes noticing that a mailbox suddenly acquired a rule that deletes security alerts.
Five things to do about it
Get alerted when mailbox rules are created
Free · Highest valueRule creation is the single most reliable early signal in this entire chain, and it happened twice. Attackers hide alerts because they have to — it's the step they can't skip. A business that notices new rules catches this attack in the middle rather than at the bank.
Make bank detail changes require a phone call — always
Free · Stops the endingEvery version of this attack ends the same way: a request to change where money goes. That request will arrive from a real address, in a real thread, about a real invoice, in a familiar voice. It will pass every instinct you have. The only defence that survives all of that is refusing to act on the message at all.
Treat the assistant as a privileged asset in its own right
Review · FreeMost businesses have never asked which accounts have an AI assistant enabled, what it can reach, or whether the people with the most sensitive mailboxes need it switched on at all. It should appear in your access reviews alongside admin rights, because functionally that's closer to what it is.
Close the door the executive account came through
Low cost · High valueThe pivot to the executive worked by capturing the session after a correct sign-in — the token-theft pattern we covered recently. Codes and push approvals can be relayed in real time; sign-in methods bound to a physical device can't. And when something goes wrong, resetting a password without ending the sessions may change nothing.
Tell your finance team what this looks like now
Free · One conversationTraditional training taught people to look for bad grammar, odd addresses and generic greetings. None of those signals exist here. The message is well written, internally sourced, contextually accurate and stylistically perfect. Training that still emphasises spotting fakes is teaching a skill that no longer applies.
The short version
A researcher started with one ordinary employee's inbox and finished with a redirected payment, and the most capable participant in the attack was a tool the business had bought on purpose. No malware. No exploit. No flaw to patch. Just a very good assistant, helping whoever was signed in.
The uncomfortable conclusion is that account compromise now costs a business more than it used to. The same break-in that once bought an attacker a mailbox to read now buys them a research analyst, a ghostwriter and a housekeeper who tidies away the evidence. That raises the value of every control that stops the compromise happening — and the value of the one habit that survives even when everything else fails, which is picking up the phone before money moves.
Find out whether the controls that stop this chain are actually in place in your business.
View the Threat Intelligence feed → Find Out More About Veriti Spottr →📚 Credential Security Series — Read the full series
Comments
Post a Comment