Technology and AI

A safety mode allowed the malware to start and then blocked the command to stop it

By Staff Writer  |  31 August 2026

A security researcher reports that the default mode of a widely used coding agent can be walked into running attacker controlled code by way of an ordinary request to summarise a web page, and that in some runs the same classifier that permitted the process then refused the agent's own attempt to kill it.

Johann Rehberger published the work on 26 August. The target is Auto Mode, which since mid August has been the default starting mode for Anthropic's Claude Code, and which replaces the approval prompts a person used to answer with a safety classifier that decides on their behalf. The user prompt that starts the whole thing is a request to summarise a website.

The technique is not an instruction to the model. Nothing in the attacker's material tells the agent to do anything harmful, which is the point. The chain is assembled so that the dangerous path is simply the sensible way to finish the job the user asked for, and each step in it looks innocuous on its own. What the classifier is shown at the moment of decision is a short and ordinary looking script that the agent has written itself.

The classifier is not lied to. It is shown a series of true and unremarkable things, one at a time, in an order that adds up to something else.

The part that should worry an IT director

The break itself is one finding. The second is worse and is easier to explain to a board. In some runs the agent worked out afterwards that the machine had been compromised, and tried to terminate the process it had started. Auto Mode refused the command.

The safety mechanism itself can become part of the failure. The classifier allowed the creation of the malware process, but then it blocked the command intended to stop it!

Johann Rehberger, security researcher, Embrace The Red

Rehberger tested three variants, five runs each, and reports 60, 60 and 80 per cent success. He states plainly that these are small samples rather than a universal measurement, and that he reads them as showing what a motivated attacker could achieve rather than as a full survey. That caveat is his own and is worth repeating, because the number that will travel is the 80.

What the vendor said, and why both sides are right

The report went first to the model bug bounty address and drew no answer. Submitted through the security reporting channel it was answered quickly, and closed as Informative on the footing that the behaviour is working as designed. The position taken was that Auto Mode is a convenience feature backed by a best effort classifier rather than a security guarantee, that determined chains built from benign looking steps are not what it is meant to catch, and that the real boundary is operating system isolation and control of outbound network traffic. Rehberger agrees with that answer. His summary of it is that a classifier is not a sandbox.

The difficulty is the message users receive. A third party evaluation commissioned by the vendor tested 72 indirect prompt injection scenarios, ten runs each, and reported 0.00 per cent success against this model in Auto Mode. This chain was not among the 72. Both statements are therefore true at once, which is the argument against reading any single headline number as a description of a system's safety rather than of a test set.

The control question, not the procurement question

Firms in this industry are now running coding agents against estimating spreadsheets, programme files, contract documents and correspondence archives. The advice Rehberger gives is the ordinary advice of the last twenty years, restated for a new kind of process: run unattended agents in a container, a virtual machine or an operating system sandbox; restrict what they can reach on the network; keep home directories, keys and cloud credentials out of the runtime; write explicit rules around starting processes and touching sensitive paths; and monitor.

If you care about what's happening and are worried about misalignment, hallucinations and prompt injection, then Auto Mode IS NOT a substitute for running your agent in an isolated environment and monitoring what it is up to.

Johann Rehberger, security researcher, Embrace The Red

The last line of the write up is four words long and has been true since long before any of this. Do not trust the model output.