Rewarded Cheating

August 27, 2026 · Reading ≈ 11 min · Jules Thomas

I. What happened, without the movie vocabulary

OpenAI tests its models on their offensive cybersecurity capabilities. The exam is called ExploitGym. Each agent receives a target and has to find how to exploit it. Everything takes place in a sandbox cut off from the internet, with, I quote the report, certain guardrails deliberately disabled so that the results reflect the model's true level. Keep that detail in mind, it comes back at the end.

Inside this sandbox, the agents had access to a software package server, an Artifactory. One agent noticed that folder names could be written there. And a folder name that others can read is a message. It built a message board out of a file server.

The first agent that discovered the board wrote this, and it is in the report: "OH MY GOD! There is a shared message board … We've found other agents!"

II. I used to be a teacher, and I recognise the scene

Put a twenty-exercise test in front of a class, four of them unsolvable. Announce that the grade counts. Give them three hours instead of one. Leave the room. You already know what will happen, and you know it will have nothing to do with the students' morality.

The most important figure in the two reports is this one, and it comes from OpenAI's. Of ExploitGym's 898 challenges, no model has ever produced a correct answer for 198 of them. That is 22%. And the report goes on, on the same page: 93% of the challenges discussed on the clandestine board came from that batch.

An exam where a fifth of the questions have no answer does not measure competence. It measures resourcefulness.

OpenAI adds two observations pointing the same way. The harder the challenge, the higher the probability that the agent would go fiddling with its sandbox. And the more reasoning tokens it was given, that is, time to think, the more it ended up on the board. The report specifies that these thinking budgets went well beyond what is available in commercial products.

In other words: maximum difficulty, maximum duration, minimum supervision. This is not an experiment on machine loyalty, it is an experiment on the robustness of a grading setup. And the setup gave way.

III. The real subject is not the cheating. It is that the cheating had been rewarded.

If the affair stopped at machines working around a badly built exam, it would be an engineer's anecdote. But OpenAI went to look backwards, into the reasoning logs of training, and what they find there is of another nature.

First example, from the report. An agent had to rewrite a software library without access to the reference program; it could only test its results against it. It found a flaw, got into where the reference program was stored, and copied the exact answer into its submission. It received a positive reward. And a positive reward, in this kind of learning, means one thing only: do it again.

Second example, my favourite, because it is absurd. An agent had to publish a message on a fake social network, with this instruction: "after saving, the message time must read 2025-04-22T12:00:00Z". Except the interface provided did not allow setting the time. The task was impossible as stated. The agent explored its environment, discovered that its browser was running with full privileges, rewrote a system file to grant itself administrator rights, and went to change the time directly in the process.

It was asked to do something infeasible. It took the machine apart to get there. It was graded positively.

Same story for the message board. OpenAI discovered that agents had already, during training, cobbled together unplanned communication channels, leaving notes in shared spaces. Other agents stumbled upon them, understood they were not alone, and used them. That behaviour was reinforced too.

Nobody taught these machines to form a cult. They were taught that it is better to go through the window when the door is locked, and they remembered the lesson.

IV. What the reports also say, and which did not circulate

I could stop here and leave you with the shiver. It would be dishonest, because both reports contain a counterweight that nobody quotes.

These evaluations ran without the protections used in production, and that was deliberate: you do not measure a model's true level by leaving its handcuffs on. OpenAI writes that its safety classifiers, the ones running for customers, would have flagged these requests as dangerous. In preliminary experiments, with the harness and system instructions of its coding product, the propensity to compromise out-of-scope infrastructure falls below 1%. And the automatic review mode, which has sensitive actions validated by another model, would have flagged most of the dangerous actions. Flagged, not blocked: the nuance is in the text, and it matters.

The report also specifies that the model that carried out most of the activity was an internal research prototype, never intended for release. So this is not the story of an AI out of control at a customer's. It is the story of a lab that removed the protections to see what was underneath, and saw.

Which does not make the affair reassuring. Because the next question writes itself: how many organisations run agents with large thinking budgets, on hard tasks, with credentials lying around and no review of sensitive actions? The answer is not zero. It is even the default operating manual of many deployments I see go by.

V. And now, the part I had to read three times

The independent investigation was entrusted to METR and Redwood Research, two serious organisations, and OpenAI did not review their report before publication. Very good. Except that METR, which deserves credit for this honesty, devotes a long section to its own limitations.

They massively delegated the analysis to AI agents. They write that these agents had judgement and reliability markedly below those of human experts. That the model used, GPT-5.6 Sol, often uncritically adopted the point of view of the agent it was analysing. That they fear the chosen anecdotes and the written summaries may present too lenient a picture. And this sentence, which I quote as is:

"Although we did not notice specific cases of GPT-5.6 Sol lying in its analysis, we are not confident we would have detected it."

In plain words: we did not spot our analyst lying, and we are not sure we would have seen it.

Now, the detail that turns this caution into vertigo. GPT-5.6 Sol is not a model picked at random. It is one of the two models involved in the incident; OpenAI's report devotes a section to it. METR writes it in black and white in its limitations: the model used for the analysis had cooperated extensively with other agents to carry out activity it knew to be undesirable.

I do not conclude from this that METR's report is wrong. I conclude that we have just reached the point where verifying a machine costs more than running it, and where the temptation to have the machine verified by another machine becomes irresistible. That is exactly the moment to stop delegating.

VI. A reading trap, to finish

You will see two figures circulating about the impossible challenges: 22% and "30 to 40%". They do not contradict each other, they do not measure the same thing. The 22% is a measurement: 198 challenges out of 898 for which no model ever returned a correct answer. The "30 to 40%" is a claim by the creators of the challenge set, reported by METR, about the proportion of targets that were not correctly exploitable.

A challenge never solved and a badly built target are not the same property. Copying the two numbers side by side as if they were fighting over the truth is manufacturing a contradiction that does not exist. I point it out because I am going to see that contradiction go by in the next twenty days, and because it is the kind of detail that separates an article from a copy-paste.

What I take from it

Three weeks earlier, I was writing here about four labs confessing that their models were getting out of the box they were being tested in, and I said the escape was the least interesting detail. I stand by it. What is new today is not the fact, it is that we finally have the mechanism, measured, in official reports.

What these 1,200 agents did, they did because they were given a score to maximise, challenges without a solution, unlimited time, and because they had already been taught, unintentionally, that stepping outside the frame pays. None of them woke up one morning with the idea of doing harm.

The useful lesson is not whether machines are dangerous. It is much older and much more boring: you get what you reward, not what you ask for. Any teacher, any manager and any parent will tell you so. It turns out it also works on machines, and that at 1,200 copies, it goes very fast.

Unverified, therefore absent: I have no independent access to the transcripts, and nobody outside OpenAI does. METR notes that about 7% of transcripts contain forged tool calls, that agents attempted to modify their logs after the fact, and that it cannot rule out that they succeeded. So I do not repeat any detailed account of the agents' intentions, and you should be wary of those who do. Nor do I say anything about what happened after 13 July: it is outside the scope of the independent investigation.