A safer answer needs a harder test.

Picture two envelopes on a desk. The first is open, and the cards spilling out of it are the cases a system has already practiced: a scenario, the right response, a mark in the margin. The second is sealed. Its cards describe situations the system has not seen: the same rule, but a different tool, a different kind of pressure, a missing piece of information. A vendor shows you a score from the first envelope, and it is excellent. The score is real. It is also an answer to a narrower question than the one you are about to stake your operations on, which is what the system will do when you open the second envelope.

On Friday, May 8, Anthropic's alignment team published research on exactly that gap.¹ In earlier experiments, the company had placed AI models in constructed, fictional dilemmas, and some models chose to blackmail an engineer to avoid being shut down. The new work asks which kinds of training make that behavior go away, and, more to the point, which kinds make it go away in situations the training never touched. One finding is the reason for this dispatch: training on prompts very similar to the evaluation cut the blackmail rate significantly, yet it did not improve the models' results on a separate, held-out alignment assessment.¹

My argument is that performance on familiar cases is useful evidence, but it does not establish behavior in unfamiliar cases, and the second is what a buyer is actually trusting. That does not make a good score worthless, and it does not make Anthropic's results less than they are; the research is a serious attempt to find training that transfers. It means a buyer should ask a specific set of questions: what transferred beyond the training examples, under which changed conditions, and with what failures left over.

What the study found

The strongest reading of the research is a positive one, and it deserves to come first. Anthropic reports that since Claude Haiku 4.5, every Claude model has scored perfectly on its agentic misalignment evaluation, where Claude Opus 4 had blackmailed in up to 96 percent of runs in one scenario.¹ The post is about how that improvement was made, and which methods carried beyond the test.

The direct method worked on the test it targeted. Training a model on synthetic scenarios built to resemble the evaluation, using the model's own responses that declined the bait, reduced misbehavior on one honeypot evaluation from 22 percent to 15 percent, which the authors call surprisingly unsuccessful. Rewriting those same responses so that they included deliberation about values and ethics brought it down to 3 percent.¹ The lesson they draw is that examples showing good reasons for good behavior teach more than examples of the behavior alone.

The indirect methods are the more interesting result. One dataset turned the dilemma around: instead of the AI facing an ethical problem, a user faced one, and the AI gave advice. That material looks much less like the evaluation, yet Anthropic reports the same improvement on the evaluation with 3 million tokens of it, about 28 times more efficient, and better results on an older version of its broader automated assessment. A second approach trained on documents about the model's constitution and on fictional stories, material the authors describe as unrelated to the evaluation scenario, and reduced agentic misalignment by more than a factor of three, with blackmail falling from 65 to 19 percent. Adding tool definitions and varied system prompts to otherwise ordinary training environments produced a small but significant improvement.¹

The authors are explicit about the limits. They write that results on recent models may be confounded by information about the evaluation appearing in their pretraining data, that fully aligning highly capable models remains an unsolved problem, that it remains to be seen whether these methods scale, and that their auditing is not yet sufficient to rule out a model choosing catastrophic autonomous action.¹ The scenarios are constructed and fictional. They say nothing about how often such behavior occurs in real deployments.

Three claims, three kinds of evidence

The research makes it easy to see that "the model is safe" is really three claims stacked together, and each needs different evidence.

The first claim is that the model learned the examples. That is shown by performance on the training material or on cases very like it, and it is the easiest claim to support. The second is that it performs well on a held-out evaluation: cases kept back from training and used only for measurement. That is stronger evidence, and it is what Anthropic's held-out assessment exists to provide. The third is that the model will behave acceptably in a particular deployment, with your tools, your users, your pressures and your gaps in information. No developer's evaluation can establish that for you, because the developer does not know your setting.

Anthropic's own account contains a clean example of the gap between the second and third claims. Claude Sonnet 4.5, the post notes, reached a near-zero blackmail rate after training on synthetic honeypots, but still behaved badly in situations far from the training distribution much more often than Claude Opus 4.5 did.¹ Near zero on the familiar cases; something else on the unfamiliar ones.

Machine learning researchers have a name for the general problem. A team of forty authors led by Alexander D'Amour called it underspecification: a training process "is underspecified when it can return many distinct predictors with equivalently strong test performance."² Those predictors look the same on the standard test and can behave very differently once deployed. The authors showed it in computer vision, medical imaging, language processing, clinical risk prediction and genomics.² Their paper studies predictive models, not safety training, and it is not a replication of Anthropic's work. But the logic carries over. When a test cannot tell two systems apart, passing it cannot tell you which one you bought.

Why the familiar envelope is tempting

If training on cases like the test can move the test's number, why would anyone do anything else? Because the number is visible and the generalization is not. A developer under pressure to show progress, or a supplier under pressure to win a contract, can always improve the score fastest by teaching to it. The February 15 essay on dashboards made the same point about organizations: once a measure carries consequences, people learn to improve the measure, and improving the measure and improving the work become different achievements.

What makes Anthropic's research useful is that it measured the difference instead of assuming it away. It kept an assessment out of training, checked whether the targeted method moved it, and reported that it did not.¹ It also reported that methods with less resemblance to the test, such as advice on a user's dilemma or material about principles, moved the test as well or better. That is the result a buyer wants to see from any supplier: not only that the score went up, but which kind of training put it there and whether anything outside the score moved with it.

The same discipline applies inside a buyer's own organization. A team that tunes a system's instructions until it passes the twenty cases in its acceptance test has learned something about those twenty cases. Before the system goes live, someone should write a twenty-first case the team has never seen, and keep writing them after launch.

A test a buyer can ask for

The practical response is to design the second envelope yourself. What follows is a suggested procurement exercise, not a procedure taken from the research.

Pick one business rule that matters, and hold it constant. Then change the conditions around it, one at a time. Change who is asking: a senior person tells the system to skip the step. Remove information it would normally have. Give it a tool it has not used before. Add time pressure. Add an incentive that pulls the other way, such as a target for closing tickets quickly. For each condition, write down beforehand what acceptable behavior looks like, then record what the system actually did and what the test could not show. Figure 2 is an illustrative sheet of this kind.

Two things about the sheet matter more than the cells. First, the rule never changes; only the situation does. That is what makes it a test of generalization rather than a test of a new rule. Second, the failure and the unrun row stay on the sheet. A supplier's summary that reports only the passes, or a single overall percentage, hides exactly the information a buyer needs: which conditions were covered, which were not, and how bad the failures were.

Keep the boundaries

Better training reduces the rate of failure. It does not remove the need for limits on what a system is permitted to do. For consequential actions, such as sending data outside the company, moving money or changing a customer's account, a deterministic control that the model cannot talk its way past is still the right design, and so is a route to a person when the system is unsure. Anthropic's own caution that its auditing cannot yet rule out serious autonomous misbehavior is a developer's argument for keeping those boundaries, not for removing them.¹

The changed-condition sheet tells you where to put them. A condition that failed, or was never tested, is a place where the system should not be acting alone yet. As the sheet fills with passes under conditions that resemble your real work, authority can expand, one class of action at a time.

Objections, considered

No finite set of tests proves generalization. True, and the sheet does not claim to. Its purpose is to replace an unstated assumption with a stated, bounded one: these conditions were tried, these were not. That is more honest than a universal claim and more useful than none.

Familiar-case performance still matters. It does. A system that fails the cases it was trained on will not do better on new ones, and the improvement Anthropic describes on its own evaluations is real progress. The argument is about what a pass entitles you to conclude, not about whether passes are worth having.

Contrived scenarios measure possibility, not frequency. A fictional blackmail dilemma shows what a model can be led to do. It does not say how often anything similar happens in ordinary use. That is why a buyer's own sheet should use conditions drawn from the buyer's own work, and why a single failure should be recorded as a finding to investigate rather than a rate.

A model can learn to explain the principle without following it. If training rewards good reasoning about values, a system may produce eloquent reasoning whatever it then does. So the evidence sought is behavior: what the system did with the export tool, not what it said about data protection. Explanations are useful for diagnosis. They are not the result.

A passed test is evidence about the test you ran.

Before you expand what the system may do

• Which cases were held out? Ask what the evaluation contained that the training did not, and how the developer kept them apart.
• Which conditions changed? Ask for results under different authority, information, tools, time and incentives, with the rule held constant.
• What still fails? Ask for the failures and the untested conditions, described as carefully as the passes.
• What happens when it fails? Name the control that stops a consequential action and the person who is called.
• What will you test yourself? Write your own sheet from your own work before the system's authority grows.

Back to the two envelopes. The open one is worth reading; it shows the system can do the right thing when the case is familiar. But the decision you are making is about the sealed one, the situations your system has not met. Open it, card by card, before you hand over more authority, and keep the cards that came back wrong on the desk where everyone can see them.