The second attempt must remember the first.

Two strips of printer paper lie on a desk in the back office of an online stationery shop. The shop, the desk and the order are invented. Both strips are payment requests that the shop's checkout sent to its card processor for the same order: two notebooks and a fountain pen, $48.60, order 5531. The first strip says it was sent at 14:02:07. The second says 14:02:38. In between, the checkout waited thirty seconds for an answer, heard nothing, and tried again. The customer pressed Pay once. How many times should the card be charged?

Everyone says once, and everyone is right. The trouble is that the processor never saw the customer press anything. It saw two requests. Whether it charges once depends on whether it can tell that the second request is the first instruction arriving again rather than a new one. My argument is that a reliable system has to keep apart two things that look identical on the wire: delivery attempts, the messages one system sends another, which can be lost, delayed or repeated, and the intended business effect, the one charge, booking or shipment that someone actually asked for. Trying again is the right response to unreliable delivery, but it is safe only when every attempt carries a shared identity for the intent and the receiving system durably remembers what it did the first time.

None of this is new, and it is not a problem that artificial intelligence invented. Engineers at payment companies and cloud providers wrote it down years ago, and the technique has a name, idempotency, which I will come to once the problem is clear. Nor is the claim that a careful design makes every action happen exactly once everywhere. The claim is narrower. When the outcome of a request is unknown, a retry is only as safe as the design that recognizes it, and that design should say plainly where its recognition ends.

The answer that never arrived

Follow the first request. The checkout sends it over the network to the processor. Three different things can now go wrong. The request can be lost before it arrives, in which case nothing happened. The processor can receive it, charge the card, record the charge, and send back a reply that is lost on the way home. Or the processor can simply be slow, so that the reply is still traveling when the checkout gives up waiting.

From where the checkout sits, all three look the same: silence. The first case calls for trying again. The second calls for doing nothing, because the money has already moved. The third calls for patience. The checkout cannot tell them apart, because the only evidence that could separate them is the reply it did not receive. A lost response does not show that the action failed. It shows only that the caller does not know.

Brandur Leach made this distinction the starting point of a 2017 post on the Stripe engineering blog. He separated failures before the server does anything, failures partway through, and the case in which the operation succeeded but the client never learned the result.¹ That last case is the one that produces double charges, and it is not exotic. It needs nothing more than a connection that drops at the wrong moment.

Why trying again is right, and dangerous

Retrying is the correct instinct. Most network failures are brief, and a request that fails once will usually succeed a moment later. Leach recommended that clients wait longer after each failure, doubling the delay each time, and add a little random variation to each wait so that thousands of clients recovering from the same outage do not all knock on the server's door at the same instant.¹ A system that never retries has its own failure mode: the customer believes they have paid, the shop does not know whether they have, and the order sits in limbo until a person sorts it out.

But a naive retry converts uncertainty into duplication. If the processor treats the 14:02:38 request as an ordinary new request, it charges the card a second time, and the shop now owes the customer a refund and an apology. Malcolm Featonby, a principal engineer at Amazon Web Services, described exactly this sequence in a 2021 article for the Amazon Builders' Library: the request succeeds, the response never makes it back, and the caller, reasonably, asks again.² He also considered the obvious patch, having the service answer the repeat with an error saying the resource already exists, and rejected it because it leaves the caller unsure what happened. An error that might mean success is not much of an answer.

Some operations are naturally safe to repeat. Reading a balance twice changes nothing. Setting a delivery address to a particular value twice leaves the same address. The HTTP standard records this property for its methods: repeating a GET, PUT or DELETE is meant to have the same effect on the server as sending it once, while POST, the usual way to create something new, carries no such promise.³ A charge is like a POST. Taken by itself, each request means "do this," and two of them, taken literally, mean "do this twice."

Give the intent a name

The repair is to give the intent an identity of its own, separate from any one message. Before the checkout sends its first attempt, it generates a unique value for this particular purchase, here K7Q4, and attaches it to every attempt it makes. Leach described the scheme directly: the client generates a unique ID for the operation, retries with the same ID, and if the operation had already succeeded, the server replies with the stored result of that success instead of performing it again.¹ Amazon's services use the same idea under a different name. Featonby describes a unique, caller-provided client request identifier, which EC2 calls a ClientToken; the AWS software development kits generate it and reuse it on retry.² The practice is common enough that an IETF working group has a draft standard for carrying it in an Idempotency-Key header. As of its October 2025 revision, that draft was still a proposal, not a standard.⁴

An operation that has the same effect however many times it is repeated is called idempotent, and the identifier is what makes a charge behave that way. The word matters less than what the identifier stands for. It names the customer's decision, not the bytes in the request.

That difference is not pedantic. Why not just compare the contents of the two requests and refuse the second if they match? Because identical contents do not mean identical intent. Later that afternoon the same customer comes back and buys the same pen and notebooks again, for a friend: same card, same amount, same items. A rule that suppressed look-alike charges would refuse a legitimate sale. The reverse problem exists too. If a request arrives carrying K7Q4 but a different amount, something has gone wrong, and the IETF draft says a key must not be reused for a request with a different payload; it recommends that the server refuse such a request rather than guess.⁴

One consequence follows for the caller. The identifier has to be created where the intent is born, when the customer presses Pay or when the order is created, and stored with the order so that every later attempt can find it. A checkout that mints a fresh key for each attempt has defeated the scheme while appearing to follow it.

What the receiving service owes

An identifier is only a label. The safety comes from what the receiving service does with it, and that is where designs most often go wrong. The service has to record the identifier and the effect together, so that either both exist or neither does. Featonby is explicit that the server-side operation must be all or nothing, and he names the two ways it fails: the token recorded but the resources never created, or the resources created but the token never recorded.²

Each failure is easy to picture. Suppose the processor charges the card and then writes K7Q4 into a log file, and the machine crashes between the two steps. The charge exists, the memory of it does not, and the retry charges again. Now reverse the order: the processor writes K7Q4, then crashes before charging. The retry finds the key, reports success, and the shop ships goods that were never paid for. A token written casually to a log, beside the real work rather than with it, gives the appearance of safety without the substance.

Three further obligations follow. The first is to remember the outcome, not just the fact that a request arrived: the draft standard says a retry of a completed request should receive the result of the earlier operation, whether that was success or an error.⁴ The second is to handle a retry that arrives while the first attempt is still being processed; the draft recommends answering it with a conflict status rather than starting the work a second time.⁴ The third is to keep the promise even when a request arrives very late. Featonby argues that a service should honor the original contract in that case, applying what he calls the principle of least astonishment.²

There is one more obligation, and it is easy to forget because it concerns time. No service remembers every key forever. The draft standard allows a service to expire keys and says it should publish its expiration policy.⁴ That retention window is part of the contract. A retry inside the window is recognized; a retry after it is treated as a new instruction. So the caller's retry schedule has to fit inside the service's memory, and someone has to know both numbers.

The figure below runs the failure in slow motion. Step through it, and switch between a processor that keeps intent K7Q4 with the charge and one that does not.

Where the guarantee stops

Follow the order past the processor. Once the charge succeeds, the shop reserves stock in its warehouse system, sends a confirmation email, and books a courier. Each of those is a separate system with its own memory, and each has the same exposure: a request sent, a reply lost, a retry. The processor's promise covers the charge. It says nothing about the courier.

Inside a single database, the shop can record the order, its intent key and its status in one transaction, so that they succeed or fail together. That guarantee ends at the edge of the database. It cannot reach into the processor's ledger or the courier's booking system. For each step that leaves the building, there are two honest options. The first is to carry an intent identity across the boundary, derived from the order (5531, payment; 5531, courier) so that each downstream service can recognize its own repeats, provided it supports that. The second, when it does not, is to reconcile after the fact: compare the charges against the orders, the bookings against the shipments, and refund or cancel what does not match. Reconciliation and compensation are not admissions of failure. They are what a design should promise at the places where recognition is not available.

Automated workflows and software agents make this old question more frequent, not different. A workflow engine that restarts a failed step, or an agent that calls a tool, waits, and calls it again, is the checkout at 14:02:38. I know of no good measurement of how much more often such systems retry than people do, and I will not invent one. The question to ask of them is the same one: when the second call arrives, does the system receiving it know that it is the first call again? Permission to take an action, the subject of an earlier essay here, does not settle that. An agent can be fully authorized to charge a card and still charge it twice.

Objections, considered

Just make it exactly once. It is tempting to ask for a system in which every message is delivered exactly once, and some products describe themselves that way. But no sender on an unreliable network can always know whether its message took effect, which is why the retry question arises in the first place. What careful designs achieve is narrower and more useful: messages may be delivered more than once, and the receiver recognizes the repeats within a stated scope and time. That is a real guarantee, as long as its boundary is written down.

This adds cost for a rare event. It does. Keys must be generated, stored, looked up and expired, and most requests succeed the first time. But the failures that matter are the rare ones, and the cost of a duplicate rises with the size of the effect: a second copy of a newsletter is a nuisance, a second wire transfer is an incident. Operations that are naturally safe to repeat, like reads and overwrites, need none of this. Spend the effort where a duplicate would hurt.

Deduplicate on content instead. Comparing the contents of requests needs no cooperation from the caller, which is its appeal. It fails in both directions. It refuses legitimate repeats, like the second purchase of the same pen, and it misses retries whose contents changed slightly, such as a request that carries its own timestamp. Content describes what was asked for; only the caller knows whether it was asked for once or twice.

Then the key solves the problem. It solves part of it. Keys expire. A caller can generate a new key for an old intent, for instance when a customer reloads the payment page and the checkout treats the reload as a new order. And effects that leave the service, like the courier booking, escape whatever transaction the key lives in. That is why the design should state its scope, its retention window and its reconciliation process, instead of promising that duplicates cannot happen.

A retry is another attempt, not necessarily another instruction.

Before you trust the retry

• Where is the intent named? The point in the process where the identifier is created, and whether every attempt, from every component, carries the same one.
• Is the memory kept with the effect? Whether the receiving service records the identifier and the change together, so that a crash cannot leave one without the other.
• How long is it remembered? The service's retention window for intents, and whether the caller's retry schedule fits inside it.
• What happens when the outcome is unknown? Who decides between waiting, retrying and asking, and what the caller is told while it waits.
• Where does the guarantee stop? The steps that leave the system, such as a charge, an email or a booking, and the reconciliation that catches what recognition cannot.

Back to the two strips on the desk. With a shared intent, the second strip is a question already answered: the processor finds K7Q4, returns the result of the first charge, and the customer's statement shows one line for $48.60. Without it, two strips become two charges and a call to customer service. The strips will keep arriving in pairs, because networks will keep dropping replies. The job is to make sure the statement does not.