Idempotency is a product feature

Picture a customer in the basement car park of a mall in Petaling Jaya. They tap Pay. The spinner turns. The signal drops to one bar, then none. After fifteen seconds the app says something went wrong. They walk to the exit, the signal comes back, and they tap Pay again.

Did they pay once or twice? The honest answer, in a lot of systems I have reviewed, is that nobody knows until the bank statement arrives. That is not a backend detail. It is the most important moment in the whole product, and it is decided by one property: whether the request is idempotent.

What the word means

An operation is idempotent if doing it twice has the same effect as doing it once. Reading a balance is idempotent. Setting a delivery address to a value is idempotent. Charging a card is not, unless you make it so.

On a phone network in Southeast Asia, every request will sometimes be sent more than once. Mobile clients retry. Load balancers retry. Your own background jobs retry. You cannot stop duplicates from arriving. You can only decide what happens when they do.

The idempotency key

The standard tool is simple. The client creates a unique key for each intent to do something, and sends it with the request. The server stores the key with the result the first time, and returns the stored result every time after that. The second tap in the car park sends the same key, so it gets the first payment back instead of a new one.

1POST /v1/payments2Idempotency-Key: 7f9c2b1e-4a0d-4b8e-9f51-2c3d8e6a1b403Content-Type: application/json45{ "amount": 4590, "currency": "MYR", "order_id": "ord_8812" }

The important word is intent. The key belongs to the customer's decision to pay for this order, not to the HTTP request. If the app generates a fresh key on every retry, the key does nothing. Create it when the customer reaches the pay screen, keep it in memory and on disk, and only throw it away when the order is settled or abandoned.

Where teams get it wrong

I see the same four problems again and again.

• The key is checked after the side effect. The server charges the card, then saves the key. A crash between the two steps means the retry charges again. Save the key first, in the same transaction that records the intent.
• Two requests with the same key arrive at the same moment, both see no stored result, and both proceed. You need a lock or a unique constraint, so that one of them wins and the other waits or gets a clear conflict response.
• The key is reused with a different body. A buggy client sends the same key for RM 45.90 and later for RM 459.00. The server should refuse the second one loudly, not return the first result as if nothing happened.
• Errors are not stored. If the first attempt failed because the card was declined, the retry should see the same decline, not try the card again and maybe succeed at a different amount.

Here is the shape I use. A table of keys with a unique constraint, a hash of the request body, a status and the stored response.

1CREATE TABLE idempotency_keys (2  key          text PRIMARY KEY,3  account_id   bigint NOT NULL,4  request_hash bytea NOT NULL,5  status       text NOT NULL CHECK (status IN ('started', 'done')),6  response     jsonb,7  created_at   timestamptz NOT NULL DEFAULT now()8);

The first request inserts a row with status started, inside the same transaction that creates the payment intent. A second request with the same key hits the primary key and stops. If the row is done, it gets the stored response. If the row is still started, it gets a 409 with a Retry-After header, and the client waits a moment and asks again. If the hash does not match, it gets a 422 that names the problem.

Use what you already have

Sometimes you do not need a separate key at all, because the domain already has one. On the Pasarloka checkout, every order starts from a quote: a priced basket with a slot and an expiry. A quote can only become one order, so the quote id is the idempotency key. There is no extra header for the app to manage and no extra table to clean up. The database enforces it with one unique index.

Look for these natural keys before you add machinery. An order id for a payment. A payout run date and seller id for a transfer. A bank's own reference for an incoming settlement line. They are easier to reason about because they mean something to people, not just to the code.

Tell the customer the truth

Idempotency on the server is half the job. The other half is the screen. If the app does not know whether a payment went through, it should say so plainly and keep checking, not show a generic error that invites another tap.

On one wallet we replaced the message 'Something went wrong, please try again' with 'We are confirming your payment with the bank. Do not pay again. This usually takes under a minute.' and a status that updated on its own. Duplicate payment complaints fell by more than half in the first month, before any backend change shipped.

The second tap in the car park is not an edge case. It is the main case, on a bad day.

Test the bad day

The last thing I ask for is a test that sends every payment request twice, in parallel, with a random delay. Run it in CI against a real database, not a mock. It finds missing locks and wrong ordering faster than any review I can do. On Tidewire's webhook system we ran the same kind of test from the other side, delivering every event twice to a fake customer endpoint, which is how we learned that our own docs told customers to deduplicate on a field we did not always send.

None of this shows up in a product demo. Nobody claps for a payment that happened exactly once. But it is the reason people trust the app enough to use it again tomorrow, and that is the product.