ReplayGuard Failure lab

The failure lab for queued workers

One receipt.
For every order.

A retry can fulfill twice. A careless fix can lose a valid order. ReplayGuard catches both.

Try it with code

Recorded local comparison · simulated fulfillment.
Separate AWS observations are linked below.

Scroll to follow the failure 01 — 04

ORDER-A · receipt committed

The receipt exists.
The response doesn’t.

A’s receipt is committed. The configured fault interrupts the response. Retrying now could fulfill the order twice.

ORDER-B · same product, different order

A valid order.
The wrong key.

B requests the same product. The product key raises a payload conflict and leaves B without a receipt.

ORDER-A · delivered again

One retry.
A second fulfillment.

Without a key, A’s retry creates another receipt. Both keyed handlers reuse A’s original receipt.

ORDER-B · returns in a new message

One receipt.
For each order.

B returns through another message. Only the order key finishes with one receipt for A and one for B.

Recorded local example4 of 4 deliveries

Two orders. Same product.

One crash. Three choices of key.

The order key protects both orders.

Each order has one receipt. The other repairs duplicate work or lose ORDER-B.

Final counts checked against independent receipt snapshots.

Started with a real AWS failureLambda + SQS + DynamoDBWatch the demo (2:34)Code, receipts and evidenceTake the failure with youOpen source. Runnable regression.

Inspect the repair

Find the order your fix missed.

Choose a handler. See every case it passed, broke or could not resolve.

Recorded example19 September 2026 · Python execution

Bundled local comparison. The opening story always follows this recording; imported files only change this inspector.

Inspect a handler

Execution & source checks
Follow the delivery trail
Source fingerprintsInspect

Captured Python source files. Hashes identify the supplied code; they do not authenticate the report’s origin.

Your file stays in this browser. Importing checks the supplied observations; it does not run Python or authenticate their origin.

A small test. A useful habit.

Break the repair.
Keep the proof.

Change the key from order to product. Watch a valid order lose its receipt. Restore the key, and keep the same case as a regression.

Download the regression kit

Extract the kit, then open a terminal in its folder.
Python 3.11+ · macOS or Linux · No packages or AWS credentials

candidate/adapter.pyPython
def build(effects):
    def handle(delivery):
        order = delivery["order"]
        return effects.fulfill(
            order, key=order["orderId"]
        )
    return handle
Copy the handler and run one case
Expected result
Inspect one receipt for each order.

Import before.json after the command finishes.

Ready for your own worker? Adapter guide & evidence limits

Your trusted Python adapter connects to the simulated receiver. Read the adapter guide.

Check the limits: the full example suite includes three handlers and deliberate evidence-limit controls. An aggregate Unresolved / exit 3 is expected.

python3 -I -S scripts/test_repair.py --output repair-results.json

The two-order AWS experiment

Three keys. Two valid orders.

Recorded AWS evidence · experiment passed

Inspect the AWS comparison

Earlier AWS experiment · 18 September 2026

The retry was real.
The lesson travels.

A Lambda worker crashed after simulated fulfillment succeeded. Standard SQS redelivered the message. A separate observer counted the receipts in DynamoDB.

Inspect the recorded AWS evidence
SQSredeliversLambdaworkerFulfillmentLambdaDynamoDBreceipts
Vulnerable handler2 receipts
Repaired handler1 receipt

16 assertions passed in the recorded AWS run.

The experiment stacks were removed after capture. AWS Amplify hosts the current viewer. New regressions run locally; this replay makes no AWS calls.

Original evidence

The supplied report.

Preserved as imported. Case selection does not rewrite the evidence.