Operations · before you trust it
Every step worked
.
Your automations say they're fine. How many jobs quietly go wrong?
Ninety-nine in a hundred. Five times over.
One job, step by step, as it passes from system to system or person to person. The starting figures are an online order, so swap in your own steps.
- 49
- Jobs in every 1,000 that break somewhere, about 1 in 20
- 59
- A month that nobody spots until a customer does. With an end check, 3
1,000 runs, step by stepstill whole
1 in 20Every step said success.
It said "Success", so I went home. Brent
Rough sums. Not a test of your systems. Each step is taken to go wrong on its own, whatever the others did. Count it from your error logs, your refunds, or the jobs someone had to redo.
How this works
A job is only whole if every step works. So the chances multiply: a step that works 990 times in 1,000, five times in a row, leaves about 951 whole jobs. The other 49 break somewhere, even though each step looks fine on its own.
Whole jobs in 1,000 = 1,000 × (1 − first ÷ 1,000) × (1 − second ÷ 1,000) × …
Nobody spots = broken × (1 − share the step flags)
With an end check = nobody spots × (1 − share the check catches)
Checking each step tells you each step ran. Checking the finished job tells you the job got done.
49broken in every 1,000 runs1 in 20
We did one like it
The work · Run
Every step worked. The job didn't.
A software business with 70 staff had automations across support, billing and admin, with nobody in charge. One workflow said success after creating a support task, even when the billing update behind it failed. Now the finished job is checked, end to end.
Manual recovery 42 to 16 per 1,000 runs
Read the job More on operations