A correct result can hide a broken process.
Published Oct 1, 2026Read first
What it saysThe authors checked results, tool calls, arguments, order and stored data across 240 agent runs. Of 164 runs that passed a broader set of seven final-state checks, 151 still broke at least one process rule. A dependency map grouped an average 6.34 failed checks under 2.65 designated roots.
Why it mattersA right-looking answer can come from the wrong scope, stale state or skipped readback. That hidden fault may break the next step even when today’s number passes.
Try thisChoose one critical workflow. Add one check for a tool argument, execution order or saved record beside the final-result check.
How to use it
Suggested steps
- Compute expected values independently for each run.
- Turn required tool calls, arguments, order and state changes into reusable checks.
- Group downstream failures under their declared parent check before review.
| Check | Question before release |
|---|---|
| Final state | Is the result and saved state correct? |
| Process | Were the right tools, scope and order used? |
| Dependency map | Which declared check groups the rest? |
Keep in mind The study covers two related skills and one input portfolio per skill. It tests fixed configurations, not real API evolution. Its narrow AI judge was manually checked on 60 cases, not comprehensively calibrated.
A quick check for yourself
If the final number is correct, can the run still be unsafe to reuse?
Show answer
Yes. Wrong scope, order or stored state can be hidden behind a correct result.
Source & why it’s here
Continuous Process-Level Evaluation for Evolving Enterprise AI Agent Skills (opens in a new tab)
Read first: the clearest practical method this week for finding failures that a task-completion score misses.