A safe agent must also finish the job.
Published Sep 26, 2026
What it saysA simulated world tested whether an assistant completed errands and avoided actions the user had not approved. The full harness completed 71% of errands and preserved trust on 94% of trap runs. Each trap had a matched control where acting was correct, exposing systems that stayed “safe” by refusing.
Why it mattersPermission checks can look excellent when an agent simply stops. Scoring trust and completion on the same runs makes that trade-off visible.
Try thisPair one ambiguous-permission case with a nearly identical clearly authorized control. Score both completion and unauthorized action.
How to use it
Suggested steps
- Define the user-approved boundary before the run.
- Grade executed side effects from the simulated world, not the assistant’s claim.
- Count unnecessary questions and handoffs beside trust and completion.
| Scenario | A good agent should… |
|---|---|
| Permission is unclear | Ask before acting |
| Permission is explicit | Act and finish |
| Both scenarios | Avoid hidden side effects |
Keep in mind The businesses and users were simulated, not real. The model grader was not calibrated against human raters, and the simulated user made a wrong decision 11% of the time.
A quick check for yourself
Why add a clearly authorized control beside a permission trap?
Show answer
It reveals whether the agent is careful or merely refusing to act.
Source & why it’s here
Trust and Task Completion in the World of Consumer AI Agents (opens in a new tab)
Second: a useful way to prevent safety metrics from rewarding agents that refuse everything.