Confidence thresholds, explained: routing decisions to humans
A confidence threshold is a number. Above it, the agent acts on its own. Below it, a supervisor reviews. That is the whole game in a human-in-the-loop system, and most of the operational quality of the agent rides on this single knob.
Where the score actually comes from
The "confidence score" is rarely a single signal. In a well-built agent it is a composite of:
- Model log-probability or self-rated certainty on the chosen action.
- Agreement between independent reasoning passes (self-consistency).
- A retrieval-quality score for any context the agent pulled.
- Policy-engine signals (dollar amount, risk tag, customer segment).
Some of these can pull the score down even when the model is confident, and that is the point. Confidence is not just "did the model think it was right." It is "is this safe to act on without a human."
How to set the threshold
Start by labeling 200–500 real production decisions: accept, edit, or reject. Sort by the agent's composite score. Find the break point above which supervisor accept rate is at or above 95%, that is your starting threshold for auto-resolution.
For the bottom segment, you tune for review time, not for accuracy. The goal is that even the lowest-confidence routed cases come with a context pack the supervisor can clear in under two minutes.
The traps
- Setting one global threshold. Different intents have different cost structures. A wrong refund and a wrong address change are not the same risk. Per-intent thresholds are not optional once you have more than a handful of decision types.
- Letting the threshold drift. Models change. Data distributions shift. A threshold that was right six weeks ago is probably wrong today. Recalibrate weekly.
- Confusing the score with calibration. A model can be confident and wrong. The threshold lives on supervisor accept rate, not on raw score. Always ground it in human-labeled data.
What good looks like
A live deployment we run hits an auto-resolution rate around 73% post-week-four with supervisor agreement at 96%. The remaining 27% routes to a supervisor with a median review time of 38 seconds. The threshold gets adjusted every week based on the previous week's labels.
That is the steering wheel. Get it right and the agent works. Get it wrong and you have either runaway false positives or a supervisor queue that swallows the team.