Autonomy is a decision, not a default
Organizations increasingly face a practical question: which decisions should an AI system execute independently, and which should be reviewed by a person? A single model accuracy number is a poor answer. Accuracy can conceal rare but consequential failures, distribution shifts, tool errors and differences between tasks. An assistant that drafts a harmless summary and an agent that changes an account record should not inherit the same permission to act merely because they use the same model.
I am interested in a reliability counter: not necessarily one literal scalar, but a compact, auditable state of evidence that informs whether a system may act, must request confirmation, or should abstain. The central research question is whether autonomy can be treated as a revocable privilege earned by verified performance under specific conditions.
What the literature already establishes
Selective classification studies the tradeoff between coverage—the share of cases a system handles—and risk on the cases it accepts. Geifman and El-Yaniv show why a reject option can improve reliability by declining uncertain cases. This is an important conceptual starting point, but multi-step agents introduce additional complications: an agent can be right about the final answer while using an impermissible tool, and one early error can propagate through a workflow.
Research on human–AI interaction, including Amershi and colleagues' guidelines, emphasizes communicating capabilities, supporting correction and making failures recoverable. NIST's AI Risk Management Framework provides a broader organizational vocabulary for governing, mapping, measuring and managing AI risks. These strands suggest that autonomy decisions need technical evidence and institutional context; neither model confidence nor a generic 'human in the loop' checkbox is sufficient.
A proposed counter as an evidence ledger
Imagine a reliability record indexed by task family, workflow version, environment, action type and consequence level. Each verified outcome updates evidence for that specific operating context. The record could include successes, failures, near misses, reversals, abstentions, time since validation and changes in the surrounding tools or data.
A Beta–Bernoulli model is one possible starting point for binary verified outcomes, with a prior and updates from observed successes and failures. It would not, by itself, capture correlated errors, delayed outcomes or severity. An alternative is a vector of evidence dimensions—task success, policy compliance, tool integrity and state verification—rather than compressing everything into one reassuring number. In either case, the output should be treated as a decision aid, not a universally calibrated probability of safety.
How human involvement might be allocated
Consider three hypothetical operating modes. Low-risk, well-verified tasks could run automatically with logging and sampled audit. Moderately consequential tasks might require confirmation when evidence is sparse or the request differs from previously validated cases. High-consequence or irreversible actions might always require explicit approval regardless of historical success. These are design possibilities, not validated thresholds.
Escalation should also be sensitive to uncertainty about the evidence itself. A system that has completed a hundred nearly identical tasks may still lack support for a novel task. Recent failures, changed APIs, newly introduced tools or a shift in user population should reduce the authority of stale performance histories. Human involvement can include approval before action, review after action, exception handling or periodic audit; the appropriate form depends on reversibility and impact.
The economics of intervention
Human review has costs: latency, staff time, attention and the possibility of automation bias. Avoiding review also has costs: error recovery, customer harm, regulatory exposure and loss of trust. A useful decision framework should compare expected total cost rather than maximize either automation or review.
One possible objective is to minimize expected execution cost plus expected failure harm plus review cost, subject to hard constraints on unacceptable actions. The hard constraints matter: a sufficiently cheap automated process should not be allowed to trade away a non-negotiable safety or authorization requirement. For ETH MTEC, this connection between technical reliability, organizational incentives and governance is especially interesting.
What could be possible, and what must be tested
An evidence-gated autonomy system could permit gradual promotion from suggestion to supervised execution to bounded autonomy, with automatic rollback when evidence deteriorates. It could also make intervention policies legible: not simply 'the model was uncertain,' but 'this workflow has insufficient verified history under the current tool version and consequence level.'
Future research could compare fixed human-review rules, model-confidence thresholds, calibrated selective prediction and context-specific reliability ledgers. Evaluation should include severe-error rate, coverage, calibration, reviewer workload, time-to-resolution, distribution shift and the frequency of unnecessary intervention. Particular attention is needed to selective labels: when humans review only difficult cases, the observed data no longer represent all decisions.
This article is a conceptual research synthesis and proposed direction. I am not claiming to have validated a counter, established an optimal intervention policy or measured improvements. The contribution I want to explore is a way to reason about autonomy as evidence-dependent, task-specific and reversible.