Human in the Loop
Automation with oversight, not automation as abandonment.
The Principle
Section titled “The Principle”- Write down which decisions need a human
- Make every checkpoint fast to clear
- Too little oversight ships irreversible mistakes (external email, destructive migration, compliance breach)
- Too much oversight makes humans the bottleneck and review becomes rubber-stamping
- The root failure is an undefined boundary; undefined lines drift toward convenience
When to Require a Human
Section titled “When to Require a Human”- High stakes: hard-to-reverse actions, external communication, significant spend, safety-critical changes
- Low confidence: high uncertainty, novel situations, conflicting signals
- Policy: compliance mandates, audit trails, formal approvals
Making Review Efficient
Section titled “Making Review Efficient”- Put the relevant context next to the decision
- Present a recommended action, not an open question
- Make approve/reject a single step
- Batch similar low-risk decisions
- For code and documents, ask for output shaped for review (Reviewable Output)
Signal of Violation
Section titled “Signal of Violation”- Undefined boundary: whether a human signs off depends on who ran the agent that day
- Rubber-stamping: approval rate near 100%, with seconds spent per decision
- Human as bottleneck: work waits in the approval queue longer than the agent took to do it
- Oversight after the fact: an external message or destructive change is found with no recorded approval
Implemented By
Section titled “Implemented By”- Adversarial Review: machine review before human review
- Verification Loops: only verified output reaches the checkpoint
- Reviewable Output: small diffs, evidence, and staged checkpoints
- Agent Architecture: trust and validation boundaries
Open Questions
Section titled “Open Questions”- Which signals show a checkpoint is mis-tuned (near-100% approval, queue latency)?
- When no reviewer is available, does the system block, fall back, or queue?
Proposal: Discussion #8