AI strategy

Budget for the Human Review Your AI Workflow Still Needs

Turn human oversight into a workable operating budget with measured handling time, specialist escalation, quality checks and peak-period coverage.

Top One Percent AI · Practical guidance with linked sources.

An AI workflow can generate its first answer in seconds and still depend on someone finding a source, checking a claim, correcting a record or deciding whether an action should proceed. Those tasks need time, skills and coverage. If the operating budget contains model usage and software licences but no review capacity, it is missing part of the service.

For leaders planning deployment, the practical question is how much human work the chosen controls require at the expected volume. “A person will check it” is a design assumption that needs a staffing plan.

Define the review before pricing it

The UK government's AI Playbook says the level of human involvement depends on factors including output complexity, potential impact and specialist knowledge. It recommends documenting where meaningful review happens and what additional information informs the final decision. This is public-sector guidance, but the operating question is useful more broadly. Read the AI Playbook's human oversight guidance.

For each workflow, describe the reviewer's actual task. Checking a summary against a short source is different from investigating an unsupported claim across several systems. Approving an external action is different from checking an internal draft after use.

Specify what the reviewer can see, what counts as an acceptable result, when specialist escalation is needed and who can stop the workflow. Establish which outputs require review before release. Sampling may be appropriate for some monitoring tasks, but it cannot replace an individual check required by law, policy or the consequences of the decision.

Build a workload budget

Use observed handling times from representative work where possible. Record routine checks separately from corrections, escalations and quality assurance. Timing only the easiest examples will understate the work.

A simple planning calculation is:

Review hours = reviewed cases × average review minutes ÷ 60.

Then add incremental escalation work, calibration, training, monitoring and coverage. Avoid double-counting escalation time already included in the average review duration. Where roles have different costs, calculate their hours separately.

Here is a hypothetical monthly example, not a benchmark or recommended review rate:

• A low-risk internal workflow handles 1,000 cases. Its approved control plan sends 200 for review.
• Each check takes six minutes, producing 20 review hours.
• Twenty of those reviewed cases need an additional 15 minutes of specialist work, adding five hours.
• Training, calibration and monitoring require another eight hours.
• The total is 33 hours. At an illustrative fully loaded rate of US$60 per hour for all these activities, the labour budget is US$1,980, or US$1.98 per incoming case.

That figure excludes model usage, software, integration and other operating costs. It also assumes sufficient coverage at the times the work arrives. Replace every input with measured workload, local costs and the controls your organisation actually needs.

Stress-test the assumptions. If average checks take nine minutes rather than six, the same example needs 43 hours and US$2,580, with other inputs unchanged. If a policy requires all 1,000 cases to be checked, recalculate the entire plan, including likely escalation work. Do not preserve an attractive budget by quietly weakening the review requirement.

Budget for effective attention

Review capacity is more than total hours. A specialist who has five hours available next week cannot clear a queue that must be resolved today. Identify response deadlines, peak arrival periods, absence cover and what happens when the queue exceeds capacity. Pausing an affected action or routing it to an established manual process may be safer than allowing unchecked output to proceed.

Quality matters too. NIST's Generative AI Profile identifies automation bias as excessive deference to automated systems, which can worsen other risks. A human approval click does not establish that useful scrutiny occurred. Read NIST's Human-AI Configuration discussion.

Allocate time to compare reviewer decisions, investigate disagreements and improve instructions. Give reviewers access to the underlying evidence rather than only the generated answer. Track consequential errors missed as well as corrections made, using a separate quality check where warranted. These are proposed operating practices, not claims that any review process can guarantee accuracy.

Give a planning session a concrete output

An organiser can run a 40-minute review-capacity exercise with operations, finance, a domain reviewer and the system team. Bring a sanitised workflow and several representative cases. Use ten minutes to map review points, ten to estimate work by role, ten to test a busy-period scenario and ten to agree the next measurement step.

Ask the group to leave with a one-page operating brief: review obligations, expected volumes, handling-time assumptions, required expertise, coverage, escalation route and budget owner. Mark estimates that still need evidence.

The strategic decision may be to fund more review capacity, narrow the workflow or redesign it so verification is easier. Putting the human work into the budget makes that choice visible before the service depends on unpaid attention and an overflowing queue.