Automating document intake does not require choosing between checking every document by hand and trusting every extraction. A safer pattern is to let software handle routine intake, validate what it can, and route uncertain or consequential cases to a person.

The useful design is not “add a human somewhere.” It is an explicit routing policy: straight through, targeted review, or manual exception. The policy should use confidence as one signal, not as a substitute for validation or judgment.

The three-lane intake model

1. Straight-through processing

A document can continue automatically when all of the following are true:

  • the document type is recognized;
  • required fields are present;
  • extracted values pass deterministic checks, such as date parsing and numeric formats;
  • cross-field rules reconcile, such as subtotal plus tax matching the total;
  • the fields that matter for the next action meet your tested acceptance criteria.

This is an editorial operating rule, not a promise made by any particular AI service. Document tools commonly expose field or block confidence, and their documentation describes using low confidence to trigger review [S1][S2]. The business workflow still has to decide whether the result is safe to send onward.

2. Targeted human review

Most useful review queues should be narrow. Send a person the flagged fields, the source location, the validation error, and the proposed value. A reviewer should not have to retype an entire clean document just because one amount or date is uncertain.

Targeted review is appropriate when a required field is missing, a rule fails, a confidence signal is below the workflow’s tested threshold, or the document is a known variation. A sample implementation of document review uses explicit pending and in-progress states so a reviewer can claim work and complete sections without silently losing ownership [S4].

3. Manual exception handling

Some documents should not proceed through the normal extraction path: an unreadable scan, an unknown document type, a suspected duplicate, or a transaction whose consequences exceed the automation policy. Route these to a person or a separate manual process with a reason attached.

The goal is not to eliminate review. It is to make review proportional to uncertainty and consequence.

What should trigger review?

Use several independent triggers rather than one magic score.

Trigger Example Default action
Missing required value No invoice number or total Review or exception
Failed deterministic check Total does not reconcile Review
Low extraction confidence Amount or date is below its tested threshold Review
Weak document classification File resembles several supported types Exception or review
High business consequence Payment, tax, legal, or account change Review or explicit approval
New format Vendor layout not represented in evaluation data Review and log as a new variation
Duplicate or suspicious input Same invoice number and vendor already received Exception

A confidence value is a model signal, not a guarantee. Microsoft describes field confidence as an estimated probability-like value and recommends considering both OCR and extraction signals; it also notes that document variation affects results [S1]. AWS likewise exposes confidence on analyzed blocks and supports configuring human-loop workflows [S2]. Those facts justify routing, but they do not justify copying a threshold from another team.

Set thresholds per field and consequence. A small formatting error in an archival label is not equivalent to a wrong payment amount. Start with a labeled sample or a review log, measure false accepts and unnecessary reviews, and adjust the policy deliberately. Treat any example number as a hypothesis to test, not a universal standard.

Why confidence alone is not enough

A document can receive a high score and still be wrong for the business process. The classifier may choose the closest known type even when the file is out of scope. OCR can read characters confidently while extraction assigns them to the wrong field. A field can look plausible while failing a relationship check with another field.

Use confidence alongside:

  1. Provenance: Can the reviewer jump from the extracted value to the exact page or region where it appeared?
  2. Schema checks: Is the value the right type and shape?
  3. Business rules: Does it agree with related values or known records?
  4. Risk policy: What happens if the value is wrong?
  5. Novelty signals: Is this document layout or vendor new to the system?

Grounding and source locations make review faster because the reviewer can verify the proposed value against the document rather than searching blindly. Microsoft documents source and confidence considerations for extracted fields, while n8n’s approval pattern demonstrates the related principle for action execution: show the person what the automation wants to do before it happens [S1][S3].

Give the reviewer a contract, not a notification

“Please check this document” is an underspecified task. A useful review request contains:

  • the original file or a safe viewer link;
  • the extracted value and the source page/region;
  • the reason for review;
  • the validation results and any failed rule;
  • the proposed correction, if one exists;
  • the next action that approval would permit;
  • the identity of the reviewer and the time limit, if applicable.

The reviewer should be able to correct, approve, reject, or request more information. For actions with material consequences, approval should be a real gate: the workflow pauses, records the decision, and resumes only after an authorized person responds. n8n documents this approve/deny pause for AI tool calls [S3]. The exact interface can differ, but the control should not be reduced to a notification that nobody must acknowledge.

Record the before value, corrected value, reason, reviewer, timestamp, model or parser version, and final disposition. An audit trail is useful both for explaining a single decision and for finding recurring document variations that deserve better rules or labeled examples.

A practical build sequence

  1. Define the output schema. List required fields, allowed formats, and fields that may never be auto-approved.
  2. Classify the input. Separate supported document types from unknown or multi-document files.
  3. Extract with provenance. Preserve page, region, or text-span references where the tool provides them.
  4. Run deterministic validation. Parse dates and amounts, check required values, reconcile totals, and compare known identifiers.
  5. Apply the risk policy. Combine confidence, validation, novelty, and business consequence into one of the three lanes.
  6. Present a focused review task. Show the source evidence and the exact reason for the escalation.
  7. Gate the downstream action. Do not write to accounting, payment, CRM, or records systems until the required approval exists.
  8. Log the outcome. Preserve the original extraction, correction, reviewer decision, and final record.
  9. Review the queue. Measure which triggers create useful escalations and which create noise.

This sequence is a design recommendation, not a report of a workflow tested by Linda. The available documentation supports the underlying capabilities—confidence signals, human-loop configuration, approval pauses, and review states—but not a guaranteed performance level for a particular business.

What to measure

Track at least:

  • percentage of documents routed to each lane;
  • review rate by document type and trigger;
  • correction rate by field;
  • false-accept findings from audits;
  • time from intake to decision;
  • age of pending review items;
  • duplicate and exception rates;
  • changes after a parser or model update.

A lower review rate is not automatically better. If reviewers find consequential errors in the straight-through lane, the policy is too permissive. If almost everything enters review, the system is a manual queue with extra steps. Optimize for safe throughput and a review workload people can actually complete.

Small-business checklist

Before enabling automatic downstream actions, answer:

  • Which document types are supported, and what happens to unknown types?
  • Which fields are required?
  • Which fields require a person regardless of confidence?
  • What deterministic checks must pass?
  • What evidence will the reviewer see?
  • Who owns and can approve the queue?
  • What happens when a review is abandoned or times out?
  • Which records are immutable after approval?
  • How will corrections improve rules or evaluation data?
  • Which metric would make you tighten the policy?

Human review works best as a designed exception path. Automate the predictable parts, make uncertainty visible, and keep a person responsible for decisions where a plausible-looking extraction can still cause real harm.

Source Notes

  • [S1] Microsoft Learn, “Interpret and improve model accuracy and confidence scores.” Supports confidence interpretation, OCR versus extraction signals, field-level review, and document variation. Used in sections “The three-lane intake model,” “What should trigger review?”, and “Why confidence alone is not enough.”
  • [S2] AWS, “AnalyzeDocument API.” Supports block confidence and HumanLoopConfig. Used in “The three-lane intake model” and “What should trigger review?”.
  • [S3] n8n Docs, “Human-in-the-loop for tools.” Supports showing tool inputs and pausing for approve/deny. Used in “Why confidence alone is not enough” and “Give the reviewer a contract.”
  • [S4] AWS Solutions Library, “Human Review.” Supports review-pending/in-progress states, ownership, and section review. Used in “The three-lane intake model.”
  • [S5] Zapier, “Human in the Loop: Pause Zaps for human review and approval.” Supporting-only background for generic approval/data-collection patterns; not cited in body.