People ask how we "use AI for invoices." The honest answer is: we use a model to guess, then we refuse to believe it.

A language model will happily invent an invoice number that looks right. It will transpose a GST amount and still produce fluent JSON. If you persist that JSON, you have built a very expensive way to corrupt accounts payable.

GPT-4 Vision proposes JSON layout · fields not truth validators schema arithmetic identity fail → review accept human
The model suggests. Math and schema decide what ships.

The model is a parser with vibes

GPT-4 Vision is strong at layout that regex hates: stamps over totals, tables that wrap, "Amount due" in a footer in another language. We send a rendered page (or a few) and ask for a strict JSON object: vendor, number, dates, currency, line items, tax, total.

Then we throw most of the trust away.

The worker runs deterministic checks:

  • Schema: required fields, ISO dates, currency codes
  • Arithmetic: line totals sum to subtotal within a cent; tax math matches the stated rate or the stated tax amount
  • Identity: invoice number is present and not a placeholder like N/A
  • Consistency: currency on lines matches header currency

Fail any of those and the document does not become "extracted." It becomes needs review, with the model's proposal shown next to the pixels.

That UI is the product.

Confidence is not a probability

Model "confidence" is not a calibrated probability of field correctness. We use it only as a tripwire: if the API is unsure, or the response is truncated, we do not pretend we have a number.

Cost control matters here. Vision on every page of a 40-page PDF is how you discover pricing. We render the first pages that look like an invoice (keywords, layout), not the entire terms-and-conditions novel. When the model times out, we do not retry forever on the hot path — we degrade.

OCR is the understudy

Tesseract plus field-specific patterns is worse at "understanding" a document. It is better at still existing when the API is down.

Fallback is not "OCR the whole page and hope." It is:

  1. Deskew and binarize
  2. Try known vendor templates when we have seen the sender before
  3. Pull candidates for dates, amounts, and IBAN-like strings
  4. Run the same arithmetic checks as the model path

If OCR cannot prove the total, humans get the document. The shared validation layer is the point. Two extractors, one definition of done.

Never let the model pick the schema

We do not ask the model to "return whatever fields seem useful." We supply the schema. Extra keys are dropped. Missing keys fail closed.

A tiny checker is more valuable than a clever prompt:

function totalsMatch(invoice: ParsedInvoice) {
  const lines = invoice.lineItems.reduce((sum, item) => sum + item.total, 0);
  return Math.abs(lines - invoice.subtotal) < 0.01;
}

If that function is false, the document is not done. No prompt engineering overrides arithmetic.

What I would tell a team starting tomorrow

  • Budget for a review queue on day one. Not after launch.
  • Log raw model output separately from accepted fields. You will need to debug "why did we trust this."
  • Cap tokens and pages. Extraction quality dies when you silently truncate.
  • Treat vendor-specific parsers as a privilege you earn after volume, not as a rewrite of the core.

The drop in manual effort we talk about on the project page came from this shape: model where it is cheap to guess, math where it is expensive to be wrong, people where neither works. The AI is the suggestion. The system is the refusal to store a suggestion as a fact.