The report the AI generates looks flawless. And several figures are wrong

Why data extracted by AI can have perfect formatting and still be wrong, and the validation layer that keeps it from reaching a decision that way

Someone on the admin team set up a process for the AI to read supplier invoices and load them into the system. It works: what used to take an afternoon now takes ten minutes. The report it produces looks flawless — aligned columns, every field filled in, dates in the right format, zero typos. And three out of every ten invoices have at least one wrong value.

Nobody noticed, because the report looks better than it did when a person filled it in by hand. No cross-outs, no cells flagged in red, no “pending confirmation.” The neatness of the output reads as a sign that it’s been carefully checked. It hasn’t.

The problem isn’t the AI, it’s where you decide to trust it

This pattern shows up in any company that started using AI to pull data out of documents: invoices, purchase orders, delivery notes, customer forms, contracts. The technology to do this has gotten a lot better, and it’s tempting: you connect it, it reads the PDF even when it’s badly scanned, and it hands back a table ready to load. The time savings are real and visible from day one.

What isn’t visible from day one is the error rate. And as volume climbs — from forty documents a week to forty a day — checking each one by hand stops being an option, exactly when it matters most. The process turns into a fast black box that people trust because it has never produced an obvious problem.

Why can a well-formatted value still be wrong?

Modern AI models, especially with “structured output” modes, guarantee the shape of the result: that every field exists, that the data type is right, that a date has the structure of a date and an amount is a number. That’s what makes the output look professional.

But the shape says nothing about the content. An amount can be a perfectly valid number and still not match the sum of the line items on the document. A date can be in the right format and be the due date when the system expected the issue date. The model fills the mold; whether what it put inside is true is a separate question.

How exactly does it get things wrong?

The errors in an automated extraction aren’t random. They repeat in patterns:

• Silent omission. An invoice with twelve line items comes back with eleven, perfectly formatted. Nothing indicates one is missing.
• Plausible substitution. A blurry or ambiguous field gets filled with the most likely value instead of being flagged as uncertain. A poorly scanned “7” becomes a “1” with full confidence.
• Wrong field. The document has two dates or three amounts; the model consistently picks one, and it isn’t always the right one.
• Normalization you didn’t ask for. A value written as “1.500,00” gets read as 1.5. A currency is assumed when the document never stated it.
• Filling in what isn’t there. A field that doesn’t appear on the document is inferred from context instead of being returned empty.

None of these errors leave a trace in the formatting. All of them pass any check that only looks at “is it complete and correctly typed?”

What AI extraction handles well	What it can break silently
Format: every field present, right type, valid date	Content: the extracted date is the due date, not the issue date
Speed: hundreds of documents in minutes	Completeness: one item out of twelve is lost and nothing flags it
Structural consistency across different documents	Uncertain values filled with the “most likely” data
Legible text even when the original is badly scanned	Normalization you didn’t ask for: “1.500,00” read as 1.5
Fields that are always in the same place	Missing data filled by inference instead of left empty

Why is this worse than human error?

A person entering data by hand also makes mistakes, but in familiar ways. They transpose digits when they’re tired, skip a line when they get interrupted. The team learns those patterns and knows where to look twice: the totals at the end of the day, the batches filed right after a holiday.

AI errors are rarer in frequency and stranger in form. They show up once in a while, in places nobody is watching, and with a presentation that invites you not to check. The result is a false sense that quality control is solved, when it has really just been moved into a blind spot.

A typical case (composite, not a specific client)

A mid-size distribution company receives about 300 supplier invoices a week. Before, two people in purchasing loaded them into the ERP by hand; it took a good chunk of the week, and every so often a typo slipped through that got caught during month-end reconciliation.

They set up an AI extraction and load time dropped from days to a couple of hours. For four months nobody checked anything, because the weekly report looked perfect and suppliers weren’t complaining. In the fifth month, an internal audit found that around 8% of invoices had the tax amount loaded wrong: the model was picking up an intermediate subtotal on an invoice layout that one large supplier had changed without notice. Four months of accounting data with a bias nobody saw, because the error came in well-formatted.

With one reconciliation rule — tax plus subtotal equals total, and total equals the sum of the line items — that invoice would never have passed the load. It would have stopped at the first record with the new layout, not four months later.

What has to exist before you can trust the extraction?

Extraction is the easy part. The part that makes the time savings real rather than apparent is the layer that validates what came out:

• Reconciliation against a source of truth. The extracted total has to match the sum of the extracted line items and the value that already lives in another system — the purchase order, the contract, the ERP. If it doesn’t reconcile, it doesn’t pass: it goes to a person.
• Validation rules on every load. Keys that don’t repeat (don’t load the same invoice twice), required fields that can’t come in empty, values within an allowed list (the currency is one of the ones I use), quantities and dates within a plausible range. The load stops and flags before the value reaches a dashboard or a ledger entry.
• A confidence signal and a human checkpoint. When the model reports low confidence in a field, or when a record fails a rule, that record goes to manual review. The rest flows. The person stops reviewing everything and starts reviewing only what the system flagged.
• Periodic sampling. Even the records that passed: once a week you pull a random 2-5% and compare them by hand against the original document. That gives you a real error rate and, more importantly, shows whether that rate is moving — when it moves, something changed upstream (a new invoice template, a different supplier, a model update).

In a data lake we operate, around 45 of these rules run automatically on every pipeline run — uniqueness, required fields, allowed values, referential integrity — and a load that fails a rule stops before it contaminates what the end user sees. The human approval step is explicit: the flow halts and waits for a person’s sign-off before the final stretch. It isn’t the sophistication of the process; it’s the boring, systematic part that makes it trustworthy.

When does this apply, and when doesn’t it?

Building a full validation layer makes sense when the extracted data feeds decisions or records that matter: what gets loaded into accounting, what goes to a dashboard management looks at, what triggers a payment. It also makes sense once volume is high enough that nobody reviews document by document.

You don’t need all this machinery if a human is going to look at every document anyway as part of another process — with the AI as just a first draft that speeds up that review — if the cost of an occasional error is trivial and easy to fix later, or if the source document already comes structured (a file from the supplier, an export from another system), in which case you don’t need AI to read it: a connector is enough.

And the other way around: if AI extraction feeds something critical and there’s no reconciliation on top of it, the problem isn’t that you need to automate more. It’s that what you already automated can’t be audited.

This week’s takeaway

If your company already uses AI to load data from documents, ask yourself a concrete question: what is the error rate of that process? If nobody has that number, pull thirty already-processed documents at random and compare them by hand against the original. That one-hour exercise tells you more about whether the process works than six months without complaints — because the errors in an automated extraction don’t complain.

FAQ

Don’t “structured output” or “JSON schema” modes solve this?

No. They guarantee the result has the right shape — all the fields, the data types, the date format — but not that the content is true. A total can be a valid number and still not match the line items on the document.

How do I know if my extraction process has this problem today?

Pull thirty to fifty already-processed documents at random and compare them by hand against the original. Count how many have at least one wrong value. If you’ve never done that count, you don’t know whether your process works; you only know it’s fast.

Does this mean AI isn’t worth using to read documents?

It is worth it, in many cases. But with a reconciliation and validation layer on top. Extraction without reconciliation is a faster way to arrive at a wrong value — and with more confidence, on top of that.

How much work is it to build the validation layer?

Less than the extraction project itself. The rules get defined once, with the people who know the business, and run on every load without intervention. That’s what turns the time savings into something real rather than apparent.

What happens when the AI model gets updated?

It can change behavior on edge cases, for better or worse. That’s why periodic sampling isn’t optional: it’s the only way to notice the error rate moved before that movement reaches a decision.

---

If you’re figuring out where trust in your data can break, the automated-extraction problem is a cousin of the one we covered in migrating to the cloud is not a data strategy: having the data is not the same as being able to trust it. And if the process you want to automate still lives in a spreadsheet, it’s worth reading your competitor monitoring spreadsheet worked until it didn’t first.