Receipt OCR, explained: how software reads a receipt
A plain-English tour of receipt OCR: the pipeline, failure modes, and how modern extractors combine layout, language models, and math checks.
Receipt OCR is the most misunderstood category in document automation. Vendors advertise 99 percent accuracy, users see totals off by a decimal point, and neither side is lying. The truth is that OCR on receipts is not one problem, it is five, and each one has a different failure mode. This guide walks through the pipeline in plain English so you know what to expect from any tool that promises to read a receipt for you.
What a receipt actually is
A paper receipt printed on thermal paper is a low-fidelity artifact. It is faded, wrinkled, sometimes glossy, sometimes upside down in the photo. The text is usually mono-spaced and printed at low DPI. Column boundaries are implied by whitespace, not lines. Line items wrap unpredictably. Currency symbols sit on either side of the amount depending on the country. Every one of these characteristics fights the OCR engine.
The five-stage pipeline
- Image cleanup: deskew the photo, remove background, adjust contrast, and correct perspective. A tilted receipt costs you 10-20 accuracy points on the next stage.
- Character recognition: convert pixels to characters. This is what most people mean when they say OCR. Modern engines are neural, not template-based.
- Layout analysis: group characters into lines and lines into columns. Columns are where receipts are hardest, because the boundaries are whitespace, not rules.
- Field extraction: pick out the merchant, date, subtotal, tax, tip, total, and each line item. Every field has its own patterns and edge cases.
- Validation: do the numbers add up. If line items plus tax do not equal the total, something is wrong in an earlier stage, and a good extractor loops back.
Why totals get wrong
The single most common receipt OCR error is a wrong total. It is almost never the OCR engine failing to see the digits. It is usually one of three things: the engine confused a decimal separator (dot versus comma) in a European receipt, it swapped a subtotal for the total when both appeared in the same column, or the printed total was cut off at the bottom of the photo. Every serious receipt extractor runs an internal math check as its last step, and rejects a result that does not reconcile.
Why line items are hardest
Merchant, date, and total are single values. Line items are variable-length lists with implicit columns, and every restaurant, retailer, and supermarket formats them differently. A grocery receipt might have three columns (description, quantity, price). A restaurant receipt often has two. A hardware store might use four including a SKU. There is no universal template, and the model has to infer the column count from the data itself.
Where language models help
Modern receipt extractors combine classical OCR with a language model that understands the semantics. The OCR engine reads the text; the language model decides that "1.99 lb organic bananas" is one line item with a quantity in pounds. This hybrid approach is more accurate than either component alone, especially on messy line items. Karchu uses the same hybrid pipeline for its receipt extractor.
Photo quality matters more than software quality
Give any modern OCR system a well-lit photo taken from directly above with the receipt flat against a contrasting background and it will hit 97 to 99 percent accuracy on the fields that matter. Give it a crumpled, backlit, angled photo and no software will save you. If you care about accuracy, standardize the capture process before you shop for extractors.
What honest accuracy numbers look like
Field-level accuracy of 95-99 percent is realistic for a good extractor on decent photos. Line-item accuracy of 85-92 percent is realistic on receipts that have more than a few items. Any vendor quoting 99.9 percent across every field on every receipt is measuring something else, usually just the merchant name field.
Where OCR is going
Recent multimodal models can go directly from a photo to structured JSON without an explicit OCR step. They are simpler to integrate and often more accurate on messy inputs. The trade-off is cost and latency: a multimodal call can take one to three seconds, versus one hundred milliseconds for a classical OCR pass. Most tools now offer both paths and pick per-receipt based on quality.