This is a guide to HTR supported languages — and to why a language count is the wrong place to start. Handwritten text recognition is governed first by script, then by language, then by the specific hand on the page. Understanding that order will tell you whether a tool can actually read your material, and save you weeks of misdirected effort.

Handwritten text recognition is defined first by script, not by language. An HTR engine reads the visual repertoire of glyphs on the page — the Latin alphabet, Cyrillic, Arabic, Hebrew, Devanagari, Han — and any language recorded in a script the engine can read is, in principle, within reach. So the honest answer to "which languages can HTR read?" is that it depends on whether a trained model exists for the specific combination of script, language, and hand in front of you. Coverage is uneven, community-built, and growing rather than a fixed vendor list. That distinction is the whole subject, and getting it right will save you weeks of misdirected effort.

Script versus language: the distinction that governs everything

A script (or writing system) is the set of glyph shapes used to encode text. A language is a linguistic system mapped onto that repertoire. The two are not the same axis, and they do not vary together.

The Latin alphabet alone carries English, French, German, Dutch, Spanish, Italian, Portuguese, Polish, Czech, Turkish, Vietnamese, and dozens more. The reverse also holds: a single language can be written in more than one script — Serbian appears in both Latin and Cyrillic. When a tool advertises "100+ languages supported," it is compressing a more complicated reality into a marketing figure. What actually determines whether you get a usable transcription is narrower and more specific than a language count.

This is the same distinction that separates HTR from OCR: both combine a visual model over the image with an optional language model that biases output toward plausible words, but HTR must generalize across the far greater shape variability of connected, cursive handwriting. The visual model handles the script. The language model handles the lexicon. Confuse the two and you will ask the wrong question of your tool.

In practice, "support" collapses into four things that marketing routinely conflates:

Engine capability — the underlying system can process the script at all.

A ready model — a trained, ready-to-use model exists for that script, language, and hand.

Trainability — you can train your own model on the engine, given ground-truth data.

Community catalogues — someone has already trained and published such a model.

These are different claims. An engine may be capable of a script while no ready model exists for your particular hand. Transkribus, for instance, markets 100+ languages and 150+ public models across 75+ languages through a marketplace of community-contributed models — a genuine strength, and also an illustration of the point: the coverage is a catalogue, not a guarantee, and its density varies widely by language and period. eScriptorium, built on the Kraken engine — which is optimized for historical and non-Latin material — ships no fixed "supported languages" list at all. You train per-corpus models. That is not a weakness so much as an honest reflection of how the field works.

The Latin-script family: where transfer helps

The most persistent misconception is that if a tool reads English handwriting, it cannot read French or German without starting from scratch. It is not true, and the reason is structural: within the Latin-script family the alphabet overlaps heavily, and the visual character shapes are largely shared. A model trained on one Latin-script language often transfers to a related one with only modest accuracy degradation. Transkribus markets exactly this with a generalized Latin-script model; multilingual transformer approaches such as TrOCR apply the same principle.

Transfer is strongest among closely related languages — English and French, Italian and Spanish — and degrades for more distant ones, or for languages carrying extensive diacritics such as Czech or Vietnamese, where the accents themselves become recognition targets the model may not have learned. So the practical rule is this: within Latin script, breadth across languages is often achievable from a single well-trained model, but you should expect the margin to widen the further you move from the training material.

This is also where a common naming trap lives. In OCR and Tesseract language packs, "Latin" refers to the Latin language, not the Latin script. A tool that lists "Latin" in its pack may still handle a medieval Latin manuscript in Caroline minuscule or textualis poorly — because that is a hand recognition problem, not a language-coverage problem. The question is never "does it do Latin?" It is always: script, then language, then hand.

The variable that undoes language lists: the hand

Once you accept that script comes first, the next reality is that the specific hand or typeface often matters more than the language. Historical writing and printing styles within the Latin script — secretary hand (English, 1500s to mid-1600s), court hand, Kurrent and Sütterlin (German, roughly the 1500s to the 1940s), italic, and Fraktur/blackletter — each impose their own visual conventions. Secretary hand, Kurrent, and Sütterlin in particular require hand-specific or typeface-specific models even when the underlying language is thoroughly supported. A German-language model trained on modern print will not read Kurrent; the language is the same, the hand is not.

This is why reported accuracy figures are only meaningful against a stated hand and dataset. A dedicated model like Transkribus's Egerton English Secretary Hand model reports 2.89% character error rate on its own validation set — a good result, and one that tells you nothing about how the same engine performs on a Dutch notarial hand or a degraded eighteenth-century ledger. CER and word error rate are edit-distance measures normalized by reference length; CER dominates in HTR because word boundaries are unreliable in cursive. The conventional interpretation bands — roughly 1–2% good, 2–10% average to good, above 10% poor — are widely cited but dataset-specific. Cross-tool CER comparisons without a matched test set, identical preprocessing, and consistent protocol are unreliable, and you should treat any single headline number with the caution that qualifier deserves. Understanding how those numbers are produced is worth the detour into how an HTR pipeline actually works, from segmentation through recognition to export.

Non-Latin scripts: the deeper coverage problem

Beyond the Latin family, the terrain changes. Arabic, Hebrew, Greek, Cyrillic, Devanagari, and Han are scripts, not languages — and each covers many languages. Arabic script carries Arabic, Persian, Urdu, and Pashto; Cyrillic carries Russian, Ukrainian, Bulgarian, and Serbian; Devanagari carries Hindi, Sanskrit, and Marathi. The right unit of analysis remains script plus language plus hand.

The engines exist. Kraken explicitly supports right-to-left, bidirectional, and top-to-bottom scripts. Specialist models report real results — a Russian-language Transkribus model at roughly 2.4% CER, and the HATFormer Arabic system reporting 8.6% and 4.2% CER on two datasets. But published evidence for non-Latin scripts remains thinner than for Latin, and the standard benchmark competitions (ICDAR, ICFHR; Bentham, IAM, RIMES) are predominantly Latin- or Arabic-script. If your material is Cyrillic, Arabic, or an Indic or East Asian script, expect a smaller catalogue of ready models and a greater likelihood that you will need to train your own.

What conventional OCR does — and where it stops

General-purpose cloud OCR advertises broad language coverage. Google Cloud Vision detects many languages; ABBYY FineReader lists 190-plus for print; Tesseract ships around 100 via downloadable traineddata files. These systems are genuinely good at what they were engineered for: clean, modern, high-contrast type, receipts, forms, born-digital PDFs. Fast, cheap, accurate.

Their accuracy drops materially on two things a language list does not warn you about: handwriting, and historical print. Fraktur and blackletter, the long s read as f, ligatures split or dropped, uneven inking and show-through read as spurious characters, and archaic orthography "corrected" by a modern language model — these break the assumptions a glyph classifier is built on. A language count of 190 tells you nothing about whether a tool can read a 1580 blackletter pamphlet. That gap is the reason the distinction between HTR and OCR matters in the first place, and why dedicated historical-print programmes such as OCR-D and IMPACT exist at all.

Reading Latin-script material without training a model

Most of this guide points toward a single practical conclusion: the useful question is not "how many languages," but "is there a ready model for my script, language, and hand — and if not, am I prepared to build one?" For a great deal of specialist HTR, the honest answer to that second half is that you will be training or fine-tuning per corpus.

This is the stage where Leo is built to help, and where it takes a deliberate position. Its transcription model, ATR-1, is zero-shot across the Latin-script family: it reads handwritten and printed material in the Latin alphabet from roughly the past five centuries — English, French, German, Dutch, Spanish, Italian, Latin among them — without asking you to train a model first. The scope is defined by the writing system, not the language on the page; the typical user is working in vernacular records — English wills, French notarial minutes, Dutch registers, German parish books — far more often than in Latin. On a randomized 97-image sample of early-modern English manuscripts from the Folger Shakespeare Library, ATR-1 recorded roughly 5% CER at release — about 61% fewer errors than the next-best model tested, with the full comparison published here. As with any such figure, it is a single corpus and a single hand; it describes that material, not a universal guarantee.

The constraint is the one this guide has been circling: ATR-1 reads Latin script only. Greek, Cyrillic, Hebrew, Arabic, and Indic or East Asian material is out of scope. Where it earns its place is in the source integrity that historical work demands — it transcribes what is on the page, preserving strikethroughs, marginalia, expansions, and archaic spelling rather than smoothing them into modern, plausible prose. That matters because the dangerous errors in this field are not the visibly garbled ones but the fluent fabrications a general chatbot produces, which read as correct and are hard to catch. Leo's flagship public corpus, ExLatinis, happens to be Latin-language — one collection among the many languages the same model reads, not the boundary of what it is for.

When a tool tells you how many languages it supports, treat the number as the beginning of the question, not the answer. Ask instead: Can the engine read my script? Is there a ready model for my language within that script — or one close enough to transfer? And does a model exist for my specific hand or typeface, or will I need to build one? Then check the conditions that undo even good coverage: a new period, a degraded scan, heavy abbreviation, code-switching across a page, dense tables and marginalia. HTR is not solved. Clean modern handwriting can reach very high accuracy; cursive, historical, and non-Latin hands remain variable, and every benchmark is specific to the material it was measured on. The scholars who get reliable results are the ones who match the tool to the page in front of them — and who read the output against the source, every time.

HTR support depends first on script, then language, then the specific hand — not on a headline language count. Any language written in a script the engine can read is, in principle, within reach: the Latin alphabet alone carries English, French, German, Dutch, Spanish, Italian, and dozens more. But usable transcription requires a trained model for your specific combination of script, language, and hand. Coverage is uneven, community-built, and growing rather than a fixed vendor list. When a tool advertises "100+ languages," treat that number as the start of the question, not the answer.

What is the difference between a script and a language in HTR?

A script is the set of glyph shapes used to encode text — the Latin alphabet, Cyrillic, Arabic, Hebrew, Devanagari, Han. A language is a linguistic system mapped onto that repertoire. The two are separate axes: the Latin alphabet carries English, French, German, and dozens of others, while a single language like Serbian can be written in both Latin and Cyrillic. In HTR, the visual model handles the script and the language model handles the lexicon. This distinction governs everything, because it determines whether a tool can read your material at all.

Can an HTR model trained on English read French or German?

Often, yes — within the Latin-script family a model trained on one language frequently transfers to a related one with only modest accuracy loss, because the alphabet overlaps heavily and the visual character shapes are largely shared. Transfer is strongest between closely related languages like English and French, or Italian and Spanish. It degrades for more distant languages, and for those carrying extensive diacritics such as Czech or Vietnamese, where the accents become recognition targets the model may not have learned. Expect the margin to widen the further you move from the training material.

Why does handwriting style matter more than the language?

Because the specific hand or typeface imposes its own visual conventions that a language-trained model may never have seen. Secretary hand, court hand, Kurrent and Sütterlin, italic, and Fraktur or blackletter each require hand-specific models even when the underlying language is thoroughly supported. A German model trained on modern print will not read Kurrent — the language is identical, the hand is not. This is also why reported accuracy figures only mean something against a stated hand and dataset; a strong result on one hand tells you nothing about performance on a different notarial hand or a degraded ledger.

Can HTR read Cyrillic, Arabic, or Indic scripts?

The engines can — Kraken supports right-to-left, bidirectional, and top-to-bottom scripts, and specialist models report real results, including a Russian Transkribus model around 2.4% CER and an Arabic system reporting 8.6% and 4.2% CER on two datasets. But published evidence for non-Latin scripts is thinner than for Latin, and the standard benchmark competitions are predominantly Latin- or Arabic-script. If your material is Cyrillic, Arabic, or an Indic or East Asian script, expect a smaller catalogue of ready models and a greater likelihood that you will need to train your own.

HTR support depends on script, then language, then the specific hand on the page. This guide explains why a language count misleads and how to match any tool to your material.

HTR Supported Languages: Script, Not Language, Comes First

Handwritten text recognition turns images of manuscript pages into searchable, editable text. This hub covers how HTR works, where it succeeds and struggles, and how you can make best use of it in your workflows.

Which Languages and Scripts Can HTR Read? A Guide to HTR Supported Languages

This is a working guide to HTR vs OCR — what each technology actually does, where each one breaks, and how to tell which fits the page in front of you. If you work with manuscripts or pre-modern print, the label on the tool matters far less than whether the model has seen material like yours; getting that wrong can cost you weeks before you notice the errors are systematic.

OCR (optical character recognition) and HTR (handwritten text recognition) both turn images of text into machine-readable characters, but they were built on different assumptions and break in different places. Classical OCR was engineered to read clean, modern print by segmenting the page into individual glyphs and matching each against a model of expected letterforms. HTR was designed for handwriting, where there are no fixed letterforms to match, so it reads a whole line as a sequence instead. The distinction matters most at the edges: cursive script, and pre-modern print, are exactly where a tool built on the wrong assumption quietly fails.

That is the short answer. The longer one is worth having, because the labels have grown slippery — and because choosing the wrong tool for a given page can cost you weeks before you notice the errors are systematic rather than incidental.

The cleanest way to understand the difference is mechanistically, not by which tool has which label on the box.

Classical OCR assumes reproducible shapes

Printed type is, by design, repeated: the same letter struck from the same sort looks the same down the page. Classical OCR exploits this. It typically binarises the image, segments it into lines, words, and individual glyphs — often by connected-component analysis — and classifies each isolated glyph against a model of expected letterforms. Character segmentation, as the survey literature has long noted, is the critical and fragile step. On clean modern print the approach is very good: general-purpose engines reach 95–99%+ character accuracy on well-scanned Latin-alphabet machine print, fast and cheap.

Handwriting has no stable font. The same scribe forms the same letter differently twice in a line; letters connect; ascenders collide with the line above. There are no clean glyph boundaries to segment. Modern HTR therefore abandons per-character segmentation and treats the line image as a single sequence: a convolutional and bidirectional-LSTM backbone produces a feature sequence, and either a CTC output layer (Graves, 2006) or an attention-based encoder–decoder aligns those features to characters. Because nothing forces the model to commit to where one letter ends and the next begins, it copes with connected and cursive script that segmentation-first OCR cannot.

So the difference is not "one reads print, one reads handwriting" as a matter of branding. It is a difference in what the system bets about the page: fixed, separable glyphs versus a continuous, variable stroke. That bet is the whole story, and it explains where each approach fails.

A fuller account of how that sequence model runs — segmentation, recognition, and why accuracy depends on how well the model matches your specific hand — is worth reading if you want the internals; the HTR pipeline from image to export is a separate subject in its own right. Here the goal is narrower: to know which bet fits your material.

Where the Labels Stop Describing the Tools

Two complications have blurred the line, and both matter in practice.

The OCR/HTR line is now largely an architectural fiction

The same Transformer-based models — image encoder plus text decoder — read both print and handwriting from a single checkpoint, dispensing with explicit line segmentation entirely. The field has coined a deliberately neutral umbrella term, Automatic Text Recognition (ATR), adopted by DARIAH-Campus, Teklia, OCR-D and READ-COOP, precisely because "OCR" and "HTR" no longer name mutually exclusive machines. A modern recognizer that reads a secretary-hand will and a printed pamphlet is not switching disciplines; it is applying one sequence model to two kinds of page. The practical lesson: describe a tool by what it demonstrably does on your material, not by the acronym in its marketing.

ICR (intelligent character recognition) is a vendor coinage — you will see it from Adobe and ABBYY — for template-based recognition with a layer of adaptive learning bolted on, aimed mostly at hand-printed fields in forms. It is best understood as an extension of classical OCR, not a separate discipline. If a page you care about is a centuries-old cursive letter rather than a boxed-field intake form, ICR is not the category you are shopping in.

The Place the Distinction Actually Bites: Historical Print

Here is the trap that catches most people, because it sits exactly where the labels mislead. The assumption is that OCR "can read anything printed." It cannot — not reliably — and the reason is instructive.

General-purpose OCR was trained on a distribution: clean, high-contrast, modern founts with regular letterforms and predictable spacing. Historical print violates nearly every term of that assumption:

Typography the model has never seen. Blackletter and Fraktur; the long s (ſ) read as f; ligatures (ct, st, æ, œ) split or dropped; swash and variant sorts. The catalogue of common Fraktur OCR errors runs long — f/ſ, a/u/n, c/e/o confusions, and systematic false "corrections" such as eigenthümlich silently modernised to eigentlich.

The physical page. Uneven inking, type bite, show-through from the verso, foxing, warped gutters — noise a glyph classifier reads as character evidence.

Typographic abbreviation. Early printers carried the scribal repertoire into type: the macron or titulus for an omitted nasal, brevigraphs, contraction marks. Conventional OCR has no representation for a mark that stands for absent letters, so it drops it, mangles it, or normalizes it away.

Orthography that is not an error. Archaic spelling and u/v, i/j interchange — which OCR post-processing tends to "fix," destroying exactly the evidence a textual scholar came for.

The measured cost is real. On the OCR-D ground-truth corpus of German and Latin printings from 1500–1835, pre-trained mixed models manage a character recognition rate of only 91–98%, with the worst results on the earliest material — and that is with models aimed at historical print, not general-purpose engines run blind.

The established remedy in the OCR-for-print world has been to train a book- or font-specific model on curated ground truth. It works: Calamari reaches roughly 5.5–8.8% CER on incunabula with as little as 50 lines of book-specific ground truth, and OCR4all reports font-face-specific models at ≥99.97% accuracy on early-modern Latin and German prints. The price is the ground-truth labour and the training step — reasonable for a single well-defined corpus, less so when your material is heterogeneous.

Cursive and Archival Hands: The Other Hard Case

The mirror image of the historical-print problem is manuscript. This is HTR's home ground, but "handwriting" is not one problem. Accuracy swings hard by hand, period, language, and image quality — and the published record is uneven. German Kurrent and Sütterlin are comparatively well served: Transkribus's Kurrent/Sütterlin/Fraktur model reports 5.40% CER across the 17th–20th centuries. But English secretary hand, French notarial cursive, and Dutch and Spanish archival hands have far thinner published benchmarks; what exists tends to be one-off case reports rather than systematic evaluation.

The dominant specialist workflow answers this the way the print projects do: with fine-tuning. eScriptorium can start text-recognition fine-tuning from as few as 2–3 corrected pages; Transkribus recommends at least 25 pages of training material and tells users to aim for CER ≤10% to be "accurate enough for search purposes." That is the standing bargain of specialist HTR: high accuracy in exchange for ground-truth labour, per corpus, before you can begin.

Reading the Accuracy Numbers Without Being Misled

Because this whole comparison is usually settled by citing an error rate, it is worth being precise about what an error rate is.

CER (character error rate) is the edit distance between the ground truth and the transcription — substitutions, deletions, insertions — divided by the number of characters in the reference. WER (word error rate) is the word-level analogue, and is usually larger, because a single wrong character fails the whole word.

Two cautions follow, and neither is optional.

First, a CER number is meaningless without its material. The same engine that posts low single-digit CER on modern print climbs into the teens on historical print, and handwriting figures move with the hand, the period, the language, the scan quality, and how much fine-tuning went in. A figure quoted without its corpus, language, period, model version, and ground-truth budget tells you almost nothing.

Second, no standardised cross-tool benchmark on a common historical test set exists. Vendor figures come from different corpora with different error definitions; widely cited analyst and invoice benchmarks are not peer-reviewed. The convenient thresholds — CER ≤5% excellent, 5–10% usable for search, 10–20% correction-heavy, over 20% generally not publishable — are practical convention from ICDAR and Transkribus practice, not codified standards. Treat them as heuristics.

When the Distinction Matters — and When It Stops Mattering

For most working researchers the decision reduces to a single question: is the shape on your page reproducible or variable? Reproducible, modern, clean print — invoices, contemporary books, forms — is general-purpose OCR territory, and the mainstream engines are excellent, fast, and cheap there. Anything else — cursive hands, and pre-modern print with its long s, ligatures, and typographic abbreviation — is where a tool built on the segmentation-and-match assumption degrades, and where you need a model built for the material.

The convergence to ATR is what changes the calculus. A single sequence model can read both a printed early-modern pamphlet and the handwritten marginalia beside it, which is convenient — but only if that model was trained on images of historical documents rather than on modern print. That is the distinction worth carrying forward: not OCR versus HTR as rival disciplines, but whether the recognizer has actually seen material like yours.

This is the specific gap Leo is built to close. Its ATR-1 model reads Latin-script material — whatever the language on the page, English wills, French notarial records, Dutch registers, German parish books, or the printed books and pamphlets alongside them — without a per-corpus training step, because it is zero-shot rather than fine-tune-first. On a randomized 97-image sample of early-modern English manuscripts from the Folger Shakespeare Library, ATR-1 scored roughly 5% CER at release — 61% fewer errors than the next-best model (Transkribus/Text Titan I at about 13%, and general LLMs such as Claude Opus at 23.3% and GPT-4.1 at 56.7%). Two things follow from how it is trained. It handles both print and manuscript, so the historical-print problem and the cursive problem are one workflow rather than two. And it is built to transcribe what is on the page — the long s stays a long s, the macron is not silently resolved, the archaic spelling survives — rather than smoothing the text into modern, plausible prose.

That last point is the real safety argument, and it is where general chatbots are most dangerous on this work. A specialist recognizer, when it errs, tends to err on a character or a word — visible, recoverable mistakes. A general LLM leaning on a downsampled image errs by producing fluent, confident, wholly invented text that reads perfectly and is wrong. The plausible error is the expensive one, because you may never catch it. Which language a tool truly handles is a separate question again — one that turns on script, language, and hand rather than a supported-languages count.

Stop asking whether a tool "is OCR" or "is HTR." The categories have converged, the labels mislead at the edges, and the marketing term will not tell you what happens to your actual page. Ask three questions instead: Is the shape reproducible or variable? Has the model seen material from my period, script, and hand? And when it makes a mistake, is that mistake the kind I can see and fix, or the kind that hides in fluent prose?

Answer those honestly and the tool choice usually makes itself — and, just as important, you will read the accuracy numbers you are quoted with the skepticism they deserve. The century, the hand, and the ink on your particular page decide more than any headline figure ever will.

What is the difference between HTR and OCR?

HTR (handwritten text recognition) and OCR (optical character recognition) both turn images of text into machine-readable characters, but they make different bets about the page. Classical OCR assumes reproducible shapes: it segments the page into individual glyphs and matches each against a model of expected letterforms, which works well on clean modern print. HTR assumes variable shapes, so it abandons per-character segmentation and reads a whole line as a single sequence — coping with cursive and connected script that segmentation-first OCR cannot. The distinction matters most at the edges: cursive script and pre-modern print, where a tool built on the wrong assumption quietly fails.

Can OCR read historical printed documents?

Not reliably. General-purpose OCR was trained on clean, high-contrast, modern founts with regular letterforms, and historical print violates nearly every part of that assumption. Blackletter and Fraktur, the long s read as f, split or dropped ligatures, uneven inking, show-through, and typographic abbreviations like the macron all trip up a glyph classifier. It also tends to "fix" archaic spelling and u/v, i/j interchange, destroying the exact evidence a textual scholar came for. Even models aimed at historical print manage only 91–98% character recognition on early German and Latin printings, with the worst results on the earliest material.

What is Automatic Text Recognition (ATR)?

Automatic Text Recognition (ATR) is a neutral umbrella term for recognizers that read both print and handwriting from a single model, adopted by DARIAH-Campus, Teklia, OCR-D and READ-COOP. It reflects that "OCR" and "HTR" no longer name mutually exclusive machines: the same Transformer-based models — an image encoder plus a text decoder — read a secretary-hand will and a printed pamphlet from one checkpoint, without explicit line segmentation. The practical lesson is to describe a tool by what it demonstrably does on your own material, not by the acronym in its marketing. A modern recognizer applying one sequence model to two kinds of page is not switching disciplines.

What is a good CER for historical document transcription?

CER (character error rate) is the edit distance between ground truth and transcription, divided by the number of characters in the reference. Practical convention from ICDAR and Transkribus treats CER ≤5% as excellent, 5–10% as usable for search, 10–20% as correction-heavy, and over 20% as generally not publishable — but these are heuristics, not codified standards. A CER number is meaningless without its material: the same engine posts low single-digit CER on modern print and climbs into the teens on historical print, and handwriting figures move with the hand, period, language, scan quality, and how much fine-tuning went in.

Why do general AI chatbots produce dangerous errors on historical documents?

General LLMs are most dangerous on manuscript and historical-print work because of how they fail. A specialist recognizer, when it errs, tends to err on a character or a word — visible, recoverable mistakes. A general LLM leaning on a downsampled image errs by producing fluent, confident, wholly invented text that reads perfectly and is wrong. The plausible error is the expensive one, because you may never catch it. General chatbots also tend to smooth text into modern, plausible prose — silently resolving a macron or modernising archaic spelling — rather than transcribing what is actually on the page.

HTR reads handwriting as a line sequence; OCR segments individual glyphs. Learn where each breaks on historical print and cursive, and what to ask before choosing a tool.

HTR vs OCR: What Matters for Historical Documents

HTR vs OCR: What the Difference Is, and When It Matters for Historical Documents

This article explains how HTR works, stage by stage — from the image on the scanner to the text you can search. It names the architectures that do the reading, marks the places where the pipeline still breaks, and shows why the accuracy you get depends far more on how closely your material matches a model's training than on any headline benchmark. If you work with manuscripts, knowing where a pipeline fails is what separates a usable transcription from a plausible-looking one.

Handwritten text recognition (HTR) is a computer-vision task that turns an image of handwriting into machine-readable text. It works as a pipeline: acquire the image, prepare it, find the text lines on the page, run a neural network that reads each line as a continuous sequence rather than letter by letter, decode that sequence into characters, and export the result. The reading step is the heart of it, but every stage carries its own error budget, and the accuracy you get in practice depends far more on how closely your material resembles what the model was trained on than on any headline benchmark number.

The rest of this article walks through each stage, names the architectures that do the work, and marks the places where the process still breaks.

HTR Is Not OCR — and the Difference Sets Up Everything Else

Optical character recognition (OCR) and HTR are treated as separate subfields for a reason. OCR exploits the regularity of printed type: fixed letterforms, uniform spacing, one reproducible shape per glyph. That regularity lets an OCR engine segment a line into characters and classify each one against a known set of shapes. Handwriting offers no such regularity. Strokes connect, letterforms vary from writer to writer and even within a single hand, and there is often no clean boundary between one character and the next. A system built to slice a line into discrete glyphs has nothing stable to slice.

The boundary blurs at one point worth naming, because it matters for historical work: early modern print. Fraktur and blackletter type, the long s (ſ) that generic OCR reads as f, ligatures, and typographic abbreviation push printed material toward the same shape-variation problems that make handwriting hard. That is why dedicated historical tooling gets deployed on old books as well as manuscripts. If you want the full treatment of where the line falls, we cover it separately in HTR vs OCR; here it is enough to say that HTR was built to cope with variation OCR assumes away, and the two now overlap on difficult historical print. This is one strand of the broader story we tell across our handwritten text recognition hub.

There is broad consensus on the ordering of an HTR pipeline. The stages below are the ones you will find in nearly every serious system, open-source or commercial.

Everything downstream is bounded by what the camera or scanner captured. Resolution and colour depth set a ceiling on accuracy that no later stage can lift: a model cannot recover a stroke the sensor never recorded. High-resolution capture matters more than it looks, because the fine distinctions between similar letters — a worn e and c, a minim count in m versus ni — live in exactly the detail that low resolution throws away. This is also why downsampling quietly costs you accuracy, a point that returns when we reach general-purpose models.

Historically this meant deskewing, denoising, and binarizing — reducing the image to black and white before recognition. The intuition was that a cleaner image is an easier image. Modern neural models have largely overturned it. Coquenet and colleagues, in their 2024 study of best practices for HTR systems, show that aggressive binarization and aspect-ratio-breaking resizing actively hurt deep models; their recommendations include preserving aspect ratio when resizing and skipping binarization altogether. Preprocessing is no longer the dominant source of error, and more of it is not better. A deep model reads grayscale evidence — the pressure of a stroke, the fade of iron-gall ink — that binarization would have discarded.

Before a model can read a line, the system has to find the lines: detect text regions, establish reading order, and locate baselines. On a clean single-column page this is nearly solved. On historical material it is not. Marginalia, multi-column layouts, tables, drop caps, and interlinear annotations remain genuinely hard, and the 2025 survey of text-line segmentation and baseline detection by Rabaev and colleagues identifies this as one of the main unresolved areas in the field. Layout analysis carries its own error budget, separate from recognition. A perfect reading model still produces garbled output if the reading order was wrong.

This is the stage most people mean when they say "HTR." A feature extractor turns the line image into a sequence of visual features; a sequence model turns those features into a sequence of characters. The architecture that does this has changed several times, and the generations are worth knowing because they are still all in production somewhere.

The crucial thing to understand is what this stage does not do: it does not read letter by letter.

The recognition model emits, in effect, a probability distribution over characters. Decoding turns that into final text — greedy selection of the most likely character at each step, or beam search, sometimes with a lexicon or a language model layered on to bias the output toward plausible words.

Here lies a trade-off that matters enormously for historical work. A language model raises surface accuracy on common words, but it does so by pulling the output toward what is common — which is precisely wrong when the source contains rare spellings, dialect forms, abbreviations, and unusual names. Coquenet and colleagues document this trade-off directly. It is the same mechanism, at a smaller scale, as the one that makes general LLMs risky for transcription: correction and normalization improve the average case and quietly corrupt the exceptional one. For a family historian, the "exceptional one" is often the surname you were searching for.

Finally the text is serialized into a format that preserves layout and provenance. The common standards are PAGE XML and ALTO XML for layout-plus-text, and TEI XML for scholarly encoding downstream. This is not a trivial stage: a transcription that cannot be exported in a form your edition, database, or catalogue can consume is a transcription you will end up re-keying.

Why the Model Does Not Read Letter by Letter

This is the most common misconception about HTR, and correcting it explains why modern systems work at all.

Older systems did try to segment characters before classifying them, and pre-segmentation failure dominated their error budgets — if you cut the line in the wrong place, every downstream classification inherited the mistake. The advance was learning to skip segmentation entirely. Alex Graves and colleagues introduced Connectionist Temporal Classification (CTC) in 2006 precisely to label unsegmented sequence data: the model consumes a whole line and emits a character sequence directly, using a blank token and dynamic programming to align input to output without ever being told where one character ends and the next begins.

Every dominant architecture since works this way. It reads the line as a whole, using context to resolve locally ambiguous shapes — much as a human paleographer infers an unclear letter from the word around it rather than staring at the glyph in isolation.

The Four Generations of Recognition Models

The recognition stage has passed through several architectural generations, and understanding them clarifies the choices behind any tool you might use.

Handcrafted features (stroke slopes, loops, projection profiles) fed into HMMs. The pre-deep-learning baseline, now historical.

Deep convolutional features feed a bidirectional LSTM trained with the CTC loss. This has been the workhorse since roughly 2013 and remains production-grade. It is the architecture behind Transkribus and PyLaia. Notably, Coquenet and colleagues show that with three careful fixes — aspect-ratio preservation, column-wise max-pooling, and an auxiliary CTC loss — this older design stays competitive with much larger Transformers on standard benchmarks. Newer is not automatically better.

Generation 3 — Attention Encoder–Decoder

A CNN or RNN encoder paired with an attention-based decoder. Flexible, but data-hungry.

Pure attention: an image-Transformer encoder and a text-Transformer decoder, pretrained on large synthetic and unlabeled corpora. Microsoft's TrOCR is the reference implementation, and this generation is the research state of the art on benchmarks like IAM.

Generation 5 (Emerging) — Multimodal Foundation Models

GPT-4o, Gemini, and similar systems applied zero-shot to handwriting. Competitive on clean English material — and unreliable on historical scripts, for reasons the next section makes concrete.

What the Benchmark Numbers Actually Tell You

You will see impressive figures quoted for HTR. TrOCR reports 3.42% CER on the IAM test set. The best system in the ICDAR 2017 competition on the Bentham dataset hit 1.91% CER. A 2025 benchmark of large language models for HTR by Crosilla and colleagues reports an average character error rate of around 2.95% across English, German, and French handwriting.

Character error rate (CER) is the standard measure: the edit distance between the model's output and a correct reference, divided by the reference length. Lower is better. It is a good metric. The problem is not the metric but the transfer.

Benchmark CER does not survive contact with a real archive. A model scoring single-digit CER on IAM can score 30% or more on out-of-distribution material — a different hand, a different period, worse image quality, another language. The generalization study frames this gap as the central unsolved deployment problem in HTR. Script, period, image quality, language, and individual writer all move the number, and almost no vendor publishes performance on representative samples of actual archive scans. When you read a single accuracy figure, the right question is never "how low is it?" but "on what material, and how close is that to mine?"

This is also why the claim that HTR is "solved" is misleading. Headlines about a general model conquering handwriting rest on clean English material; the same model class can fail on historical manuscripts. A 2024 study found Gemini 1.5 Pro underperforming a fine-tuned TrOCR on real historical documents — one corpus, and still requiring verification, but a useful corrective to the headline. HTR for difficult material is not solved.

The Two Operating Models: Train-Your-Own Versus Zero-Shot

Faced with the recognition stage, tools split into two camps.

User-trained engines — Transkribus, Kraken, PyLaia — ask you to supply ground-truth transcriptions for a representative sample of pages, then return a model tuned to your collection. On a genuinely idiosyncratic hand this remains the most reliable path, and it is the honest recommendation when your material is far from anything a general model has seen. The cost is real: transcribing ground truth is the very bottleneck you were trying to relieve, and the effort recurs with each new collection.

Zero-shot models — TrOCR out of the box, and the general multimodal systems — require no training data. Their accuracy depends entirely on how close your material sits to their training distribution. For clean modern handwriting that can be excellent. For historical hands it is a gamble, and it is where the failure mode turns dangerous rather than merely inconvenient.

That failure mode is worth stating plainly, because it is where the choice of tool stops being academic. On a safety spectrum, image-first recognition engines tend to err on individual characters — a misread letter, a dropped mark — errors that are visible and recoverable against the page. General LLMs sit at the other end: because they lean heavily on textual plausibility and downsample the high-resolution image that carries the real evidence, they err by producing fluent, confident, well-formed text that is simply wrong. GPT-4 Vision has been observed generating plausible but incorrect transcriptions of historical handwriting. A character error announces itself. A fluent fabrication does not — and in a scholarly edition or a family tree, the fabrication you cannot see is the one that does the damage.

This is the stage where the choice of tool matters most, and it is worth being direct about it. A model trained specifically on images of historical documents occupies useful ground between the two camps: zero-shot, so there is no per-collection training step, but built for manuscript hands rather than for clean modern text. This is where Leo's ATR-1 sits. It is a Transformer, like TrOCR and the general models, but fine-tuned on historical material and trained to transcribe what is on the page rather than to normalize it — so the long s stays a long s, an archaic spelling survives, a macron standing for an omitted nasal is not silently resolved into modern letters.

On a randomized 97-image sample of early modern English manuscripts from the Folger Shakespeare Library, ATR-1 recorded roughly 5% character error rate at release — about 61% fewer errors than the next-best system tested (Transkribus/Text Titan I at ~13%, Claude Opus ~23.3%, Gemini 2.5 Pro ~24.8%, GPT-4.1 ~56.7%), with the full comparison published here. The usual caveat applies with full force: that is one corpus, one script family, one period, and your material will move the number. It reads Latin-script material in any language written in that alphabet — English wills, French notarial records, Dutch registers, German parish books, and Latin among them — not any single language. Corrections users make feed back into training, so accuracy compounds across releases. It is not perfect, and no honest HTR is; the point is that its errors tend to be the recoverable kind, and it does not smooth the page into modern prose.

The value in understanding the HTR pipeline is not academic. It tells you where to look when a transcription disappoints. A garbled paragraph in the middle of a clean page usually points to layout analysis — the reading order was wrong, not the reading. Consistently mangled rare words point to an over-eager language model or a post-correction step normalizing what it should have preserved. A whole document that reads fluently but diverges from the source on names and dates points to a model erring on plausibility rather than evidence, and that is the pattern to fear most, because it hides.

Match the model to the material. Evaluate on held-out pages from your own collection, never on a public benchmark. Treat any language-model or post-correction layer as something to inspect rather than trust. And read the output against the page — because the transcription that looks most confident is not always the one that tells you the truth about what the writer actually wrote.

HTR works as a pipeline that turns an image of handwriting into machine-readable text. It acquires the image, prepares it, finds the text lines on the page, runs a neural network that reads each line as a continuous sequence rather than letter by letter, decodes that sequence into characters, and exports the result. The reading step is the heart of it, but every stage carries its own error budget. In practice, the accuracy you get depends far more on how closely your material resembles what the model was trained on than on any headline benchmark number.

OCR reads printed type by exploiting its regularity: fixed letterforms, uniform spacing, and one reproducible shape per glyph, which lets the engine slice a line into characters and classify each against a known set of shapes. HTR was built to cope with the variation OCR assumes away — connected strokes, letterforms that vary between and within hands, and no clean boundary between characters. The two now overlap on difficult historical print such as Fraktur, blackletter, the long s, and typographic abbreviation, which push printed material toward the same shape-variation problems that make handwriting hard.

Why doesn't an HTR model read letter by letter?

Modern HTR models read a whole line at once and use context to resolve locally ambiguous shapes, rather than segmenting characters first. Older systems did try to cut lines into characters before classifying them, and that segmentation step dominated their errors — cut in the wrong place and every downstream classification inherited the mistake. Connectionist Temporal Classification, introduced by Graves and colleagues in 2006, let a model label unsegmented sequences directly, using a blank token and dynamic programming to align input to output. Every dominant architecture since works this way, much as a paleographer infers an unclear letter from the surrounding word.

Why do benchmark accuracy numbers not match what I get on my own documents?

Benchmark accuracy does not survive contact with a real archive. A model scoring single-digit character error rate on a clean dataset like IAM can score 30% or more on out-of-distribution material — a different hand, a different period, worse image quality, another language. Script, period, image quality, language, and individual writer all move the number, and almost no vendor publishes performance on representative samples of actual archive scans. When you read a single accuracy figure, the right question is never "how low is it?" but "on what material, and how close is that to mine?"

Why are general AI models like GPT-4o risky for transcribing historical handwriting?

General multimodal models lean heavily on textual plausibility and downsample the high-resolution image that carries the real evidence, so they tend to produce fluent, confident, well-formed text that is simply wrong. A character error from an image-first engine announces itself and is recoverable against the page; a fluent fabrication does not, and in a scholarly edition or family tree the error you cannot see does the damage. These models can be excellent on clean modern handwriting, but on historical hands the failure mode turns dangerous rather than merely inconvenient. GPT-4 Vision has been observed generating plausible but incorrect transcriptions of historical handwriting.

A stage-by-stage look at the HTR pipeline, from image capture to export, naming the architectures that do the reading and the points where accuracy breaks down.

How Handwritten Text Recognition Works | Leo

How Handwritten Text Recognition Works: A Look Inside the HTR Pipeline

Handwritten Text Recognition (HTR)

Which Languages and Scripts Can HTR Read? A Guide to HTR Supported Languages

HTR vs OCR: What the Difference Is, and When It Matters for Historical Documents

How Handwritten Text Recognition Works: A Look Inside the HTR Pipeline