The world's documentary heritage lies scattered across thousands of local archives, paywalled databases, and hard drives. Most of it has never been catalogued, still less digitized. Leo Collections is  an ambitious attempt to draw it together. We're building a crowdsourced, machine-readable archive of the world's textual past, in four steps: gathering images, transcribing them, making collections speak to one another, and surfacing the patterns held within.

Patrick Manning, Mellon Professor of World History at Pittsburgh and a former president of the American Historical Association, spent much of his career on a problem his colleagues deemed unrealistic, even mad: to develop the infrastructure needed to write a comparative history of the world. Such a history, he argued, could not be built on traditional scholarly habits. It needed shared databases, standardized metadata, and systems interoperable enough to hold evidence from different languages, societies, and documentary traditions within one view. Through the World History Network and the Collaborative for Historical Information and Analysis at Pittsburgh, he brought historians and data scientists together to pool evidence about population, migration, and social structure.

Though it inspired much fruitful work, Manning never quite realized his dream. The world's archival heritage remains scattered across miles of shelving in national and local repositories, unmarked boxes in municipal offices, parish registers, ship's logs, and census records strewn across jurisdictions in specialist collections. The fraction that has been catalogued is described under wildly inconsistent schemes, and the even smaller portion that has been digitized is dispersed across hard drives, brittle links, and project sites that vanish when funding expires. 

Every dimension of the task cuts against unification: the sheer volume across thousands of institutions; the heterogeneity of sources that resist clean transcription and tidy standardization; metadata schemas devised in different cultural and computational worlds; and a professional reward structure that prizes novel findings over the unglamorous work of building shared infrastructure.

Despite the practical difficulties, the potential rewards of a global archive are immense. You can catch a glimpse of them where a historian has linked far-flung records by hand. Natalie Zemon Davis, in Trickster Travels, reconstructed the life of al-Hasan al-Wazzan, a Muslim diplomat from Fez, seized by Christian pirates and presented to a pope, who became one of Renaissance Europe's principal informants on Africa and was baptized as Leo Africanus. Tracing those connections across languages and continents took Davis years. In a truly interconnected archive, they could surface in seconds.

Such a world archive would change what kinds of questions could be asked about the past. Patterns now buried across dispersed ledgers and unindexed manuscripts would come into view, from the webs of migration and exchange that tied distant societies together to the movement of ideas across linguistic and imperial borders. Fernand Braudel's attentiveness to the longue durée, the deep structures of geography and demography beneath the surface of events, was constrained by the limited evidence available to test it. Where Braudel and the annalistes were forced to work from selective samples and informed conjecture, a universal archive would offer a more secure evidentiary foundation for the same panoramic view.

Ramelli's bookwheel, from Le diverse et artificiose machine: an imaginary machine that held a dozen open books at once and turned them sequentially past the reader's gaze. Source ↗.

A potted history of the universal archive

Manning did not invent the idea of gathering knowledge into a single ordered store. The practice has no clear beginning at all, since just about every literate culture has cared to preserve its written heritage. Some fifteen thousand clay tablets survive from the royal palace at Mari, on the Euphrates, dating to the second millennium BC. Such gathering tends, soon enough, to lead into dreaming of gathering everything, of a one, unified emporium containing all possible knowledge. Thomas Richards evocatively describes the "imperial archive" as a kind of utopian fantasy, of knowledge unified in the service of state power.

That vision took firmer institutional shape after the turn of the first millennium, when written documents multiplied across Europe. Charters that defined legal relationships were produced in growing numbers, rulers gathered information about their subjects as a means of governing them, and a class of trained scribes, secretaries, and copyists emerged to produce and keep the new profusion of records. In Markus Friedrich's retelling, the cartulary, an organized book collecting a monastery's or a noble house's important documents in one place, and the register, a systematic copy of documents issued or received, were critical early steps toward the deliberate, organized preservation of records.

By the end of the Middle Ages, written documentation had worked its way into nearly every aspect of public and private life. Records that once traveled with a ruler's retinue solidified into fixed, permanent repositories, and there were many more of them: the Spanish crown's great central archive at Simancas, established between 1540 and 1561, alongside a proliferation of diplomatic, civic, guild, and ecclesiastical collections. The explosion of archives came, in early modern Europe, with a wider impulse to systematize. In the mid-seventeenth century the Czech reformer Jan Amos Comenius proposed to bring all knowledge into a single frame, developing a "pansophic" understanding of the world as one interdependent whole in the hope of fostering peace. The scientific dictionaries that proliferated during the early eighteenth century, as Richard Yeo has shown, took up the same ambition with a sharper sense of its limits, compressing and classifying the overwhelming flow of printed knowledge. Where the dictionary tried to contain knowledge, the encyclopedia tried to connect and circulate it: Chambers's Cyclopædia of 1728 gave up the dream of one stable map of the sciences for the pragmatism of the alphabet, and Diderot and d'Alembert carried the ambition to its famous height.

The frontispiece of Diderot and d'Alembert's Encyclopédie, engraved by Bonaventure-Louis Prévost, where Truth stands veiled and radiant beneath an Ionic temple while Reason and Philosophy tear away her veil, with the sciences and arts in ranks below. Source ↗.

The same Enlightenment drive, to systematize knowledge such that it would be accessible to any rational mind, was institutionalized at the turn of the nineteenth century in the archival state. In the 1790s revolutionary France turned the documents once held by the crown and church into the property of the nation. The archive in this way ceased to be the private memory of the state and became, in principle, the collective memory of the people. Though the shift from personal to institutional record-keeping was long and contested, as public officials continued to keep and sell their papers, the nineteenth century in general saw a turn toward more open record keeping. Britain, once the fear of revolution had passed, established its Public Record Office in 1838. 

Max Weber would later place the archive at the center of his account of bureaucracy and rationalization and, indeed, the Victorian period saw the archive bound to the counting state. National statistical offices began systematizing social and economic data, while colonial administrations did the same on a global scale, gathering information about subject populations in the service of control. The British Empire was, for all its later air of grand design, something of an improvisation, run by an overstretched and undertrained service that compensated for its lack of men on the ground with information, surveying, mapping, counting, listing, and sorting the results into ever-shifting classifications.

In the twentieth century, the same impulse would be expressed in a variety of forms, as successive technological innovations seemed to herald the dawn of the universal archive. In 1895, Paul Otlet and Henri La Fontaine began the Universal Bibliographic Repertory, millions of index cards intended to link all global knowledge, which by 1910 had grown into the Mundaneum in Brussels. The microfilm boom of the 1930s, as Ian Milligan recounts, raised hopes of universal access and permanent preservation. UNESCO and other postwar institutions recast the imperial ambition to survey subject populations as global cooperation. In turn the advent of computing brought new possibilities. Ted Nelson coined the term "hypertext" in his effort to create Xanadu, a free, democratic library of linked information, while the cliometricians of the 1960s digitized census and economic records in bulk.

None of these projects ever became what their founders imagined, yet each achieved more than a cautious effort would have. It is no exaggeration to say that, in archival history, hubris is the method.

A plate from Paul Otlet's Atlas Monde, drawn in the 1930s for an Encyclopaedia Universalis Mundaneum that would have rendered the whole of knowledge in diagrams. Source ↗.

To set out to assemble the textual past of the whole world is, by any sober measure, absurd. A corpus so vast will never be comprehensively gathered by anyone. But such hubris is methodologically valuable. Where a project scaled to what is immediately reachable builds only the tools that task requires, an all-encompassing one must confront problems that a narrower effort would simply design around. The apparatus built to chase such an impossible goal can long outlast the dream that inspired it. Otlet's Mundaneum collapsed, but the Universal Decimal Classification he developed to organize it can still be found on library shelves around the world. Even if a universal archive is never assembled, what gets built along the way can dramatically expand the possibilities of historical research.

Still, some sobriety is in order. If the turn of the twenty-first century does herald the beginning of a more truly global archive, it is worth recalling just how many times that moment has already seemed to arrive. A crowdsourced archive on this scale would require an intermediary with an unusual combination of qualities to hold it together. It would need technical, linguistic, paleographic, and historical expertise, durable funding and stewardship, serious attention to rights, ethics, and data security, as well as the ability to keep archives, scholars, and communities coordinated over decades. It would also need the thing such efforts most often lack: an incentive structure that makes sustained participation worthwhile for the individual scholar and archivist, rather than relying on goodwill or the rhythm of grant cycles.

What makes the dream more plausible today than at any other point in history is that the most stubborn obstacles are finally giving way. Digitization and mass photography now make it possible to crowdsource images of archival material at low expense; machine transcription can turn those reproductions into usable text; and interoperable frameworks and semantic tagging promise to weave the results into a single, searchable whole.

Leo is moving toward this goal in four steps. First, we are coordinating the crowdsourced gathering of images of archival material, to break the most basic bottleneck in bringing the world's textual heritage online. Second, we are transforming those images into machine-readable text using our state-of-the-art transcription model, which gives researchers a reason to upload their materials in the first place. Third, we are building a platform for institutions, archivists, and historians to correct those transcriptions, collaborate on assembling collections, and reconcile metadata and description across incompatible standards. Fourth, we are planning the semantic, statistical, and visualization layers that will let scholars trace patterns of movement, exchange, and transformation across time and space.

The vast quantities of material that would constitute a universal archive must first of all be brought together. Digitization makes this possible, but since the advent of the Internet, only a tiny share of the world's archival heritage has been brought online. Traditional digitization is slow and expensive, requiring specialist departments, professional imaging rigs, high-resolution scans, and, for most of the cultural heritage sector, a reliance on public-private partnerships to foot the bill. A euphoric wave of digitization work in the 1990s and 2000s, much of it driven by improvements in optical character recognition for printed sources, brought vast quantities of books, pamphlets, and newspapers online, but it also produced the fragmented landscape we now inhabit. The result is a patchwork of vendors and genealogy companies whose holdings sit in walled-off silos, the majority of them behind paywalls.

The opening comes from the advance in portable photography over the past decade. The camera in an ordinary smartphone is now more than good enough to capture every character on a typical page, and many historians already hold large personal collections of such images, accumulated over years of archive visits. Simply by uploading images they already have, scholars can gradually build an enormous shared archive of primary sources.

Though some altruistic scholars may digitize manuscripts purely out of generosity, most are only likely to do so with the prospect of getting something tangible in return. To incentivize users not just to upload their material for private use, but to publish it for the benefit of others, we have launched the Leo Transcription Grant, offering free transcription credits in return for releasing the images and corresponding transcriptions without copyright or restriction. 

The key, then, is to offer fast and accurate transcriptions for the widest possible range of source material. Our state-of-the-art model handles both printed and handwritten text without requiring users to train it themselves or to wrestle with preprocessing and segmentation. Its scope, for now, is limited to Latin scripts from roughly the past five hundred years, including English, French, Italian, Dutch, Spanish, and German, but not yet Greek, Arabic, Hebrew, Cyrillic, or Indic and East Asian scripts. The plan is to put the foundations in place before widening the scope. Our immediate priorities, on which everything else depends, are crowdsourcing collections and building the machinery to keep improving the transcription model. That is why we are focusing our energies on making Leo as user-friendly and powerful as possible, encouraging scholars to correct transcriptions within the web app and thereby generating a continuous flow of data that can be used to fine-tune the model.

Gathering and transcribing images is the foundation, but a heap of images and transcriptions is not yet an archive. The next step is the hard problem of imposing order on such an unwieldy mass of material.

Traditionally, the answer to the question of archival organization has been systematic arrangement according to some central schema. The cabinet improved upon the medieval chest by introducing doors and drawers that mirrored the divisions of knowledge. But where such approaches imposed a single source of order, under the physical constraint that a document could occupy only one position, digital reproductions allow for more flexibility. More than that, once an image has been transcribed, the text becomes a source of structure in its own right. A model can extract the names, dates, and places it contains, generate descriptive metadata, and align it with what other collections already hold.

A cabinet of hooks, each labelled with a subject heading, on which a scholar hung slips of paper excerpted from his reading. From Vincent Placcius, De arte excerpendi, 1689. Source ↗.

What this makes possible is organization through interoperability rather than centralization, where separate collections, in different languages and formats, are made to speak to one another. Perhaps a better word, for humanists, would be translation. If Leo can act as such a mediating layer, it can become the connective tissue that Manning imagined for making global history possible.

In practice, rendering diverse collections interoperable is its own immense challenge, not least because the drive to standardize has itself bred a thicket of competing schemes. The Text Encoding Initiative (TEI), the standard for marking up archival text, was designed for interoperability, yet it is so verbose and so variously applied that one TEI file is often incompatible with the next, and any tool meant to work across corpora must first normalize them. One possible approach is to keep TEI at the center for archival fidelity while pairing it with lighter, machine-friendly formats for analysis and exchange, with linked-data ontologies such as CIDOC-CRM for the semantic web, and with IIIF for interoperable images, and then to map archival description standards onto a shared core rather than forcing every collection onto a single standard. Here too machine learning will play a critical role. The same kind of model that turns an image into text can translate between schemas, automating much of the mapping between incompatible standards that once had to be done by hand.

While such minutiae still need to be resolved, we see the path forward as a minimal shared core with automated integrations between standards, creating enough common ground to make collections legible to one another without forcing each to abandon its traditional indexing. Models for such an approach already exist, for instance in the Native Northeast Portal, formerly the Yale Indian Papers Project, which draws together primary sources relating to First Peoples from institutions across the northeastern United States, reorganizing them around Indigenous polities and concerns while preserving each document's reference to its original archival home. In doing so it repairs connections that earlier collecting practices had severed and helps reconcile the work of community historians with their professional counterparts.

The task presents most difficulties where resources are limited. There are already aggregators that normalize metadata across collections, including Archives Hub in Britain, the Digital Public Library of America and ArchiveGrid in the United States, and Europeana and Archives Portal Europe on the continent. But many regions have no comparable center of gravity. One task for a universal archive is to connect those aggregators where they exist and to stand in where none do.

Finally, on top of all this sits the layer that makes the archive usable as evidence at scale: the standard apparatus of corpus linguistics, including frequency lists, keyness statistics, n-grams, collocations, and concordances, as well as semantic tools that extract entities and link them into knowledge graphs. This demands more engineering work than simply calling a large language model, because historians need tools that keep provenance in view. A navigable graph of evidence has to be traceable back to the transcriptions and original images, rather than serving up plausible answers whose workings are hidden.

Tagging is where this apparatus begins. To mark a span of text as a person, a place, a date, or a crime is not simply to label it but to make an interpretive claim, and the categories that matter are seldom given in advance. What one historian needs to identify—whether it is a kind of dwelling, a category of offense, or a form of kinship—another will have no use for, and a system that ships with a fixed vocabulary forecloses the possibilities that such an archive is intended to enable. A tagging feature should let scholars define their own categories rather than choose from a menu, and attach attributes beneath each tag. That way, a single person can be linked simultaneously to a stable set of facts, such as a date of birth or an occupation, while recurring across documents in different roles, for instance as a plaintiff in one and a defendant in another. Extracting persistent identifiers for people, places, and events means that the same person mentioned in a Flanders parish register might be traced again in a Bahia ship's log.

The components of such a system already exist, but no single tool yet brings them together. Transkribus lets users export tags together with their surrounding text. OCHRE, at the University of Chicago, stores material in a graph rather than a relational database, giving it the flexibility to model the tangled, many-to-many relationships that historical records contain. What none yet offers is meaningful assistance from AI tools. Every tag must still be placed by hand, which leaves tagging, like traditional approaches to imaging, transcribing, and cataloguing archival materials, impossible to accomplish at scale.

The long-term vision for Leo is thus not just to assemble, transcribe, and organize archival materials, but also to make sense of them. Here, even more than in transcription, the model cannot be left to work alone. Because a tag is an interpretation, the historian's judgment is a central part of the process. To work well, the model will require constant human input as feedback to sharpen its suggestions. Such an approach keeps every claim traceable to the transcription and image beneath it, holding the work to the discipline's standards of evidence, while relieving archival historians of the drudgery of sifting through countless sources by hand and freeing them for the interpretive work that only they can do.

To propose assembling the textual past of the entire world may be immodest to the point of absurdity. The work will never be finished, but it is worth attempting anyway. It is in that spirit that we introduce Leo Collections. Our first public collection, ExLatinis, sets out to translate every Latin work printed in Europe between 1450 and 1750, a small but concrete start on a very long road. You can read more about that project here.

If you have images of archival material, apply for a Leo Transcription Grant and we will transcribe them in exchange for your releasing the images and transcriptions in the public domain. Or if you have another idea about how to get involved, get in touch at jon@tryleo.ai.

Leo Collections is a crowdsourced, machine-readable archive of the world's textual past—gathering, transcribing, and connecting scattered documentary heritage.

Leo Collections: Our Plan to Build a Universal Archive

Thiago Krause is a historian of Brazil and the Atlantic world at Wayne State University, where he writes about slavery, inequality, and commerce between 1450 and 1850. Born and raised in Rio de Janeiro, he earned his Ph.D. from the Universidade Federal do Rio de Janeiro in 2015 and taught at UERJ and Unirio before moving to Detroit. He is currently writing a global history of Salvador da Bahia, reconstructed from more than a hundred archives across Europe, the Americas, and beyond — a project that has made him an early and unusually sophisticated adopter of handwritten text recognition and AI-assisted archival research.

João Teixeira Albernaz, “Planta da Restituição da Bahia,” 1631, Wikimedia Commons.

To set the stage for our readers, could you introduce yourself and tell us about your research? What drew you to this specific topic?

I am a historian of colonial Brazil and the early modern Atlantic world. My current book project is a global history of Salvador da Bahia from the sixteenth century to 1763, when it ceased to be the capital of Portuguese America. I am interested in Salvador not only as a Brazilian city, but also as one of the central hubs of the early modern world: a port with direct and indirect connections to American plantations, African slave markets, European consumers, Asian textiles, Indigenous North America, the Mediterranean, the Río de la Plata, and the Indian Ocean.

The book does two things at once. First, it restores Salvador to the place it occupied in its own time. For much of the seventeenth and eighteenth centuries, it was one of the largest cities in the Atlantic seaboard of the Americas and its most important slaving port. Second, it uses Salvador to rethink larger histories of globalization, capitalism, consumption, and slavery. I argue that early modern globalization was not simply directed from London, Amsterdam, Paris, or Lisbon; it was equally shaped by colonial merchants, African rulers and consumers, Indigenous traders, enslaved workers, imperial officials, and many others whose choices and coercions structured commercial life.

What drew me to the topic was, in part, a mismatch between historical importance and historiographical visibility. Salvador appears everywhere once one starts looking across archives: in French merchant correspondence, English commercial papers, Dutch company records from West Africa, Spanish American criminal investigations, Genoese notarial deeds, Portuguese imperial documentation, and even sources about Macau and China, among many others. Yet colonial Salvador is still much less visible than it should be, especially outside Luso-Brazilian historiography. I became increasingly interested in using one place to write an empirical global history, rather than writing a largely synthetic overview in which places disappear into abstract networks.

A recurring example is Brazilian tobacco. It was cultivated and processed in Bahia by enslaved workers, but became desirable in West Africa, Spain, Italy, Central Europe, China, and among Indigenous consumers in North America. Its history shows how a commodity could travel globally not only because Europeans imposed it, but because consumers in many places developed specific preferences and refused substitutes. At the same time, those preferences were tied to coercion: tobacco helped purchase enslaved Africans, who in turn grew that same crop. That tension, between choice and violence, between local agency and structural domination, is at the center of my work.

When you presented your job talk a few years ago, your research already drew from 50 archives. Now that number has expanded to well over 100 scattered repositories. Could you describe what a typical historical research workflow looked like before you integrated tools powered by recent developments in AI? What were the main bottlenecks when dealing with thousands of pages of handwritten archival records?

Before these tools, the workflow was much more linear and much more constrained by human reading time. I would identify a promising archive, series, or volume; travel there or use digitized material; photograph or download as much as possible (or, until a few years ago, transcribe and take notes in situ); organize the images manually; and then inspect the material page by page, taking notes in Word. If a volume looked promising, I might spend days or weeks reading it. If it looked only vaguely relevant, I often had to make a judgment call: invest the time and perhaps find nothing, or move on and risk missing something important.

That is the basic problem of scattered archival research. To trace transimperial connections, one cannot rely on documents being indexed under Bahia or Brazil. They appear in correspondence about the French Caribbean, Dutch West Africa, English tobacco policy, Spanish American contraband, Genoese commerce, or the North American fur trade. Catalogues rarely capture that level of detail. A document can be catalogued as a routine diplomatic letter and still contain a crucial paragraph on Bahian sugar, tobacco, gold, or slaving voyages. So the bottleneck was discovery.

Handwriting compounded the problem. Early modern corpora are multilingual, inconsistent, and messy. Spelling is unstable; the same place or person may appear under several forms; and a single “item” can contain a letter, enclosures, summaries, copies of earlier correspondence, and later docketing. Before HTR and recent LLM tools, the only reliable method was slow reading by someone with enough paleographic and historical knowledge to know what mattered.

The result was a permanent tension between breadth and depth. I could read closely, or I could cover more ground, but doing both across multiple countries and dozens of archives was extremely difficult. Digitization had already changed the scale of what was accessible, but it created a new problem: there were suddenly far more images than any single historian could read. LLMs do not solve the problem of interpretation, but they have changed the scale at which initial discovery and triage can happen.

From bespoke models to out-of-the-box HTR

You mentioned that you spent two years working primarily with Transkribus, training bespoke machine-learning models for difficult handwriting; the process required upfront work on layout recognition but paid off for highly complex scripts. More recently, though, you adopted Leo for the bulk of your Latin-script documents. Could you talk about what prompted that shift? From a practical standpoint, what are the distinct advantages of moving away from training custom models toward a platform that reads Latin-script documents immediately out of the box?

Transkribus was essential for me because some of my sources are genuinely difficult. When the handwriting is especially hard, bespoke models can still be extremely useful. The most important examples for my research are Dutch and Brazilian notarial records, along with one particular set of merchant correspondence. LLMs still cannot read those reliably. For difficult scripts, training a model can be worth the investment, particularly if it is a collective enterprise dedicated to a large collection.

But the investment is real. You need to segment pages, correct transcriptions, train, test, retrain, and then repeat the process for a different hand or archive. That makes sense when the corpus is bounded and difficult enough to justify the cost. It makes less sense when you are dealing with tens of thousands of documents in many languages and from innumerable hands, where the main problem is not achieving a perfect transcription of one corpus but making a vast amount of material searchable quickly enough to know what deserves close attention.

That is what prompted the shift. For the bulk of my materials, Leo lowered the threshold between image and usable text. Instead of asking, “Can I train a model for this archive?” I can ask, “Can I process this corpus now, preserve the images and metadata, and make it searchable enough for discovery?” This is a different research regime, one where perfection matters less than discoverability. After all, I still have to read and check the images for every document I am going to use, to be sure of what they actually say.

The practical advantage is speed: not the speed of skipping the reading, but the speed of getting from a box, a volume, or a folder to a searchable research environment. That matters enormously for scattered research. I do not need every line to be perfect in order to identify a memorandum about Bahian tobacco in Genoa, a French letter about smuggling in Salvador, or an English comment on Brazilian sugar. I need a reliable enough first pass that points me toward documents worth human verification. The result is that I work with a much larger source base, and in the end spend more time reading manuscripts than I ever did.

Again, I would not frame this as replacing custom models entirely. For some material, especially difficult hands or non-standard layouts, bespoke training remains important. But for large bodies of Latin-script material, an out-of-the-box system changes the economics of research. Transkribus has its Titan II model, which is quite good, but Leo usually produces more readable output and does not require separate layout analysis, so it is generally more efficient. It allows the historian to reserve the most intensive labor for interpretation, verification, and close reading, rather than spending months merely creating the conditions for search, as I did for the first seventeen years or so of my career.

Many scholars remain hesitant to build large digital corpora because they assume the technical or computational hurdles are still too high. In your experience, how does having access to a platform that mostly works out of the box change what a single project can achieve? What shift in thinking is required for historians to take advantage of this?

The main change is that large corpora become feasible for individual scholars, not only for funded teams. That does not mean every historian should build an enormous corpus, or that scale is automatically good. But it does mean that the old assumption, that a single scholar can only work with what they can personally read line by line from the start, is no longer quite right.

For my project, this has meant that I can work at two levels simultaneously. I can build a broad corpus across many archives, languages, and imperial contexts, and then use tools to identify the smaller subset of documents that require traditional historical attention. The machine helps with reconnaissance. The historian still has to decide what the corpus is, what counts as relevance, how to handle uncertainty, what must be checked against the image, and how a document fits into a larger argument.

The required shift is to stop thinking of HTR as either a perfect transcription or a failure. For many research purposes, it is better understood as a probabilistic finding aid. It lets us ask better questions of uncatalogued or poorly catalogued material, but it does not remove the need for archival judgment: machine output should function as a map of possible evidence, not as evidence itself.

There is also a methodological shift. Historians need to think more explicitly in terms of bounded corpora, provenance, metadata, selection criteria, and verification. What exactly did I process, and what did I exclude? Which languages or hands are underperforming, what kinds of errors is the model likely to make, and what would count as a false negative? These questions are not alien to historical research; they are versions of source criticism. But digital corpora force us to make them more explicit.

In that sense, the most important change is intellectual, not simply computational. These tools make it possible to write histories that are less constrained by national archives and imperial catalogues. But they only do so when historians design the corpus carefully and resist the temptation to treat search results as transparent access to the past.

Querying uncatalogued material with AI agents

You recently used Leo to transcribe around seven thousand merchant letters, but this is more than you could possibly have time to read individually. Could you walk us through how you combine handwritten text recognition and AI agents to query uncatalogued material? How does this capability change the possibilities of archival discovery, particularly for scattered or poorly indexed collections? And what are the risks to be aware of in designing these processes?

The workflow begins before transcription. First, I try to define a bounded corpus: a particular archive, series, volume, merchant collection, date range, or set of images. I want to preserve original order, image numbers, and archival metadata, because without provenance the results become much less useful. Then I transcribe the material with Leo, export the output, and use LLM agents to index it in a spreadsheet.

The agents are not asked to “write the history.” They are asked to help identify candidates. For example, I might ask them to highlight letters concerning Brazilian tobacco, sugar prices, West African trade, Portuguese merchants, contraband, Bahia, Pernambuco, Lisbon reexports, or specific people and places, always with variant spellings and multilingual terminology in mind. A useful output is a table of candidate letters, not a synthetic essay: dates, image references, snippets, confidence, and a short explanation of why the item may matter.

Likewise, I’ve recently begun using agents to work through hundreds of thousands of images in multiple collections, identifying possible candidates for transcription and examination. I’ve lost count, but I’m sure they have processed more than 300,000 images in three months. The collections include Portuguese Inquisition denunciations and imperial records, French diplomatic and imperial correspondence, Antwerp notary books, Dutch company records, and more. I’m doing that because the cost of transcribing all those images with Leo would be astronomical, and visual triage costs only roughly 1% as much; I will nevertheless use Leo to transcribe the tens of thousands of selected images.

Then comes the essential step: I return to the document. I check the image, the HTR, the surrounding pages, and the document boundaries. Is this really one letter, or is there an enclosure? Is the key phrase in the body of the letter or only in later docketing, and did the model misread a place name? Does the passage actually concern Brazil or is it a false positive? Only after that do I decide whether the document enters my research notes, databases, or selected corpus for close reading.

This changes archival discovery because uncatalogued material becomes queryable at a conceptual level. Merchant letters are a good example. A catalogue may tell you the name of a merchant house and the dates of its correspondence, but not that three letters in the middle of a volume discuss Brazilian tobacco in Naples, rumors about Bahian sugar in Hamburg, or attempts to bypass Portuguese restrictions in Bahia. HTR plus agents make it possible to find those needles without pretending that the entire haystack has become transparent.

Arquivo Nacional da Torre do Tombo, Lisbon, Junta da Administração do Tabaco, Livro 185: 153, Willem de Bruijn & Paulo Cloots, Lisbon, to Christopher Schutter, John Corsley and others, Bristol, January 7, 1719. In this letter written in French, two Dutch merchants mention the possibility of a slaving voyage from Madagascar to Brazil.

The risks are serious. HTR can miss the most important word on a page. LLM agents can and do overinterpret, normalize away uncertainty, or produce plausible but wrong summaries. They can also reinforce the researcher’s expectations: if I only ask for Brazil, I may miss documents that matter precisely because they describe the same circuit from an African, Spanish, Dutch, or French perspective. And absence is especially dangerous. The fact that an agent did not find something does not mean it is not there.

So the workflow has to be designed defensively. I try to keep the corpus bounded, preserve provenance, inspect results against images, sample negative results when possible, and avoid using OCR silence as evidence of absence. The point is not to automate trust but to create a better system for deciding where human attention should go.

What historians must not outsource to machines

How do you decide what to outsource to an automated system and what must stay as the province of the human scholar? If a specialized model can transcribe the manuscript and agents can find relevant passages in the document, what, in other words, is the core of the historian’s craft that we must refuse to outsource to machines?

I outsource tasks that are repetitive, mechanical, or preliminary: downloading, file organization, first-pass transcription, rough search, candidate identification, table creation, and sometimes the extraction of names, dates, and places. These are labor-intensive tasks, and historians have always used tools for them: calendars, indexes, catalogues, research assistants, databases. AI changes the scale and speed, but not the basic distinction between assistance and judgment.

What I do not outsource is the decision about what a document means. A model can identify a passage about tobacco, but deciding why that passage matters for the history of Atlantic slavery, consumer preference, or Portuguese imperial governance takes the historian’s framework. The same is true of a reference to Bahia: nothing in the model reliably distinguishes a passing mention from a document that matters analytically. And a model can summarize a letter without grasping the archive’s silences, the institutional context of the collection, or the historiographical stakes of the evidence. Despite improvements in internal knowledge and context windows, these systems remain far from being able to hold all the context historical research requires.

The core of the historian’s craft is judgment under conditions of incomplete and biased evidence. Historians decide which questions are worth asking, which archives might answer them, what counts as evidence, how much weight to give a source, and how to build an argument without flattening contradiction or uncertainty. That requires knowledge of language, paleography, institutions, historiography, and human behavior. It also requires ethical judgment, especially in histories of slavery, where the archive was often produced by enslavers, merchants, judges, and imperial officials.

There is also something more basic: historians must preserve the capacity to be surprised by the archive. LLMs are trained to identify patterns and tend to average things out, so they often reproduce what is already known. Automated systems work best when we define what we are looking for. But some of the most important discoveries come when a document does not fit the categories we brought to it. If we only use machines to retrieve expected answers, we risk turning archives into confirmation devices. The scholar’s role is to keep the research open to anomaly, context, and contradiction.

So I do not see the line as “machines transcribe, humans write.” It is more fundamental than that: machines can help us move through evidence, but they cannot assume responsibility for historical understanding.

You’ve argued elsewhere that these tools augment the work of those who already possess historical judgment, but risk producing atrophy in students who are still building those skills. How can we teach historical research in a way that makes use of these new technologies while preserving the learning process?

I think the key is sequencing. Students can use these tools because they are becoming part of historical research, though many excellent historians choose not to and will continue not to; that is no judgment on their work. But students should not encounter them first as shortcuts. They need to learn what the tools are shortcutting.

That means students should still do basic archival work manually: read manuscript images, struggle with handwriting, identify document boundaries, compare catalogue descriptions with actual documents, transcribe a page, build a small source table, and explain why a document is or is not relevant. Only then does it make sense to introduce HTR or LLM agents and ask: what did the tool get right, what did it miss, and why?

One useful assignment would be comparative. Give students a small manuscript corpus. First, have them inspect it manually and build a basic index. Then have them run HTR or query LLM-generated transcriptions. Finally, have them compare the results: false positives, false negatives, mistranscriptions, missed marginalia, misunderstood names, and misleading summaries. The point is not to make them distrust the tools but to help them understand the tools as fallible instruments that require historical supervision.

We should also teach prompt design as a form of methodological design. A prompt is not just a technical instruction; it encodes assumptions about relevance, evidence, categories, and exclusion. If students ask a bad question, the machine may give them a tidy but useless answer. So they need to learn how to define a bounded corpus, specify inclusion and exclusion criteria, preserve provenance, and require citations back to images or documents.

The danger is that students may confuse fluency with understanding. A generated summary can sound authoritative even when it rests on a misread word or a shallow grasp of context. To prevent that, I would require students to cite original documents, not LLM outputs; to verify claims against images; and to explain uncertainty rather than hide it.

In the end, these tools should be taught as part of archival literacy, not as a replacement for it. The goal is not to train students to avoid the hard parts of historical research but to help them reach more sources while becoming more explicit, disciplined, and critical about how they use them.

A historian working across 100+ archives on how HTR and AI agents surface documents no catalogue indexes — and why he now reads more manuscripts, not fewer.

Thiago Krause on HTR, AI Agents and Archival Discovery

“A Probabilistic Finding Aid”: How Professor Thiago Krause is Using Leo for a Global History of Salvador da Bahia

We’re offering up to 100,000 free credits for students, academics, and archives to use Leo, our AI-powered transcription and document management platform

Leo is a web app designed for historians, archivists, and other researchers working with handwritten manuscripts and printed text in Latin scripts (including English, French, Italian, Dutch, Spanish, German, etc.) from the later medieval period to the present. It uses cutting-edge AI to transform images of historical documents into plain text that’s accessible, searchable, and ready for advanced analysis. Beyond transcription, it also provides an intuitive environment for storing, editing, organizing, searching, and exporting documents. For more information, click here.

This grant is open to everyone*: undergraduates, graduate students, faculty, independent researchers, research groups, libraries, archives, and cultural organizations. We welcome proposals for projects of all scales, from smaller initiatives requiring only a modest number of credits to larger undertakings that make use of the full allocation.

Each grantee will receive up to 100,000 credits, in addition to a 24-month subscription package with storage to host those images. As every credit transcribes a single image, whether of a page or a double-page spread, awardees will be able to transform as many as 200,000 pages of handwritten manuscripts into searchable plain text. They will also gain full access to Leo’s suite of tools to organize, edit, manage, and export their collections. 

*There is only one condition of receiving a Leo Transcription Grant. The awardee must publish the transcription output, either on our platform or another they choose, within 24 months of receiving the grant. The transcriptions along with the corresponding images must be made available online free of restriction and copyright (CC0/ equivalent public-domain open license).

Entries will be evaluated on their scholarly quality, originality, and for potential impact for unlocking new insights about the past. We’re especially interested in projects that extend access to the historical record, demonstrate thoughtful engagement with new technologies in the service of scholarship, and/or that would have been difficult or impossible to accomplish without our support. 

LTGs may be for academic, charitable, or business purposes. Entrants will be asked to complete a short form, providing their name, institutional affiliation (if applicable), a brief project proposal (up to 500 words), which explains how their project will contribute to historical scholarship or public understanding of the past, as well as a short description (up to 100 words) of the materials they intend to transcribe.

Awards will be made on an ongoing basis. We will follow up on your application and/or provide you with a final decision within one week.

Please note that printed materials are not eligible for this prize and that awarded credits are non-transferable. To enter the competition, please complete the form here.

Leo Transcription Grants: apply for up to 100,000 free credits

We’re excited to announce that Leo has won an Emergent Ventures grant worth $50,000 to support the growth of the platform in the coming months.

Leo is an all-in-one transcription and document management platform designed for professional historians, archivists, and researchers working with handwritten manuscripts and printed text in Latin scripts from the later medieval period to the present. It uses cutting-edge AI to transform images of historical documents into plain text that’s accessible, searchable, and ready for advanced analysis. It also provides an intuitive platform to store, organize, edit, and search documents, with file structure management, advanced search, and metadata filtering capabilities.

Emergent Ventures is a fellowship and grant program administered by the economist and talent scout Tyler Cowen at the Mercatus Center. It “supports entrepreneurs and brilliant minds with highly scalable, ‘zero to one’ ideas for meaningfully improving society.” The program backs focused, high-upside projects around the world, favoring speed, scrappiness, and ambitious outcomes.

Expand training data across scripts, languages, and periods

Train our model for more epochs to improve accuracy and robustness

Introduce semantic search using vector embeddings

Add interpretation features, allowing users to translate, summarize, and interpret transcripts

Allow users to fine-tune our model on corrected transcripts

Improve onboarding and documentation to make the platform easier to use

Fix bugs and smaller issues on the web-app

We’ll also invest in marketing, including paid advertising, community seeding, and a brand relaunch that leverages our lion mascot as a bumbling yet enthusiastic palaeographer for reach on social media.

To date, we have fully bootstrapped the venture, investing personal capital to hire developers and cover computing costs. With the help of this generous grant, we’ll be able to continue to develop Leo into a self-sustaining business, without losing the speed and operational agility that got us here. Thank you to Emergent Ventures for their support!

We’re excited to announce that Leo has won an Emergent Ventures grant worth $50,000 to support the growth of the platform in the coming months

Leo wins Emergent Ventures grant to accelerate development and growth

Fewer hallucinations: Leo now more consistently avoids transcription errors like repeated or invented text, especially at the end of documents

More complete outputs: Significantly fewer blank pages and skipped lines, giving you fuller, more faithful transcripts

Smarter formatting: Enhanced recognition of structured layouts, including tables, margin notes, and other non-linear text features

Better with books: Improved performance on double-page spreads and printed texts that previously caused issues

Support for later medieval manuscripts (c. 1100–1500) written in Latin scripts

Improved transcription accuracy for modern European languages, including French, German, Italian, and more

More robust handling of damaged, distorted, or rotated images, enabling reliable transcription even in more challenging cases

ATR-1 is now the default model on Leo. You can transcribe new or existing images using it by hitting “Transcribe” as usual. To get started, click here to access your Leo dashboard.

We’re excited to launch our most advanced model yet. ATR-1 sets a new benchmark for automated text recognition, transcribing handwritten and printed text with unprecedented accuracy and reliability.

Introducing ATR-1 — our most advanced text recognition model yet

ATR-1 — our most advanced text recognition model yet