PDF to Markdown for LLMs and RAG: A Practical Guide

If you're building a RAG pipeline or feeding documents to an LLM, your source material is probably full of PDFs: papers, manuals, contracts, books. And if you've ever pasted raw PDF-extracted text into a prompt, you know the problem — the structure is gone, tables collapse into word soup, and retrieval quality suffers.

Converting PDFs to Markdown first fixes most of this. Here's why, and how to do it well.

Why raw PDF text hurts LLM qualityCopy Anchor Link

PDF stores positioned glyphs, not document structure. A naive text extraction gives you:

• Headings indistinguishable from body text
• Table cells flattened into ambiguous sequences of words
• Multi-column layouts interleaved line by line
• Footnotes, headers, and page numbers mixed into the content

Every one of these degrades embeddings and confuses generation. The model can't tell a section title from a sentence, so chunk boundaries land in the wrong places and retrieval pulls in noise.

Why Markdown is the right intermediate formatCopy Anchor Link

Markdown preserves exactly the structure LLMs benefit from, in a compact, token-efficient syntax:

• # / ## headings give explicit section hierarchy — perfect for structure-aware chunking
• Markdown tables keep rows and columns aligned, so the model reads tabular data correctly
• Lists, quotes, and code blocks stay semantically marked
• No XML/HTML verbosity eating your context window

Compared to HTML, Markdown typically costs far fewer tokens for the same content. Compared to plain text, it carries dramatically more signal. It's the sweet spot.

Chapter-level splitting beats blind chunkingCopy Anchor Link

Most RAG tutorials chunk by fixed token counts with overlap. That works, but it slices through logical units — a chunk often ends mid-paragraph or merges unrelated sections.

A better default: chunk by document structure first. If your PDF becomes one Markdown file per chapter (and sections are headed properly inside each file), you can:

1. Split on headings for semantically coherent chunks
2. Attach chapter/section titles as metadata for filtering and citation
3. Keep tables and code intact instead of severed

This is exactly what PDF Chapter Markdown produces: you upload a PDF, and download one Markdown file per chapter, with headings, tables, and images preserved. Drop the files into your ingestion script and chunk from there.

A minimal ingestion workflowCopy Anchor Link

import glob

for path in glob.glob("converted/*.md"):
    chapter = path.split("/")[-1].replace(".md", "")
    text = open(path, encoding="utf-8").read()
    # Split on '## ' section headings, embed, and upsert
    # with metadata: {"chapter": chapter, "section": ...}

Because each file is already a clean chapter with real headings, even a naive heading-based splitter outperforms token-window chunking on raw text.

Practical tipsCopy Anchor Link

• Scanned documents need OCR. Text extraction on a scan returns nothing. Use an OCR-capable mode (in PDF Chapter Markdown, that's Accurate).
• Keep the Markdown, not just the vectors. Storing the source Markdown per chunk makes debugging retrieval issues far easier.
• Filter boilerplate early. Page headers/footers that survive conversion should be stripped before embedding — they're pure noise for retrieval.
• Mind the context budget. Markdown is token-efficient, but a dense table is still a dense table. Cap chunk sizes as usual.

Limits to knowCopy Anchor Link

Conversions run up to 200 pages and 50 MiB per file — enough for most books and reports. For a corpus of hundreds of documents, convert in batches and keep the chapter files organized by source document.

Convert your first PDF on the convert page — credit packs start at $9.90 and never expire for 12 months, so a single pack covers a decent-sized corpus.