Why does RAG find the right PDF but return the wrong table value?
Question from public RAG practitioner discussions (paraphrased, not a Dodo customer testimonial): Why does my RAG find the correct document but give me a wrong number or mismatch a table row?
Short answer: the retrieval system may have correctly retrieved text that was already damaged during extraction. A table header, value, unit, or footnote can lose its relationship before the embedding is created. Better reranking cannot reconstruct missing structure reliably.
What to check first
Compare the extracted table against the source page. Preserve row/column relationships, headings, units and source-page anchors. Create a small gold set of questions that require the right cell, not merely the right page. Inspect your retrieval results and evaluate source fidelity separately from answer fluency.
How QualityRag changes the problem
Our research explores typed semantic representations and explicit PackSpec information requirements. Rather than assuming a high similarity score proves completeness, PackSpec can declare that a particular answer requires a specific table value, its label, unit and applicable exception. Missing evidence leads to INCONCLUSIVE rather than an unsupported admission.
The existing document adapter indexes PDF selectable text, DOCX paragraphs, HTML, CSV, JSON and subtitles with source anchors. It does not yet guarantee table layout preservation, OCR, or semantic correctness for arbitrary PDFs. That gap remains an explicit research objective.
The larger future
We are working toward information retrieval that can retain semantic relationships across documents, images, video frames and structured records. Semantic Codex and an evolving grammar are intended to transform representations without silently discarding what matters for a question. That is a research direction, not a claim of universal cross-media semantic equivalence today.
Original public discussion: https://www.reddit.com/r/Rag/comments/1vejp3f/what_breaks_first_when_rag_moves_from_demo_pdfs/
Related structured-data discussion: https://www.reddit.com/r/Rag/comments/1pfuxis/why_rag_fails_on_tables_graphs_and_structured_data/
Implementation boundaries: https://github.com/cyrillesaxo/QualityRag/blob/main/docs/SEMANTIC-CODEX-DOCUMENTS-V0.3.md
Explore the open source: https://github.com/cyrillesaxo/QualityRag
For one workflow with a costly retrieval failure, request a paid RAG Reliability Pilot from DODO LLC at https://www.dodollc.net/book-now.
Experience the failure in 90 seconds
Try DODO LLC's interactive QualityRag Failure Lab. Explore three evidence failures and see how the declared requirements change the outcome. The demo uses synthetic scenarios and is free, with no signup required.
Comments