A financial-document RAG pipeline (text + tables), evaluated end-to-end on 7,318 real filings and validated through paired statistical testing (McNemar, power analysis) — not intuition.
MIT License245 unit tests, 19 filesZero paid API calls in CI
Retrieval accuracy over the closest published comparison
Architecture
IngestionFinQA + ConvFinQA + TAT-DQA
→
Chunking + enrichmentClaude Haiku 4.5, optional
→
EmbeddingVoyage-4, per-dataset routing
→
Hybrid retrievalMongoDB Atlas $rankFusion, top-50
→
RerankingCohere Rerank v4.0 Pro, top-5
→
GenerationClaude Sonnet 5
→
LLM-judge evaluationClaude Sonnet 5 + is_close_v2
Every component (reranker, enrichment, judge model, embedding routing, pool size) is switchable via config, not hardcoded — validated at startup by a config schema.
Retrieval vs. published work
Configuration
Akarsu et al., Recall@5
This project, Recall@5
Hybrid RRF (BM25 + dense)
0.695
0.808
Hybrid + Cohere Rerank v4.0 Pro
0.816
0.944
Same dataset (T²-RAGBench, 23,088 queries / 7,318 documents) as Akarsu et al., arXiv:2604.01733 — the closest available apples-to-apples comparison for this project's retrieval numbers.
Honest limitations
Research/evaluation-grade, not a production system: no query-time source classifier, serving API, or monitoring beyond what's reported here. Per-dataset embedding routing improves retrieval but its end-to-end effect isn't yet statistically confirmed at the current sample size. Contextual enrichment's gain did not replicate at full scale once a reranker was already in the pipeline. Full account, every number's sample size, and what's still open: see the README on GitHub.