02 / Evaluated RAG engine2026
A claim verification system over scientific abstracts that enforces grounded citations and gates CI on retrieval quality.
0.94
nDCG@5
0.88
Recall@5
100%
grounded citations
Author
my role
LLMs hallucinate, especially on scientific literature. Warrant addresses this by grounding generations in retrieved evidence from the SciFact corpus. More importantly, it treats retrieval as an engineering problem: the pipeline uses a hybrid BM25 and dense embedding search with Reciprocal Rank Fusion (RRF), re-ranked by a domain-specific biomedical cross-encoder. The CI pipeline fails if retrieval metrics (nDCG, Recall) regress against human judgments.
What I did
- 01Implemented a hybrid retrieval pipeline using BM25, SentenceTransformers, and a MedCPT cross-encoder.
- 02Built a rigorous evaluation harness that scores retrieval against the human-annotated SciFact benchmark.
- 03Configured GitHub Actions to gate pull requests on retrieval quality regressions (nDCG@5, Recall@5).
- 04Enforced strict citation constraints on the LLM generation step to ensure all claims are verifiable.