Retrieval-Augmented Technical Assistant

The problem

Engineers were repeatedly searching a large corpus of internal technical documentation for answers that existed but were hard to find. A plain vector search returned passages that were topically close and practically useless.

What I built

A retrieval-augmented pipeline with hybrid dense and sparse retrieval, a cross-encoder reranker over the candidates, and a LoRA fine-tune so the model handles domain terminology correctly.

How it works

• Hybrid retrieval: dense embeddings plus BM25, fused before reranking.
• Cross-encoder reranker over the top candidates — the single biggest accuracy gain in the system.
• LoRA fine-tune on domain terminology using Hugging Face PEFT.
• FastAPI service, containerised with Docker; answers cite the passages they came from.

How it was tested

Scored against a human-labelled evaluation set rather than vibes, with retrieval and generation measured separately so a regression can be attributed to the right stage.

Results

• Answer accuracy raised from XX% to YY% against the human-labelled set.
• Reranking alone accounted for X points of that improvement.
• Median response time of X ms.

What I learned

• Retrieval quality, not the generator, was the bottleneck. Time spent on prompt wording was mostly wasted next to time spent on reranking.
• Measuring retrieval and generation as one number made early regressions impossible to diagnose. Splitting the metric was the fix.