AI / ML / Backend / Developer ToolsSHIPPED

Production RAG Agent

Hybrid retrieval, reranking & grounded generation

A production-oriented retrieval-augmented generation system built around hybrid search, reranking, grounded answers and a publicly deployed demo.

Live demo GitHubGrounded Retrieval Engineer

Overview

A production-oriented RAG application designed to demonstrate that useful LLM systems depend on retrieval quality, evidence handling and predictable backend behavior—not generation alone. The project includes a public Hugging Face demo and a reusable FastAPI backend.

The problem

Naive semantic retrieval can miss exact terminology while keyword search can miss semantic intent. The system therefore needed a retrieval pipeline that could combine complementary signals, improve ranking quality and keep generated answers grounded in retrieved evidence.

Engineering challenges

Supporting both semantic and lexical retrieval without coupling the application to one retrieval strategy.
Combining retrieval results into a stable candidate set before generation.
Improving candidate ordering with a dedicated reranking stage.
Preventing unsupported generation when retrieval evidence is weak.
Keeping runtime initialization predictable so the public demo does not repeatedly rebuild expensive resources.

Hybrid retrieval → reranking → grounded generation

The backend uses FastAPI as the application boundary. Documents are transformed into searchable chunks and represented through dense embeddings for semantic retrieval while BM25 provides lexical retrieval for exact terms and identifiers.

Results from the complementary retrievers are combined using Reciprocal Rank Fusion before a cross-encoder reranking stage produces the final evidence candidates. Generation is then constrained by the retrieved context, with an extractive fallback available when generation is not appropriate.

Documents → parsing → chunking
Chunks → SentenceTransformers embeddings → FAISS
Chunks → BM25 lexical index
Dense + lexical candidates → Reciprocal Rank Fusion
Fused candidates → cross-encoder reranking
Ranked evidence → grounded answer generation
Weak/unsupported generation → extractive fallback

Engineering decisions

Hybrid retrieval instead of vector search alone

Context

Semantic retrieval is strong for meaning but can miss exact identifiers, names and terminology.

Decision

Use dense FAISS retrieval alongside BM25 lexical retrieval.

Rationale

The two retrieval signals fail differently, so combining them improves recall across both semantic and exact-match queries.

Trade-offs

Requires maintaining two indexes.
Adds retrieval orchestration and fusion logic.

Reciprocal Rank Fusion before reranking

Context

Independent retrievers produce candidate lists with different scoring semantics.

Decision

Fuse rankings rather than comparing raw scores directly.

Rationale

RRF provides a simple score-independent way to combine complementary ranked lists before more expensive reranking.

Rerank before generation

Context

Retrieval recall alone does not guarantee that the most useful evidence appears first.

Decision

Use a cross-encoder reranking stage on the fused candidate set.

Rationale

A second-stage relevance model can spend more computation on a smaller candidate set and improve evidence ordering before generation.

Grounding with an extractive fallback

Context

A generative model should not invent an answer when the retrieved context is insufficient.

Decision

Keep an extractive response path for cases where generation is weak or unavailable.

Rationale

The fallback provides deterministic evidence-oriented behavior and makes the system more resilient to inference constraints.

Backend and retrieval stack

The reusable backend is implemented with FastAPI, SentenceTransformers, FAISS and BM25-based retrieval. The repository separates ingestion, retrieval and generation concerns so retrieval strategies can evolve independently.

Public demo

A sanitized demo corpus is exposed through a Hugging Face Space so the system can be evaluated interactively without requiring access to private data or infrastructure.

Predictable runtime behavior

The application is designed to initialize retrieval resources once and reuse them rather than rebuilding indexes or models for every request. This keeps the demo responsive and makes the backend architecture closer to a deployable service than a notebook prototype.

Grounded and sanitized by design

The public demonstration uses a sanitized corpus rather than exposing proprietary documents. The architecture also treats unsupported generation as a system behavior to control rather than an acceptable failure mode.

Evidence

What the project can prove.

DeploymentHugging Face Space

Public interactive demonstration

BackendFastAPI

Reusable API-oriented application layer

RetrievalFAISS + BM25

Dense and lexical retrieval combined through RRF

RankingCross-encoder

Second-stage relevance reranking

RepositoryGitHub

Source and implementation evidence available publicly

Lessons learned

RAG quality is primarily a retrieval and evidence-engineering problem, not just a model-selection problem.
Hybrid retrieval is valuable when a knowledge base contains both semantic concepts and exact technical terminology.
A fallback path makes an AI system more predictable when generation or inference is unavailable.
Separating ingestion, retrieval, ranking and generation makes the system easier to test and evolve.

What comes next

Add a formal retrieval evaluation suite with recall and ranking metrics.
Expand observability around retrieval latency, reranking latency and generation failures.
Add configurable chunking and retrieval strategies for different document domains.
Introduce stronger evaluation and regression datasets before each release.