Large language models have a structural problem that adding more parameters alone cannot fix: they don't know what they didn't see during training, and what they did see is frozen in time. Ask an LLM about a contract signed yesterday, a support ticket opened this morning, or your company's internal procurement policy. It doesn't know. It can't know.
That gap is precisely what Retrieval-Augmented Generation — RAG — was designed to close. Not by expanding or retraining the model, but by giving it dynamic access to external knowledge at the exact moment it responds. Since Patrick Lewis and his team at Facebook AI Research published the foundational paper in 2020, RAG has evolved from an academic curiosity into the backbone of most production AI applications.
But RAG is not a single architecture. It's a family. And understanding its variants is now a foundational skill for anyone building serious AI systems.
The core mechanism: index, retrieve, generate
In its most essential form, RAG has three stages. First, indexing: source documents are split into chunks, converted into numerical vectors via embedding models, and stored in vector databases like Pinecone, Chroma, or pgvector. Second, retrieval: when a query arrives, it's also converted to a vector and the most similar chunks are found via cosine similarity. Third, augmented generation: the retrieved fragments are inserted into the prompt alongside the original question, and the LLM responds grounded in that external context.
The result is more accurate, up-to-date responses that — when the system is well built — are traceable back to their source. On that foundation, seven distinct variants have emerged, each with a different performance profile and use case.
The seven variants: from simple to sophisticated
1. Naive RAG — The starting point
The most direct implementation of the original paper. Documents are indexed, the most similar chunks retrieved, and passed to the LLM without additional filtering. Its strength is simplicity: a working system can be assembled in hours. Its weakness is that vector search returns the nearest chunks in embedding space, not necessarily the most useful ones. With large corpora, false positives proliferate quietly. Best suited for prototypes and Q&A over small, well-structured datasets.
2. Retrieve + Rerank — The production upgrade
Adds a critical intermediate stage: a cross-encoder model that evaluates the genuine relevance of each retrieved fragment before it reaches the LLM. The pipeline first retrieves a broad candidate set (20–50 chunks), then reranks them with a model that sees both query and document together, capturing subtle semantic interactions that a standalone embedding misses. Models like Cohere Rerank or BGE Reranker are common choices. For most production projects with medium-to-large corpora, this is the right baseline.
3. Multimodal RAG — When text isn't enough
An enormous amount of critical knowledge lives in visual formats: technical diagrams, financial charts, blueprints, complex tables. Multimodal RAG closes that gap through three possible strategies: converting visuals to text using vision models (GPT-4V, Gemini) at indexing time; multimodal embeddings like CLIP that operate in a shared semantic space for text and images; or passing visual content directly into the generation context when a multimodal model handles the final response. Especially high-value in manufacturing, healthcare, and engineering domains.
4. Graph RAG — When relationships matter more than similarity
Vector search assumes that relevant knowledge is whatever is semantically similar to the query. But some questions require navigating explicit relationships between entities: «what regulations affect the products distributed by supplier X?» isn't a similarity question — it's a traversal: supplier → products → regulatory category → applicable rules. Graph RAG builds a knowledge graph where entities are nodes and typed relationships are edges. A 2024 Microsoft Research paper demonstrated that this architecture consistently outperforms vector RAG on questions requiring multi-document synthesis or relational reasoning.
5. Hybrid RAG — The best of both worlds
Combines dense vector search (semantics), sparse lexical search with BM25 (which catches exact matches of technical terms or proper nouns that embeddings sometimes blur), and knowledge graph traversal. Results from all three channels are fused via Reciprocal Rank Fusion (RRF) before final reranking. The tradeoff is greater operational complexity — maintaining both a vector index and an up-to-date knowledge graph in sync. For high-value domains like legal, financial, and biomedical, that infrastructure cost is well justified by the quality gains.
6. Agentic RAG — The system that decides how to retrieve
Introduces a decision agent — typically an LLM with tool access — that analyzes each query and dynamically selects the best retrieval strategy: direct vector search, parallel multi-source queries, graph traversal, or real-time web search. It implements reasoning loops like ReAct (Reasoning + Acting): reason about what to do, act, observe the result, decide the next step. Adaptability is the key advantage; latency and token cost are the primary constraints. This is the dominant pattern in high-quality enterprise AI assistants where queries are diverse and error tolerance is low.
7. Multi-Agent RAG — Orchestration at scale
Multiple specialized agents coordinated by an orchestrator: one agent for vector databases, another for external APIs, another for knowledge graph reasoning, another for fact-checking, another for code execution and computation. The orchestrator decomposes tasks, assigns subtasks — sometimes in parallel, sometimes sequentially — resolves contradictions across sources, and synthesizes a final response. Frameworks like LangGraph, AutoGen, and CrewAI enable this pattern. This transforms RAG into full cognitive workflow orchestration. The operational complexity is considerable, but for enterprise systems integrating heterogeneous sources at scale, it's the architecture that performs best.
Which one should you choose?
The key is not choosing the most sophisticated architecture available, but the minimum one necessary to solve your problem with sufficient quality. Starting with Naive RAG is reasonable when exploring. Reaching robust production typically requires at least Retrieve + Rerank as a baseline, then adding layers — multimodality, graphs, agency — incrementally as real domain requirements demand it. A well-built RAG system answers accurately, cites its sources, scales with volume, and fails predictably when it doesn't know something. That's the standard worth building toward.
