Retrieval-augmented generation (RAG)

State of the art (2026)

By mid-2026, naive vector RAG is a baseline, not a finished system. The default production pattern is hybrid retrieval (semantic plus keyword) followed by a reranking pass, with agentic RAG – a reasoning loop that decomposes the query, retrieves, critiques and retrieves again – now standard for hard questions. Microsoft GraphRAG, open-sourced in 2024, earns its cost on cross-document, connect-the-dots queries, and adaptive routing of each query to the cheapest sufficient pipeline is the emerging best practice. Long context has not killed RAG: Claude Opus and Sonnet 4.6 and Gemini 3 Pro all ship 1M-token windows, yet recall degrades past roughly 600-700K tokens, so retrieval still grounds enterprise systems. Glean ($7.2B, ~$300M ARR) anchors the productised layer.

Core thesis

RAG gives a language model access to private or current information that was not in its training data, by retrieving relevant documents and passing them to the model. Tools such as LlamaIndex and LangChain build the pipelines, and vector databases such as Pinecone and Weaviate store the indexed text. Answer quality depends on how documents are split, which embedding model is used and how precise retrieval is. Retrieval does not guarantee a correct answer. Research continues: Microsoft's GraphRAG builds a graph and layered summaries of a document set to answer questions about the whole set, which simple retrieval handles poorly.