What is retrieval-augmented generation (RAG)?

Technical definition

RAG combines a retrieval step with generation. Source content is split into chunks and converted to embeddings, numeric representations of meaning, stored in a vector database. At query time the question is embedded, semantically similar chunks are retrieved, and those chunks are supplied to the model alongside the question.

The model then answers from provided context rather than from training alone. This grounds the response in specific, current, and private material, and allows citation of the source.

In plain terms

RAG means giving the AI your documents to answer from.

Without it, a model answers from what it learned in training, which does not include your prices, your policies, or your product. With it, the AI looks up the relevant part of your material first, then answers.

Why it matters

RAG is the difference between an AI feature that is useful and one that is a liability.

A model asked about your return policy without RAG will produce a confident, plausible, invented policy. It has no way to know yours and no way to signal that it is guessing. Grounded in your actual policy document, it answers correctly and can point at the source.

The other reasons it matters: your data stays where it is rather than being used to train a model, updating the answer means updating a document rather than retraining anything, and the citation gives a user a way to check.

What determines whether a RAG system works is retrieval quality rather than the model. If the retrieval step returns the wrong three paragraphs, a good model gives a good answer to the wrong material. Chunk sizes, how content is structured, and testing against real questions matter more than model choice.

And one thing RAG does not do: it does not stop hallucination. It reduces it substantially by giving the model the right facts. A model can still misread or overreach on what it was given.