RAG, end to end

Ingest, chunk, embed, retrieve, re-rank — and how to know it's actually finding the right thing.

0/4
☰
RAG · architecture

RAG is two jobs: file your documents ahead of time, then fetch the few passages that matter for each question. When an answer is wrong, check the second job first: it is the cheapest to inspect.

+ The whole system in one paragraph

The model only knows your data if it's in the prompt. RAG (retrieval-augmented generation) is the machinery that puts the right pieces there. It runs in two phases: ingestion, done ahead of time (parse, chunk, embed, index), and query time (rewrite, search, re-rank, build the prompt). This page builds it up one piece at a time, shows where each one fails, and ends with how to measure retrieval separately from the answer.

In this lesson

4 chapters · ~7 min · a quick check in each

Everything is on this page — just scroll down ↓

New to this? Warm up first+ optional
2
At a glance

Words you'll meet

Skim these now and come back whenever a word shows up.

ⓘ Details & sources

The terms, in plain words.

RAGRetrieval-augmented generation: look up theright passages first, then answer from them.ChunkA small piece of a document, cut so each one isabout a single thing.EmbeddingA list of numbers that captures a text'smeaning; similar meanings get similar numbers.Vector indexA store that finds the chunks whose numbers areclosest to the question's.BM25Classic keyword search: it finds exact wordsand phrases that meaning-search can miss.Hybrid searchRunning meaning-search and keyword searchtogether and merging the results.Re-rankerA second model that re-orders search results byhow well they answer the question.TokenA small piece of text, a short word or part ofone. Models read and are billed in tokens.Agent (in Uber's post)Uber calls each single-job model step an agent,e.g. one that rewrites the question. (Theagents topic uses the word for a model thatpicks its own steps.)Top-kHow many of the best results you keep: the top20, the top 10.Recall@kOut of the passages that should have beenfound, how many are in the top k.MRRHow high the first right passage sits in thelist: 1st place counts more than 5th.Golden setReal questions with the passages and answersthat are known to be right.LLM judgeA model that grades another model's answersagainst a rubric or a reference.
3
Side by side

Do you even need RAG?

Anthropic's 2024 guidance: under ~200K tokens (~500 pages), just put it all in the prompt. Windows are bigger today, but cost and quality still favor retrieval as data grows.

ⓘ Details & sources

Decide this first: it decides whether the rest of this page applies to you.[2]

Everything in the promptKnowledge under ~200K tokens (Anthropic,2024)Nothing to index or keep in syncPrompt caching (the provider reuses aprompt it already read) cuts repeat costThe model sees all of itvsRetrieval (RAG)Knowledge too big for the promptChanges often: update the indexOnly relevant chunks per questionRetrieval quality to testStart with the prompt; add retrievalwhen the data outgrows it.
1
Requirements

What it must do — and how well

Yellow notes are what RAG must do; blue ones are how well. A demo only needs the yellow notes. Production needs the blue ones, and most of the engineering goes into them.

Answer from YOURdocuments.Find the rightpassage even indifferent words.Stay current whendocuments change.Say so when theanswer isn't there.Findable: the rightpassage reaches themodel.Grounded: the answersticks to it.Measured: retrievalis scored alone.Fresh: edits show upin search.
Chapter 1 of 4

The whole system

4
Pipeline

Build it up, one piece at a time

The full architecture has twelve boxes. You never build them all at once: each one answers a failure the previous version had.

ⓘ Details & sources

Start at step 1. Move on only when your own failures force you to.[2][6]

1 · Chunk + embedthe simplest RAG1tables lost2 · Better parsingkeep the structure2misses exact terms3 · + Keyword searchhybrid3right chunk too low4 · + Re-rankersharper top-k4vague questions5 · + Query rewriteask better5

5 steps. Follow the flow one step at a time — what happens first, and what comes next.

5
Architecture

RAG, on one page

Top row: ingestion, done ahead of time. Middle: the index and the golden questions, which measure the retriever on its own. Bottom row: what happens on each question.

ⓘ Details & sources

Don't memorise it. Click a box, play a flow, and see where each piece sits.[1][2][3][6]

Watch it happen:
1234measureDocuments: Everything the app should know: manuals, wiki pages, tickets, policies.Documentswikis · PDFs · docsParse & clean: Turns files into clean text. A bad parser loses tables and headings before search even starts.Parse & cleankeep tables intactChunk: Cuts each document into small pieces, each about one thing, so search can return just the useful piece.Chunksplit · add contextEmbed: Turns each piece into a list of numbers that captures its meaning.Embedtext → vectorSearch index: Where the pieces are filed so both meaning-search and word-search can find them fast.Search indexvectors + keywordsGolden questions: A test set: real questions with the passages that should be found for each.Golden questionsknown good answersQuestion: What the user typed, as they typed it.Questionfrom the userQuery rewrite: Cleans up a vague question, or splits a big one into smaller ones, before searching.Query rewriteclarify · splitRetriever: Runs both searches and merges the results into one list of candidates.Retrievermeaning + keywordsRe-ranker: A second model reads the question with each candidate and re-orders them by how well they answer it.Re-rankerscore each chunkPrompt builder: Packs the instructions, the best chunks and the question into one prompt.Prompt builderinstructions + chunksModel: The AI model. It answers from what the prompt contains.Modelwrites the answer
youyour systemoutside services dashed = our simplification
1234
Step 1 of 4

A document arrives and is parsed into clean text, keeping its tables and headings.

Curious? Go deeper+ 1 optional
6
Up close

How a search by meaning works

Each chunk becomes a point on a map where similar meanings sit close together. A question becomes a point too, and the nearest chunks are the ones retrieved.

Meaning as a maprefund policyreturn a ticketcancel farebaggage feesseat mapslounge access"Can I get my money back?"The nearest chunks go to the prompt. (tap)
Quick check

Why build RAG one piece at a time instead of all twelve boxes at once?

Pick one — or just keep scrolling, it's not graded.

Keep going — it's free

Create your free account to unlock the rest of this lesson, keep your progress and pick up right where you left off.

  1. 2Ingestion
  2. 3Query time
  3. 4Measuring
Create my free account →I already have one — log in