rag›response_generator.py
response_generator.py
Hints
Let's say you hire an assistant who is brilliant, but has never seen your company's documents.
You ask them a question. Instead of guessing, they walk over to a filing cabinet, pull out the folders that look related, and answer using only what's in those folders.
That's RAG, Retrieval-Augmented Generation. Retrieve first, then generate.
Now ask them something the cabinet has nothing to do with, like your parental leave policy in a country your company doesn't operate in.
The assistant still walks to the cabinet. They still come back with folders. Not because any of them are actually about your question, but because you told them to always bring back the three closest matches, and there is always a closest match to something. Even if closest means barely related at all.
They don't say none of these are relevant. They just hand you the folders and answer like the folders are the answer.
results = retriever.search("parental leave in Portugal", top_k=3)
# results always has 3 items, even when the best one scores 0.32
No error thrown. No empty list. Just three folders, and a similarity score that quietly sits at 0.32 instead of 0.88.
The pipeline above describes retrieval as "the closest-matching chunks win," simple enough on its own. What it doesn't show is what "closest" actually measures, or what happens when nothing in the corpus is truly close at all.
Embeddings are positions in space. Meaning becomes coordinates. An embedding model doesn't store what a sentence means; it converts the sentence into a list of numbers, a single point somewhere in a very high-dimensional space. Two chunks that talk about similar things end up as points sitting close together. Two unrelated chunks end up far apart. That's the entire trick behind "semantic search": you're never matching words, you're measuring distance between coordinates.
Similarity is just a number: the distance between two positions. When a question comes in, it goes through that same embedding model and becomes a point too. From there, retrieval is pure arithmetic. Measure the distance from the query's point to every chunk's point, usually with cosine similarity, and hand back whichever ones are closest. Nothing in that step asks whether "closest" is actually relevant. It's a sort, not a judgment call.
Retrieval never fails. It always returns top-k, ranked by least bad, even when everything is bad. Here's the part that trips people up: a nearest-neighbor search always has a nearest neighbor. Even if your entire corpus is about employee travel policy and someone asks about parental leave in a country you don't operate in, retrieval will still find the "closest" chunks and hand them back. The similarity score just quietly drops, say from 0.88 to 0.32. There's no exception thrown, no empty list, nothing that looks like failure from the outside. An irrelevant match comes back through the exact same code path as a relevant one.
The trap: guarding against empty results doesn't guard against wrong results. Anyone who's been burned by a null or an empty list before writes if not chunks: refuse on reflex. It never fires, because retrieval essentially never returns an empty list. It doesn't have to. The chunks are always sitting right there, ready to be handed to the model. They're just the wrong chunks. What you actually need to check isn't whether a match exists. It's how good the best one really is.
Don't take our word for it. Try out queries in the visualizer.
Interact with the 2D embedding space. Select a query preset and watch the nearest-neighbor search run. Slide the threshold to filter out weak matches.
Select the "Portugal Leave" query preset and drag the slider to filter out weak matches to unlock the problem stage.