Topic

Retrieval (RAG)

Retrieval-augmented generation (RAG) answers questions with a language model by first retrieving the relevant pieces of your own material. These notes cover retrieval that works on structured material such as code, contracts and catalogues, and how to know whether a change made it better.

Retrieval-augmented generation is simple to demo and hard to run, and these notes are about the hard part. Token-count chunking cuts code, contracts and catalogues into fragments that look right and miss the answer. For DevGuardian AI we parsed code into classes and methods with Roslyn, modelled its dependencies in a Neo4j graph (GraphRAG), combined semantic, symbolic and graph search, and re-indexed only what each commit changed. For Valco AI the conversational layer sits over a modelled market database rather than a pile of documents, because when the truth is numeric, retrieval over text produces confident, wrong answers. Another note proposes documentation in which every statement carries its evidence and one of three labels: observed, documented or inferred.

Whatever the material, retrieval quality is measured with evals: a gold set of real questions with known answers, scored before and after every change. Our RAG and knowledge systems work starts the same way, with a two-week retrieval assessment on a sample of your material.

RAG and knowledge systems

Working on one of these problems?

The patterns in these notes are the ones we use on client work. Tell us what you are building.

Book an AI consultation