Ask ChatGPT about your company's Q3 churn rate and it will invent a number. It has never seen your data. RAG for analytics (retrieval augmented generation) fixes that first problem. It does not fix the second one.
RAG connects an AI model to your real data before it answers. That stops the invented figures. It does not settle which definition of "revenue" the answer uses. This post covers how RAG works, where it stops, and what closes the gap.
The problem with traditional AI in analytics
Standard LLMs are trained on public data. Training stops at a fixed date. They know nothing about what happened in your business yesterday. They have never seen your internal reports.
An analyst needs answers based on records, not probability. If an AI guesses your sales figures, it is useless. Worse, someone may act on the guess.
What is RAG for analytics?
RAG for analytics connects an AI model to your business data before it generates an answer. The model retrieves relevant documents, reports or database records first. Then it writes a response grounded in what it found.
The knowledge base is often held in a vector database, which allows fast semantic search. The system finds the relevant material. Then it uses that material to write the answer.
How the process works in practice
The workflow has three steps. First, the system retrieves. It searches your PDFs, spreadsheets and notes for the most relevant pieces.
Second, it augments the prompt. It adds the retrieved material to your original question. This gives the model the context it needs.
Finally, it generates the answer. The model writes a response from the material it was given.
Three ways analysts use retrieval today
1. Natural language database queries
Describe what you want in plain English. The system reads your schema, including table names and column definitions. It then writes the SQL for you.
2. Conversational data exploration
Ask for the Q4 demand forecast for the Northeast. Then follow up with "break that down by state". Each question builds on the last.
3. Answers from past reports
Ask what your team concluded about a competitor's pricing. The system searches past reports and memos and summarises what it finds. You skip the folder dig.
RAG versus fine-tuning
Some teams consider fine-tuning a model on their data instead. RAG has two clear advantages for analytics.
First, your data changes every day. With retrieval, you update the source and the next answer reflects it. Fine-tuning has to be repeated, and each round takes time and money.
Second, retrieval can point to the source it used. You can check the answer against that source. A fine-tuned model gives you no such trail.
Why retrieval alone does not fix inconsistent metrics
Retrieval finds the right rows. It does not choose the right definition. That is a separate problem.
Take "revenue". Finance counts it net of refunds. The sales dashboard counts it at booking. A spreadsheet from last quarter counts it in another way again. A RAG system can retrieve any of these. It may pick one, or blend two, and write a fluent answer. That answer can disagree with the BI tool, and nothing in the text will say why.
More retrieval makes this worse, not better. The model has more material and more conflicting definitions to choose from.
Two things close the gap. The first is a governed definition layer: each metric defined once, and every answer resolved through it. The second is a visible query. If you can read the SQL behind an answer, you can see which definition it used and check it against the BI tool. Semantic layers are the usual way to hold the first part. The semantic layer explainer covers how they work.
Where Veritly fits
Veritly is an integrated analysis environment. It puts the BI tool, the warehouse and the spreadsheets in one governed place. One semantic model resolves every dashboard, document and job. A metric is defined once and read everywhere.
You ask in plain English. Veritly writes the query and shows you the SQL it ran. The answer traces back to the definition it used. You can check it, and so can your data lead.
RAG is closely related to the Model Context Protocol, which provides the live data connection that many RAG setups build on. Connection gets an AI to your data. A governed definition and a visible query make the answer trustworthy.
Veritly is in private beta. Join the waitlist — access opens as capacity does.
See also: natural language to SQL.
