Typing a question in plain English and getting a chart back is the easy part of a natural language to SQL demo. The hard part comes after: can you act on that number, or do you need to check it first?
That's not a hypothetical. A wrong number is worse than no number, because it gets reported before anyone notices. It moves into a slide deck, then a decision, then a quarter. By the time someone checks the source, the number has already shaped a plan around it.
Why "it sounds right" isn't good enough
A language model is fluent, not necessarily correct. A wrong join, a stale table reference, or a metric re-derived slightly differently than usual can still read confidently. It can look exactly like a correct answer. There's no visual difference between a right one and a wrong one — not unless you can see the work behind it.
A stale table reference is a good example. The table was renamed after a migration. The query silently keeps working — against a copy nobody updates anymore — and the answer it returns looks exactly as fresh as a correct one.
Where this goes wrong in practice
Consider two questions asked five minutes apart: "How many users were active last week?" and "How many users engaged last week?" A natural language to SQL tool with no governing model can resolve each one independently. One query might run against login events. The other might run against a broader activity table. Both return a number. Neither is flagged as unusual, and both charts render cleanly.
The mismatch only surfaces if someone happens to compare the two answers side by side. In a governed setup, that comparison shouldn't be necessary at all. A tool resolving through one semantic model would have answered both questions the same way, because "active" would already have one definition, not two.
How natural language to SQL builds trust
- Visible SQL. The query should be shown next to the answer, not hidden behind the chart. If you can't see what was actually run against the warehouse, you're trusting the tool's word for it.
- A governed model to resolve against. An agent that re-derives "revenue" from raw tables on every question can get it wrong differently each time. One that resolves through a semantic layer uses the same metric definitions your dashboards already use.
- Permissions applied before generation, not after. The agent should never be able to construct a query that reaches data the person asking isn't cleared for. Access is checked before the query runs, not filtered from the answer afterward.
None of these three depends on the other two working by luck. Visible SQL doesn't help if the model behind it drifts between questions. A governed model doesn't help if the agent could have queried past what the person asking was allowed to see. All three have to hold at once, or the answer is only provisionally trustworthy.
What to ask before you trust a tool's answer
- Can I see the exact query that produced this number?
- Does it use the same metric definitions as my existing dashboards, or does it derive its own?
- Does it respect my existing row and column permissions, or does it query with elevated access I don't control?
- Does the answer change if I ask the same question in different words?
- When it's wrong, how would I actually find out?
Where Veritly fits
Veritly's agent doesn't answer from memory. Every response comes from a SQL query run against your warehouse — this is natural language to SQL with the model resolved, not guessed. That query resolves through the same governed semantic model every other surface reads from, with permissions applied before it runs, not after. The query itself is shown next to the answer. You can verify it before acting on it, rather than deciding whether to trust the tool in the abstract. That's the difference between a natural language to SQL tool you have to take on faith, and one you can check.
Join the private beta waitlist — early access includes a setup session with the team that built it.
Related reading: why a metric defined once should never drift, and retrieval augmented generation for analytics.
See also: ChatGPT for Excel.
