Business Intelligence
Querio NL2SQL: How Natural Language Becomes Trusted, Inspectable SQL
How NL2SQL works, where it fails on real warehouses, and how Querio uses governed context and inspectable notebooks to make English-to-SQL trustworthy.
NL2SQL — natural language to SQL — turns a plain-English question like "what was monthly churn by plan last quarter?" into a SQL query that runs against your warehouse. Querio's approach differs from a bare text-to-SQL model in one respect that decides whether the output is trustworthy: the generated SQL is written into a reactive notebook you can open, read and edit, using metric definitions stored as plain files in your own GitHub repository rather than re-derived from scratch on every question. Getting from English to SQL is the easy part; getting from English to the SQL your data team would have written is the hard part, and it is a context problem, not a model problem.
This article explains how NL2SQL actually works, the specific ways it goes wrong in production, what Querio does about each of them, and how the approach compares with building your own, with BI copilots, and with pointing an assistant straight at a warehouse MCP server.
How NL2SQL Works
Every natural-language-to-SQL system does roughly four things:
Schema linking. Map the words in the question to tables and columns. "Revenue" has to resolve to a specific column in a specific table, and "customers" has to resolve to the right one of the four tables with "customer" in the name.
Intent parsing. Work out the aggregation, grouping, filters and time window implied by the question. This is where "last month" becomes either the previous calendar month or the trailing 30 days — two different answers.
Query generation. Emit SQL in the correct dialect. BigQuery, Snowflake, Postgres and SQL Server disagree about date functions, quoting and window syntax.
Execution and repair. Run the query, and when it errors or returns something implausible, fix and retry.
Modern language models are good at step 3 and increasingly decent at step 4. Steps 1 and 2 are where accuracy is won or lost, and both depend almost entirely on how much true context about your business the system has when it starts. For a primer on the underlying mechanics, see how natural language becomes SQL and semantic parsing for text-to-SQL in BI.
Where NL2SQL Goes Wrong in Production
Public benchmarks make text-to-SQL look close to solved. Real warehouses are less forgiving, because they contain history, exceptions and undocumented conventions. These are the failure modes that show up in practice:
Ambiguous time windows. "Last month", "last quarter" and "year to date" all have at least two defensible readings. A system that silently picks one produces a number that is wrong by a knowable amount and looks completely normal.
Incomplete current periods. Week-over-week comparisons that include a partial current week show a fake decline every single time.
Schema linking errors. Picking a customers table rather than the customers table. Legacy tables that were never dropped are the usual culprits.
Join fan-out. Joining orders to line items and then summing an order-level column double counts revenue. The SQL is valid; the number is nonsense.
Missing business filters. Internal test accounts, cancelled orders, refunds, staging rows. Every company has a filter that "everyone knows" to apply, and nobody has written down.
Metric drift. Two people ask the same question in different words, get two different queries, and therefore two different numbers — which destroys trust faster than any single wrong answer.
Confident hallucination. The worst case: the data cannot answer the question, and the system invents something plausible instead of saying so.
None of these are solved by a better model alone. They are solved by giving the system the same context a new analyst gets in their first month — and by making the output checkable.
How Querio's NL2SQL Approach Addresses Each Failure
Context Stored as Files You Own
Joins, metric definitions, business terms and trusted queries live as plain SQL, Markdown and Python files, synced to GitHub in the same repository as your dbt project. The agent proposes what it learns — "save this as a reusable definition?" — and only logged-in humans approve and commit it. That fixes missing business filters and metric drift directly: once the churn definition is committed, every question, notebook, dashboard and MCP call uses it, no matter who asks or how they phrase it.
It also fixes something subtler. Because the context is files rather than a proprietary store, it keeps working with Claude Code or any other agent, even without Querio's MCP. Context lock-in is the real vendor risk in this category, and the answer is that month three should be better than week one and the repository should be yours. More on the pattern in what a governed context layer is.
Real SQL and Python in a Reactive Notebook
Querio does not return a number with the query hidden behind it. Every answer is produced as code in a reactive notebook: cells are connected, so charts update automatically when the SQL changes and nothing has to be rerun from scratch. If a result looks wrong, you open it, read the joins, and fix the query — which is how join fan-out and schema linking errors get caught within seconds instead of appearing in a board meeting. This is the difference we describe in inspectable SQL vs black-box AI.
Refusal Instead of Invention
Querio answers from what is actually in your data. If something isn't there, it says so rather than producing a plausible number. That single behaviour is worth more than several percentage points of benchmark accuracy, because a wrong answer you can see is a bug and a wrong answer you cannot see is a decision.
Live, Read-Only Warehouse Connections
Queries run against live data through encrypted, read-only credentials — Snowflake, BigQuery, Redshift, ClickHouse, MotherDuck, Postgres, MySQL, MariaDB, SQL Server and MongoDB — with no extracts and no data duplication. Dialect differences are handled at generation time. Warehouse-first also means timezone normalisation and lineage are consistent, and when two systems disagree about a number, that gets surfaced rather than smoothed over.
Permissions That Follow the User
Role-based access control applies across surfaces, and the MCP server uses OAuth so an agent's queries inherit the permissions of the person who asked. Asking through Claude does not become a way around your access model.
The Same Answer on Every Surface
The same NL2SQL engine and the same context serve the web app, the Slack and Teams bots, MCP clients such as Claude, the API and embedded analytics. A question asked in Slack spins up a real notebook in the app, so there is a full audit trail rather than an answer that scrolls out of history. See answering data questions in Slack.
Comparison: Four Ways to Get NL2SQL
Approach | Time to first answer | Context handling | Output you can audit | Ongoing ownership cost | Best for |
|---|---|---|---|---|---|
Querio | Days — connect a warehouse and ask | Governed context as files in your GitHub repo, approved by humans | SQL and Python in a reactive notebook | Reviewing proposed context changes | Teams that want governed self-serve and deep analysis in one platform |
Build your own on an LLM API | A weekend to a prototype; months to trustworthy | Whatever you build and maintain | Whatever you build | High and permanent — someone owns the harness forever | Companies where analytics infrastructure is the product |
BI copilot inside an existing tool | Immediate if you already own the tool | Depends on a curated semantic model | Varies; often partial | Semantic model maintenance | Teams heavily invested in one BI vendor |
Assistant plus raw warehouse MCP | Under an hour | Schema rediscovered each session | Yes — the assistant shows its queries | Low to set up, high per query in tokens and time | Exploratory work by technical users |
Build vs Buy: What the DIY Route Actually Costs
A competent engineer can wire an LLM to a warehouse and get correct answers to simple questions in a weekend. That prototype is real, and it is also the cheapest part of the project. The costs that follow are:
The harness. Prompt scaffolding, retry logic, result validation, chart rendering, conversation state. This is the actual product, and it is never finished.
The sync treadmill. dbt models change faster than the YAML, the agent's skills and the dashboards that depend on them. Keeping context aligned with a moving warehouse is the recurring cost nobody budgets for.
Permissions. Making sure the agent cannot answer a question the asker was not entitled to ask, across every surface you expose it on.
Evaluation. Without a regression suite of real questions with verified answers, you will not know when a model or schema change made things worse.
On-call. Once the sales team relies on it, it is production software with an owner.
Build if analytics infrastructure is your product. Otherwise, buy the platform that lets you keep the context in your own repository, so the build-vs-buy decision stays reversible rather than becoming a one-way door. If you have already built one, the maintenance wall is exactly what a shared context repo is designed to remove.
How to Evaluate Any NL2SQL Tool in an Afternoon
Bring twenty real questions. Take them from your actual request queue, not from the vendor's demo script. Include three the data cannot answer.
Verify every answer against hand-written SQL. Score correct, wrong, and refused. Track wrong answers separately from refusals — a refusal is a good outcome, a confident wrong answer is not.
Test ambiguity deliberately. Ask for "last month" and check which interpretation you get. Ask for a week-over-week change mid-week and see whether the partial period is handled.
Force a fan-out. Ask a question that requires joining a one-to-many relationship and summing a parent-level column. This catches more real bugs than any other single test.
Ask the same question two ways. Phrase it differently and check you get the same number. Divergence here means definitions are being re-derived per question.
Check the audit trail. Come back tomorrow and try to find the query behind yesterday's answer.
We publish our own methodology for this in the metrics that actually test text-to-SQL accuracy, and a broader field comparison in the best text-to-SQL query tools.
Getting Started
Querio's MCP and API tier is free to start — 100 questions a month, no payment details, no contract wall — which is enough to run the evaluation above against your own warehouse before talking to anyone. On the SaaS side, Starter is $500 per month for up to 10 users, and Core is $1,999 per month, or $1,699 per month billed annually, with unlimited users, three data connections and guided onboarding. Enterprise is custom for self-hosting, physical data separation or complex deployments. Plans are per workspace, month-to-month, with a free trial, a money-back guarantee, AI usage included with no per-question charges, and an optional hard cap so usage stops at the limit rather than generating a surprise bill. Details are on the pricing page.
FAQs
How accurate is NL2SQL in practice?
Accuracy depends far more on context quality than on the model. The same question against a well-documented warehouse with agreed metric definitions and against an undocumented one produces very different results from identical software. Rather than trusting a headline benchmark, run twenty of your own questions and count three categories: correct, refused, and confidently wrong. The third number is the one that should drive your decision.
Can NL2SQL handle multi-table joins and complex questions?
Yes, when the join paths are defined rather than guessed. The reliable pattern is to store join logic and metric definitions as reviewed context, so the system composes from known-good building blocks instead of inferring relationships from column names. Complex analysis that genuinely needs statistics or modelling is better served by an environment where the agent can also write Python, which is why Querio pairs SQL with a notebook rather than returning SQL alone.
Does NL2SQL replace analysts?
No — it changes what they spend time on. It removes the repetitive request queue that turns a data team into a human API, and moves their effort to defining metrics, reviewing context the agent proposes, and doing the analysis that actually requires judgement. The teams that get the most from it are the ones who treat the analyst as the owner of definitions rather than the writer of every query.
Is it safe to let an AI write queries against production data?
With the right constraints, yes. Use encrypted read-only credentials so nothing can be written or dropped, enforce role-based access at the query layer, use OAuth so agent queries inherit the asker's permissions, and run execution in a sandbox. Querio is SOC 2 Type II, runs annual third-party penetration tests, is HIPAA compliant and signs BAAs, with SSO and optional self-hosted deployment for stricter requirements.
How is this different from connecting Claude to a warehouse MCP server?
Raw warehouse MCP servers work, and they are a good way to explore. The difference is context: a bare server makes the assistant rediscover your schema, joins and filters every session, which costs tokens and reintroduces every ambiguity each time. An analytics platform's MCP server arrives with the governed context already loaded and returns answers as inspectable artefacts rather than transient chat messages. See MCP servers for Snowflake, BigQuery and Postgres compared.
