The data-agent stack went free. I'm building the missing piece.

Two companion posts set this one up. What a data agent is, and what makes one good laid out the concept and the bar it has to clear; the readiness series, why the data is the ceiling and what ready data actually looks like, laid out the foundation it stands on. This post is the build: one opinionated way to assemble, deploy, and maintain a data agent that clears that bar, with an unusual property attached. I am going to build it, in public, and measure it against a live leaderboard.

Two things changed since I first mapped this space, and both changed in the same direction. First, the economics: the entire dbt leg of the stack is now permissively licensed end to end. Fivetran and dbt Labs closed their merger on June 1, the Fusion engine open-sourced as dbt Core v2.0 under Apache 2.0, and MetricFlow, the metrics engine, quietly relicensed from a restrictive license to Apache 2.0 in October 2025, one day after the merger was announced. Second, the harness matured: the Claude Agent SDK treats the filesystem as context, which means the thing the readiness work produces, versioned files full of business meaning, is not an add-on to the agent. It is the native input format.

Opinionated architecture and concepts

Start with the shape, because the tool choices only make sense once the architecture is fixed. It is one agent, not many. Anthropic's own guidance sets the bar: "finding the simplest solution possible, and only increasing complexity when needed", with a warning that multi-agent systems burn about fifteen times the tokens of a chat session. One agent, one loop: gather context, take action, verify the work, repeat.

The properties that make that agent good are not mine and not up for debate; every serious vendor converged on the same skeleton. Snowflake's Cortex Analyst constrains generation with a semantic model, retrieves human-verified queries as trust anchors, and runs an error-correction pass against the SQL compiler before anything executes. Wren's model definition language makes the constraint blunt: "The agent can only name modeled objects, so undeclared columns and legacy tables are invisible." Cube states the governance version: access rules are enforced before any SQL runs. Databricks productized the eval side, up to 500 benchmark questions per agent, and its newest move, Genie Ontology, is a context graph that ranks definitions by authority, PageRank for business meaning. None of them agrees on a product shape; all of them agree the semantic layer is an enforced contract, SQL is validated before execution, and a benchmark suite is a first-class object. The architecture argument is over. What is left to decide is how you assemble it, and how the context gets better over time.

That is why I am not building a framework. The lane is closed: ktx, the Apache-licensed context layer, tops the Spider 2.0-Lite leaderboard ahead of entries from Snowflake's research group and Oracle, and when the category owner tops the public benchmark, building a competing engine is a donation, not a gap. Cube keeps its open-source semantic engine but paywalls its MCP server; Vanna, the most popular open-source text-to-SQL project, was archived in March. The defensible move for one person is assembly and evidence, not another engine.

The evidence points at one gap. Everyone ships the loop where an agent proposes context changes and a human reviews the pull request. Wren ships an auto-pilot mode where the agent "proposes context changes with evidence, and waits for review... Nothing is silently absorbed into a black box." ktx built its whole product on context as reviewable Git files: "Your team reviews. Agents query." Atlan describes the same shape, with rejection reasons feeding back into the proposer. The proposal loop is published, three times over. What nobody publishes is that loop with a gate in it. No public system wires an agent-drafted pull request to the semantic layer through a regression eval suite in CI, with before-and-after accuracy deltas on a reproducible open warehouse, before a human merges. Context changes that make the agent worse should fail a build, the same way code changes that break tests do. That is the missing piece, and it is small enough for one person to close.

Put simply: One agent, one loop, on the skeleton every vendor already agrees on: semantic contract, validate before execute, golden questions. Do not build another framework; the category owner already tops the benchmark. Build the one thing nobody ships, a gate that makes agent-proposed context changes pass an exam before they merge.

Opinionated tooling and stack

The warehouse: DuckDB, reached through the MotherDuck MCP server. MIT-licensed, actively maintained, and, the detail that matters most for an agent: read-only by default, with write access requiring an explicit flag. The old official Postgres and SQLite MCP reference servers are archived, a detail most stack posts have not caught up with. DuckDB also happens to be the engine under the benchmark this project will be scored on, which is not a coincidence I intend to waste.

The semantic engine: dbt plus MetricFlow, run locally. The dbt MCP server exposes governed metrics to agents (list_metrics, query_metrics, compiled SQL on request), and the dbt-metricflow package installs with a DuckDB adapter, which makes a zero-cost, fully local, Apache-licensed metrics engine a real thing you can pin in a requirements file. One honest caveat: the hosted semantic-layer APIs and the remote MCP tools ride the paid dbt platform; locally you get the CLI tools and MetricFlow directly, which is exactly enough for a proof of concept.

Context as code: the layer you write. Markdown for what the business means, YAML for what the metrics are, all of it in Git. This is the gold layer on the diagram because it is the only one with no vendor: ktx and Wren AI both formalize the same shape, and the pattern works precisely because it is boring. Files get reviewed, diffed, and merged like everything else you trust.

The harness: the Claude Agent SDK. The SDK reads CLAUDE.md, rules, and skills off the filesystem by default, re-injects the durable context on every request so it survives compaction, and supports skills as progressively disclosed playbooks: metadata always loaded, full instructions only when relevant. Structured outputs turn an agent run into typed, scoreable JSON, which the eval layer needs.

Governance: three layers, because the docs say so. The SDK's own documentation is refreshingly blunt that a tool allowlist is an approval list, not a boundary. Enforcement gets three layers: deny rules and a locked-down permission mode as the wall, a PreToolUse hook that inspects every SQL call before it runs (hook denies apply even in bypass mode, per the docs), and a thin in-process MCP proxy so the agent's only path to the warehouse is a read-only, governed one. The credential itself is SELECT-only with row and timeout caps, which is not paranoia, it is OWASP's own example of what excessive agency looks like when you skip it. And because a warehouse agent holds private data by definition, Simon Willison's lethal trifecta rule applies: no external-communication tool anywhere in the query path.

Evals: the layer that gates everything else. Golden questions with expected results, scored by result-match, with LLM-as-judge reserved for free-form answers only: the two-tier methodology Databricks ships in Genie's benchmark product. Braintrust's MIT-licensed autoevals library plus plain pytest covers a one-person harness. The wry footnote of the year: promptfoo, the other obvious choice, is MIT-licensed and excellent, and was acquired by OpenAI in March. On a Claude-stack build, that gets a raised eyebrow and a dependency pin.

Put simply: Every layer is now available under a permissive license, and the only layer without a vendor is the one that determines quality. Buy nothing yet. Pin six dependencies, write your context as files, and spend your money on the layer nobody sells.

Build and usage preview, with performance

Here is the build, stated before it starts so the result can be checked against the promise. One repository: a DuckDB demo warehouse, a dbt project with MetricFlow metrics pinned to the stable release line, a context directory of markdown and YAML, a Claude Agent SDK harness with the three governance layers above, and a golden-question eval suite that runs headless in CI on every change. The agent answers questions through governed metrics; when it hits a gap, it drafts a context pull request with its evidence and trace attached; the eval suite decides whether the change survives.

The operating half is what separates a working agent from a demo, and it is short enough to state completely.

Run it headless, on an API key. The SDK runs as a supervised process with the CLI underneath; in CI, bare mode skips the discovery and makes eval runs reproducible. Anthropic's docs require API-key authentication for products built on the SDK, not consumer-subscription login, and its credential guidance is the right paranoia: tool credentials stay out of the agent's environment entirely, injected by a proxy after the request leaves the container, with deployable containers published in Anthropic's cookbook.

Watch it with standards, not screenshots. The SDK exports OpenTelemetry metrics and traces to any OTLP backend; the GenAI conventions are still marked in-development, so treat span names as changeable, and land everything in a self-hosted, MIT-licensed Langfuse so the audit trail is yours. Every answer carries its trace, which context files loaded, which tools ran, what the SQL was, and that trace is what the context pull requests attach as evidence.

Model the cost like an operator. Prompt caching does the heavy lifting for an always-on agent: cache reads are a tenth of base input price, and the stable prefix, the semantic model and rules, is exactly what caches. Two honesty notes from the docs themselves: the SDK's cost fields are client-side estimates, not billing, and Sonnet's price is introductory until September 1. Budget on the standard price, reconcile against the real usage API, and alert on the OWASP failure mode with the best name in the field, denial of wallet.

Maintenance is the loop. The agent's quality decays as the business drifts, definitions change, tables appear, metrics get renamed. Drift shows up as failed golden questions, fixes arrive as agent-drafted pull requests with evidence, and the suite decides what merges. Maintaining a data agent stops being re-prompt-until-it-behaves and becomes the same discipline as maintaining code.

The measurement is the part I like most. Spider 2.0 retired its original setting and now runs three tracks; the Snow track is nearly saturated at 96.7%, but the DBT track, 68 repository-level tasks on exactly this stack, dbt on DuckDB, free to run, tops out at 65.6%. A near-stock agent already posts 61.2% on the sister Lite track, so the floor is known and the ceiling is close. A one-person kit with governed context has a legitimate shot at a citable public number, and the numbers get published either way: if governed context does not beat the raw baseline on the golden set and the public track, that result ships too, because a null result about the industry's favorite thesis would be worth more than a demo.

Put simply: A proof of concept with no benchmark is a demo. This one ships with its exam attached: a golden set gating every merge, a public leaderboard for the headline claim, traces and cost modeled like an operator would, and a commitment to publish the deltas whichever way they point.

Adaptability across warehouse platforms

The obvious objection is that this is a DuckDB kit, and most companies do not run on DuckDB. They do not, and that is deliberate: DuckDB is the demo substrate because it is free, local, and the exact engine the public benchmark runs on, which makes the whole build reproducible by anyone with a laptop. What matters is that only the bottom layer is DuckDB-specific. Everything above it is warehouse-agnostic by construction.

The semantic engine already proves the point. dbt and MetricFlow run on Snowflake, BigQuery, Databricks, Redshift, and Postgres through the same adapter interface, so moving the demo to a real warehouse is a profile change, not a rewrite; the metrics, the context files, and the eval suite come along untouched. The governance pattern is identical everywhere: a read-only credential, a pre-execution hook, and a proxy do not care what dialect they are guarding. And the destination vendors are already living proof that the architecture is universal, not a DuckDB quirk. Snowflake ships this shape as Cortex Analyst, Databricks as Genie on top of Unity Catalog; both enforce a semantic contract and validate before execution, which is the same skeleton this kit assembles from open parts. The swap that is genuinely warehouse-specific, the MCP server for the warehouse itself, is the one component every serious platform now provides.

So the honest scope: the picks are a reproducible reference, not a mandate. Keep the architecture, the context-as-code layer, the governance, and the eval gate; swap the warehouse and its adapter for whatever your data already lives in. The part that ports is the part that matters, because the value was never in the engine.

Put simply: DuckDB is the reproducible demo, not a lock-in. The semantic layer, context, governance, and eval gate move to Snowflake, BigQuery, Databricks, or Postgres as a config change; only the bottom warehouse layer swaps. The vendors that own those warehouses ship the same architecture, which is the point.

What compounds, and where I can help

The pattern underneath this whole build is the one my profit-pools work keeps finding in other people's industries, now applied at home. The models are rented, and they get cheaper and better on someone else's capex. The frameworks are venture-funded gifts. What compounds is the layer this stack puts in gold: the context, the evals that protect it, and the record of every change that made the meaning better, which is the rare kind of data asset that actually flywheels rather than just piling up. That is the asset the proof of concept exists to demonstrate.

So two invitations, depending on where you are. If you want to watch it get built, the build log is this series, and the leaderboard number, positive or null, gets published here when it lands; follow along. And if you are standing up a data agent of your own and any of this maps onto a decision you are making right now, the architecture, the governance, the eval gate, or the readiness work underneath it, I would like to hear about it. Find me on LinkedIn and let us compare notes, or bring me in to help.

Put simply: Anything you rent gets cheaper without you; anything you own has to be built and kept. Put your one scarce resource into the layer nobody sells, the context and the tests that guard it. I am building that in public, so follow the build for the result, or get in touch if you want help pointing it at your own warehouse.