Skip to content
bricksentry

Docs · 07

Data quality monitoring on Databricks

Monitoring tells you a table went wrong after it landed. bricksentry checks the change before it lands.

Two different questions

Data quality monitoring watches tables over time: freshness, volume, distributions. That answers "is this table healthy now?". bricksentry answers a narrower question at the moment of change: "what will this pull request, job run or agent write do to production data, and who depends on it?". Use both; bricksentry does not replace Databricks' own monitoring features.

What bricksentry checks

  • Data diff. Production vs. candidate on your key columns: rows added, removed and changed, duplicate and NULL keys, and how totals moved. All diffs run as SQL on your own warehouse.
  • Blast radius. Downstream tables, jobs and dashboards from Unity Catalog lineage, with owners and tags, down to columns.
  • Policy verdict. Rules in policy.yml return pass, warn, needs_human or block. The worst matching rule wins; no LLM decides.

Where it sits in the workflow

  1. Pull request: assess the branch's candidate tables and post a review comment. See CI / pull requests.
  2. Job run: a task after your write task fails the job on a blocking verdict. See Databricks jobs.
  3. AI agent: preview_write runs the statement on a shallow clone first. See Agents & MCP.
  4. After landing: your monitoring keeps watching the table over time.

Try it on one table

shell
uvx --from "bricksentry[duckdb]" bricksentry demo

# or with pip
pip install "bricksentry[duckdb]"
bricksentry demo

Then follow the Quickstart to point it at a Databricks table, and set rules in Policy.