brooklyn-data / dbt_artifacts

A dbt package for modelling dbt metadata. https://brooklyn-data.github.io/dbt_artifacts
Apache License 2.0
309 stars 113 forks source link

[Bug]: test "failures" has a misleading definition #400

Open Stephen986 opened 8 months ago

Stephen986 commented 8 months ago

Overview

The definition of "failures" is misleading: https://github.com/brooklyn-data/dbt_artifacts/blob/main/models/docs.md?plain=1#L184. It makes it sound like a field that is either 1 or 0, but failures can be much higher than 0 if multiple records fail the test.

How to reproduce

  1. Read the docs on "failures"
  2. Query your dbt artifacts data:
  3. SELECT * FROM dbt-artifacts.martdbt_artifacts.fct_dbttest_executions where failures > 1 LIMIT 10;

Expected behaviour

The docs should indicate that failures can be greater than 0. I'm not sure the best possible description, but maybe something like: "A single test execution can have multiple failures if multiple rows had incorrect results."

Environment

Results of running dbt --version:

1.6
jared-rimmer commented 5 months ago

@Stephen986 happy to accept a PR with a definition you propose