dbt-labs / dbt-core

dbt enables data analysts and engineers to transform their data using the same practices that software engineers use to build applications.
https://getdbt.com
Apache License 2.0
9.51k stars 1.58k forks source link

[Feature] When a test fails, `dbt retry` should also rebuild the model(s) against which the test failed instead of re-running the same doomed test against the original model #10164

Open joellabes opened 3 months ago

joellabes commented 3 months ago

Is this your first time submitting a feature request?

Describe the feature

Given:

--models/model_a.sql
select 1 as id
union all 
select null as id
# models/model_a.yml
models: 
  - name: model_a
    columns: 
      - name: id
        tests: 
          - not_null
  1. Run dbt build - fails because id has a null record
  2. Delete the lines adding null to model_a
  3. Run dbt retry --state yada/yada
  4. Note that the build still fails because id has a null record, because only the test is re-run.

Instead, dbt retry should re-run the model that is attached to the test.

Today, retry only works for transient errors (e.g. a relationship test failing because fivetran hasn't finished on both sides of the PK/FK divide) or errors that cause a SQL failure in the model. If tests pick up a logic mistake, then dbt retry doesn't work for mopping up afterwards.

Describe alternatives you've considered

rolling my own with something like dbt build -s 1+status:error+ (freehanded, but you get my point even if it's not strictly correct syntax)

Who will this benefit?

No response

Are you interested in contributing this feature?

Nah

Anything else?

Instigating feedback from the Cloud feedback form: https://dbt-labs.productboard.com/insights/all-notes/notes/39794606

I have seen multiple other threads about this too though so not an isolated incident.

dave-connors-3 commented 3 months ago

This was requested in a customer conversation yesterday, and is an all around no brainer of an enhancement IMHO!