search

dbt-labs / dbt-core

dbt enables data analysts and engineers to transform their data using the same practices that software engineers use to build applications.
https://getdbt.com
Apache License 2.0
9.96k stars 1.63k forks source link

[Bug] Changes under `data_tests:` top-level key within `properties.yml` files not seen by partial parsing #10801

Open dbeatty10 opened 1 month ago

dbeatty10 commented 1 month ago

Is this a new bug in dbt-core?

Current Behavior

With the change in https://github.com/dbt-labs/dbt-core/pull/10792, any changes to the data_tests: within _properties.yml / schema.yml (patch) files are not picked up by partial parsing, and the --no-partial-parse flag is necessary to pick them up.

Expected Behavior

I expected that I could change the content within data_tests:, and dbt would see those changes, even when using partial parsing.

Steps To Reproduce

See https://github.com/dbt-labs/docs.getdbt.com/issues/6132#issue-2545975324 for an example of how to use the data_tests: top-level key.

Specifically:

data_tests:
  - name: my_singular_test
    description: Some docs

versus:

data_tests:
  - name: my_singular_test
    description: Some docs
    config:
      error_if: ">10"
    meta:
      some_key: some_val

Relevant log output

Note that this is a FAIL because changes within the data_tests: top-level key were not picked up:

partial success

$ dbt test -s my_project.my_singular_test

20:08:55  Running with dbt=1.9.0-a1
20:08:55  Registered adapter: postgres=1.9.0-a1
20:08:56  Found 1 test, 423 macros
20:08:56  
20:08:56  Concurrency: 5 threads (target='postgres')
20:08:56  
20:08:57  1 of 1 START test my_singular_test ............................................. [RUN]
20:08:57  1 of 1 FAIL 2 my_singular_test ................................................. [FAIL 2 in 0.06s]
20:08:57  
20:08:57  Finished running 1 test in 0 hours 0 minutes and 1.08 seconds (1.08s).
20:08:57  
20:08:57  Completed with 1 error, 0 partial successs, and 0 warnings:
20:08:57  
20:08:57  Failure in test my_singular_test (tests/my_singular_test.sql)
20:08:57    Got 2 results, configured to fail if != 0
20:08:57  
20:08:57    compiled code at target/compiled/my_project/tests/my_singular_test.sql
20:08:57  
20:08:57  Done. PASS=0 WARN=0 ERROR=1 SKIP=0 TOTAL=1

But this is a WARN because changes within the data_tests: top-level key were picked up by disabling partial parsing:

$ dbt test -s my_project.my_singular_test --no-partial-parse

20:09:05  Running with dbt=1.9.0-a1
20:09:05  Registered adapter: postgres=1.9.0-a1
20:09:07  Found 1 test, 423 macros
20:09:07  
20:09:07  Concurrency: 5 threads (target='postgres')
20:09:07  
20:09:07  1 of 1 START test my_singular_test ............................................. [RUN]
20:09:07  1 of 1 WARN 2 my_singular_test ................................................. [WARN 2 in 0.08s]
20:09:07  
20:09:07  Finished running 1 test in 0 hours 0 minutes and 0.31 seconds (0.31s).
20:09:07  
20:09:07  Completed with 1 warning:
20:09:07  
20:09:07  Warning in test my_singular_test (tests/my_singular_test.sql)
20:09:07  Got 2 results, configured to warn if != 0
20:09:07  
20:09:07    compiled code at target/compiled/my_project/tests/my_singular_test.sql
20:09:07  
20:09:07  Done. PASS=0 WARN=1 ERROR=0 SKIP=0 TOTAL=1

Environment

- OS:
- Python:
- dbt:

Which database adapter are you using with dbt?

No response

Additional Context

No response