[Feature] Disable unit tests whose model is disabled

SimonCarryer commented 1 month ago

Is this a new bug in dbt-core?

[X] I believe this is a new bug in dbt-core
[X] I have searched the existing issues, and I could not find an existing issue for this bug

Current Behavior

If a model is set to disabled by a macro (for example the standard config macro), then the enabled status is not known at compile time, and unit tests for the model are still compiled. This causes a compilation error:

Compilation Error
  'unit_test.data_staging.my_cool_report.my_cool_unit_test' depends on 'model.data_staging.my_cool_report' which is not in the graph!

Expected Behavior

Usually, if a model is disabled (i.e. enabled: false), then unit tests which test that model are not compiled.

Steps To Reproduce

Create a model which is disabled by a macro (for example {{config(enabled=false)}})
Create a unit test for that model
Run the project

Relevant log output

No response

Environment

- OS: Windows
- Python: 3.12.4
- dbt: 1.8.2

Which database adapter are you using with dbt?

postgres

Additional Context

No response

dbeatty10 commented 1 month ago

I see what you are saying @SimonCarryer 👍

Specifically, both generic and singular data tests are disabled when they depend on a model that is disabled, and you'd like unit tests to behave the same way. This looks like the expected behavior to me though, so I'm going to update this to a feature request for further consideration.

We should also consider unit tests through the lens of indirect selection for data tests.

This is similar to https://github.com/dbt-labs/dbt-core/issues/9109, but distinct.

### Reprex

`models/my_cool_report.sql` ```sql select 1 as id ``` `models/_models.yml` ```yaml models: - name: my_cool_report config: enabled: false columns: - name: id data_tests: - not_null ``` `tests/assert_coolness.sql` ```sql select * from {{ ref('my_cool_report')}} where id is null ``` `models/_unit_tests.yml` ```yaml unit_tests: - name: my_cool_unit_test model: my_cool_report given: [] expect: rows: - {id: 1} ``` Commands: ```shell dbt build ``` ``` Compilation Error 'test.my_project.assert_coolness' depends on 'model.my_project.my_cool_report' which is not in the graph! ``` Output: ```shell dbt build --no-partial-parse ``` Output: ``` 15:25:52 Encountered an error: 'model.my_project.my_cool_report' 15:25:52 Traceback (most recent call last): File "/Users/dbeatty/projects/environments/dbt_1.8/lib/python3.10/site-packages/dbt/cli/requires.py", line 138, in wrapper result, success = func(*args, **kwargs) File "/Users/dbeatty/projects/environments/dbt_1.8/lib/python3.10/site-packages/dbt/cli/requires.py", line 101, in wrapper return func(*args, **kwargs) File "/Users/dbeatty/projects/environments/dbt_1.8/lib/python3.10/site-packages/dbt/cli/requires.py", line 218, in wrapper return func(*args, **kwargs) File "/Users/dbeatty/projects/environments/dbt_1.8/lib/python3.10/site-packages/dbt/cli/requires.py", line 247, in wrapper return func(*args, **kwargs) File "/Users/dbeatty/projects/environments/dbt_1.8/lib/python3.10/site-packages/dbt/cli/requires.py", line 294, in wrapper return func(*args, **kwargs) File "/Users/dbeatty/projects/environments/dbt_1.8/lib/python3.10/site-packages/dbt/cli/requires.py", line 320, in wrapper ctx.obj["manifest"] = parse_manifest( File "/Users/dbeatty/projects/environments/dbt_1.8/lib/python3.10/site-packages/dbt/parser/manifest.py", line 1898, in parse_manifest manifest = ManifestLoader.get_full_manifest( File "/Users/dbeatty/projects/environments/dbt_1.8/lib/python3.10/site-packages/dbt/parser/manifest.py", line 330, in get_full_manifest manifest = loader.load() File "/Users/dbeatty/projects/environments/dbt_1.8/lib/python3.10/site-packages/dbt/parser/manifest.py", line 487, in load self.process_unit_tests(self.root_project.project_name) File "/Users/dbeatty/projects/environments/dbt_1.8/lib/python3.10/site-packages/dbt/parser/manifest.py", line 1294, in process_unit_tests process_models_for_unit_test( File "/Users/dbeatty/projects/environments/dbt_1.8/lib/python3.10/site-packages/dbt/parser/unit_tests.py", line 468, in process_models_for_unit_test target_model = manifest.nodes[target_model_id] KeyError: 'model.my_project.my_cool_report' ```

SimonCarryer commented 1 month ago

Oh no! As best I can tell, unit tests on disabled models are already disabled. The issue is that if the model is disabled by a macro, rather than in plain text, then the unit test is still compiled (and causes a compilation error).

dbeatty10 commented 1 month ago

The example in https://github.com/dbt-labs/dbt-core/issues/10540#issuecomment-2278217616 just disables the model in YAML like this (which uses plain text rather than a macro):

models:
  - name: my_cool_report
    config:
      enabled: false

And it leaves all the unit tests related to that model as enabled and gives the compilation error you mentioned. Are you seeing something different? i.e., do you have an example where the unit test is disabled?

As an aside, when I disable a model using {{ config(enabled=false) }} within the model definition rather than YAML, I get a Parsing Error rather than a Compilation Error:

Parsing Error
  Unable to find model 'my_project.my_cool_report' for unit test 'my_cool_unit_test' in models/_unit_tests.yml

dbt-labs / dbt-core