Open graciegoheen opened 11 months ago
Notes from estimation:
dbt_project.yml
, which requires a config object)If you disable unit tests, they won't show up in the manifest.
enabled
can be configured for all the other resource types that I tried out... except for unit tests:
resource type | dbt_project.yml |
properties YAML | config() macro |
---|---|---|---|
models | ✅ | ✅ | ✅ |
snapshots | ✅ | ✅ | ✅ |
seeds | ✅ | ✅ | ➖ |
sources | ✅ | ✅ | ➖ |
exposures | ✅ | ✅ | ➖ |
semantic models | ✅ | ✅ | ➖ |
analyses | ❌ (#8061) | ✅ | ✅ |
singular data tests | ✅ | ❌ (#9005) | ✅ |
generic data tests | ✅ | ✅ | ➖ |
unit tests | ❌ (#9109) | ❌ (#9109) | ➖ |
✅ Supported ❌ Not supported but could be ➖ Not supported and can't be
Currently, there's a handful of ways to prevent a unit testing from running:
#
to the beginning of each linemodel-paths: [directorypath]
configured in dbt_project.yml
(e.g. disabled_unit_tests)tag
and then --exclude
that tag (using the "tag" method)We think option (1) is the easiest:
# you can even leave a comment about why you are commenting it out
# unit_tests:
# - name: test_that_we_want_to_disable
# model: stg_orders
# given: []
# expect:
# rows:
# - {id: 10}
I need to conditionally enable/disable unit tests in the audit helper repo, because (for example) some use SQL fixtures which have different syntax on different platforms.
I've gone with this, but I do not like it:
unit_tests:
[...]
config:
tags: "{{ 'works_on_my_machine' if (target.type in ['snowflake', 'bigquery']) else 'skip' }}"
And the CircleCI job will have this defined: dbt build --exclude tag:skip
@joellabes sounds like what you're actual after is something like conditional fixtures?
(just an idea)
unit_tests:
- name: test_my_model
model: my_model
given:
- input: ref('my_model_a')
format: csv
fixture: "{{ 'my_model_a_fixture_snow_bq' if (target.type in ['snowflake', 'bigquery']) else 'my_model_a_fixture_other' }}"
...
Ooooh that's a fun idea! In practice I wouldn't want to have to make a dozen or more fixture files though, the fact that I can put it inline in the yaml is pretty compelling to me.
Secondarily, I also worry that the conditional Jinja logic would get horrendous if it wasn't just a binary option. Although I guess I could do {{ target.type }}_fixture_1.sql
instead of an if block (were I willing to make files for each of them)
Just adding a +1 for this issue. We use enabled:
to dynamically turn models on and off depending on the deployment region (not all models are applicable in all geographies).
Applying unit tests to models which are disabled in a particular region causes the DBT compile to fail. This means we're unable to adopt the new unit testing functionality :'(.
@benw-at-birdie it makes sense to me to add enabled
for unit tests because every other resource type has it.
In the meantime, did you try out the workaround using tags, by any chance?
We would want to move the disabled unit tests to the disabled dictionary.
In the meantime, did you try out the workaround using tags, by any chance?
Hey @dbeatty10 apologies for the slow reply. Unfortunately the tags approach doesn't work for our current setup...
production-frankfurt:data ❯ dbt test -s alerts --exclude tag:skip
15:31:20 Running with dbt=1.8.1
15:31:21 Registered adapter: snowflake=1.8.2
15:31:21 Unable to do partial parsing because profile has changed
15:31:29 Encountered an error:
Parsing Error
Unable to find model 'snowflake_db_transformations.alerts' for unit test 'unresolved_qualified_and_resolved_alerts' in models/inbox/alerts/alerts.yml
In the example above the alerts
model has a unit test which has been tagged with skip
but because this model is disabled in the selected environment (production-frankfurt
) the compilation step fails before the test command can be run. Does that make sense?
I think the only way to make the tags approach work is to enable all models in all environments (and use tags to control the execution of models in different regions). We tried this approach but ran into complications with slim-ci because the manifest.json doesn't accurately reflect the models deployed in a given environment (it includes all of them regardless of whether or not the model is deployed in the given environment).
Let's do #10540 together with this issue.
I can confirm that the tags approach doesn't work
Hey, any idea when this would be implemented? Our team would like to disable unit tests from running using this flag.
@adionisio8 we're hoping to get this merged soon! PR has been opened
Housekeeping
Short description
Folks may want to turn unit tests on/off.
In order to do configs we'd have to have model config objects for unit test definitions, create a project object unit_tests dictionary, and merge configs from dbt_project and each individual unit test.
Acceptance criteria
https://docs.getdbt.com/reference/resource-configs/enabled
Unit tests should support the
enabled
,meta
, andtags
config:By resource path in
dbt_project.yml
:Or in unit test definition yml:
We already support
meta
andtags
, but not yetenabled
.Impact to Other Teams
none
Will backports be required?
no
Context
No response