Closed tsaastam closed 7 months ago
Ah, I have actually just discovered how to use the standard flags provided by dbt itself to accomplish what I need. In dbt_project.yml
:
on-run-end: |
{% if flags.WHICH == "test" %}
select 1 from dual;
{% else %}
drop table if exists {{ builtins.ref('my_temp_table_1') }};
drop table if exists {{ builtins.ref('my_temp_table_2') }};
{% endif %}
So this issue can be closed.
Perhaps this example could be added to some documentation somewhere, to make it easier for other people to figure out. :)
I have a SQL pipeline I'm converting to use dbt. The pipeline runs in Snowflake and it uses a lot of temporary tables, which are not directly supported by dbt's Snowflake implementation. For this reason I am instead using transient tables (which are relatively cheap in Snowflake, but permanent) and then dropping them at the end of the run. This is done with an
on-run-end
directive in mydbt_project.yml
:The problem is that when I run my unit tests, dbt also wants to run these drop statements. While I could make this work, it would be much nicer to have an option somewhere to only run the
on-run-end
statements when I'm not running a unit test - there's no reason to (try to) drop actual tables inside a unit test. How can I accomplish this?I've discovered a macro,
dbt_unit_testing.running_unit_test()
, which could in theory be used to only run thedrop table
statements when the current run is not a unit test:However, the macro doesn't seem to exist at the time when the
on-run-end
hook is called:Is it possible to configure the dbt-unit-testing package in such a way that I could use this macro from inside
dbt_project.yml
? Or otherwise, how can I accomplish what I want to do? Is this something that would require additional functionality in dbt itself?Thank you,