Closed jtcohen6 closed 2 years ago
@jtcohen6 -- thanks for the heads up! you're right that the end-user experience will change, but this will affect our integration tests.
I think making these changes will allow things to work. my questions are:
@swanderz Really good questions.
- we need to first wait for upstream repos to change, right?
I thought about this, and realized - you could include both syntaxes if you wanted! As soon as you're running dbt v0.19.2 or higher, this should work just fine:
dispatch:
- macro_namespace: dbt_utils
search_order: ['tsql_utils', 'tsql_utils_dbt_utils_integration_tests', 'dbt_utils']
vars:
dbt_utils_dispatch_list: ['tsql_utils', 'tsql_utils_dbt_utils_integration_tests']
If you've installed the latest version of dbt-utils
(0.7+), you've got the dispatch syntax in place. If you've installed an older version of dbt-utils
(<0.7.) that uses the old dispatch syntax, great, you have the var there to make it work.
But I don't think this will even be an issue, because each of the four packages for which tsql-utils provides compatibility—dbt_utils
, audit_helper
, dbt_date
, dbt_expectations
—have already or will soon be releasing a new minor version that:
So as soon as you're running your integration tests on dbt v0.20.0, and against new compliant package versions, you could make a hard cutover to the new syntax.
- is there a way to associate versions of tsql-utils with specific versions of the parent pacakges without explicitly making the parent packages dependencies? I forsee a potentially tedious table in the readme the lists the version correspondance.
The code that's actually in tsql-utils
will be compatible with all versions of these other packages, no matter what, regardless of whether they use the old or new dispatch syntax. The thing that will differ is what we called out above: the var/config in dbt_project.yml
for your integration tests, and for folks installing the package and setting it up to shim.
Again, because there's going to be a clear cutover from pre-v0.20 support to post-v0.20 support in these other packages, I don't think any user should find themselves in the awkward place of running dbt v0.19.1 with a brand-new version of these packages, or running v0.20.0 with an old version of these packages.
So I don't think there's any compatibility matrix for you to document, really. It's more like: If you're using dbt v0.18 or v0.19, use this old syntax (vars
). If you're using v0.20 or newer, use this new syntax (dispatch
config); and you could even use both, if you really want.
dispach: - macro_namespace: dbt_utils search_order: ['tsql_utils', 'dbt_utils'] - macro_namespace: dbt_date search_order: ['tsql_utils', 'dbt_date'] - macro_namespace: dbt_expectations search_order: ['tsql_utils', 'dbt_expectations']
Carefull with the typo there! dispach
-> dispatch
Ah! Thanks for the catch, just edited above
Hey T-SQL team!
Given that this package exists alongside dispatch, but doesn't dispatch macros itself, I think this will only require a change to the README. Once users and packages have upgraded to the new dispatch syntax in dbt v0.20, they'll want to specify a first-order project config in
dbt_project.yml
:Instead of:
Two crucial differences in the new syntax:
dbt_utils
, they should only specify that one.In the meantime, there is backwards compatibility for the old syntax, so users on old versions of dbt, or using old versions of packages, can still specify those vars.