search

calogica / dbt-expectations

Port(ish) of Great Expectations to dbt test macros
https://calogica.github.io/dbt-expectations/
Apache License 2.0
1.06k stars 128 forks source link

Fix dangling datediff reference #233

Closed clausherther closed 1 year ago

forgxyz commented 1 year ago

FYI this just nuked some incrementals with the following error. I'm sure it's a quick fix on our end to adjust the test we use for recency checks, but just a heads up @clausherther

(At least I am assuming it was this based on timing, only just investigating now)

Compilation Error in test dbt_expectations_expect_row_values_to_have_recent_data_silver__blocks_BLOCK_TIMESTAMP__day__1 (models/silver/silver__blocks.yml)
  'dict object' has no attribute 'dateadd'

  > in macro test_expect_row_values_to_have_recent_data (macros/schema_tests/table_shape/expect_row_values_to_have_recent_data.sql)
  > called by macro default__test_expect_row_values_to_have_recent_data (macros/schema_tests/table_shape/expect_row_values_to_have_recent_data.sql)
  > called by test dbt_expectations_expect_row_values_to_have_recent_data_silver__blocks_BLOCK_TIMESTAMP__day__1 (models/silver/silver__blocks.yml)
clausherther commented 1 year ago

Hmm this only fixed the ref for datediff to be explicit, so instead of calling dbt-core datediff it calls dbt.datediff. Could you open a new issue for this and provide the necessary info to reproduce? Thanks!

clausherther commented 1 year ago

If anything, I would have been this commit which is in 0.8.1 https://github.com/calogica/dbt-expectations/pull/232/commits/3df4494876cdcc49ae787e527eddad233959f76d

forgxyz commented 1 year ago

Yea, it may have been that one as the hourly incremental at 12 eastern is what started failing. Jumped a bit early to this one.

clausherther commented 1 year ago

If you wouldn't mind, please open a BUG issue. This shouldn't happen if you're on dbt >= 1.2 I think.

forgxyz commented 1 year ago

Yea, once we get the issue fixed I can ticket what happened. Looks like the models are on at least dbt 1.2.3

forgxyz commented 1 year ago

Either hard-coding to 0.8.0 or bumping up to dbt 1.3.0 addresses the error

clausherther commented 1 year ago

Interesting, I was under the impression the date macros were all in 1.2+, but tbh we're not explicitly testing the CI pipeline for 1.2. I'll take a look, but have a feeling we won't revert the release but instead clarify the dbt version dependency.

clausherther commented 1 year ago

I can't reproduce on dbt 1.2, I have a feeling you might have some other cross-package issues with dbt-utils maybe.

forgxyz commented 1 year ago

Likely a cross dependency like that, we'll have to address in greater detail when everyone is back next week. Appreciate your responsiveness!

clausherther commented 1 year ago

We pushed a couple of updates to dbt-date and dbt-expectations that should help with issues related to also including dbt-utils. Let me know if that fixes things for you.