Closed LewisDavies closed 7 months ago
Hey there @LewisDavies , thanks for taking the time to dig into this!
your solution definitely makes sense, but i just have one question first to see if we can simplify the test further - does the following test also return a failure?
# stg_facebook_ads.yml
- name: stg_facebook_ads__creative_history
description: Each record in this table reflects a Facebook creative.
tests:
- dbt_utils.unique_combination_of_columns:
combination_of_columns:
- source_relation
- _fivetran_id
No worries! Just tried your config and the tests are passing for me:
dbt test -s stg_facebook_ads__creative_history
13:36:37 Running with dbt=1.6.9
13:36:37 Registered adapter: bigquery=1.6.9
13:36:38 Found 107 models, 3 seeds, 181 tests, 35 sources, 0 exposures, 9 metrics, 907 macros, 0 groups, 1 semantic model
13:36:38
13:36:39 Concurrency: 4 threads (target='dev')
13:36:39
13:36:39 1 of 3 START test dbt_utils_unique_combination_of_columns_stg_facebook_ads__creative_history_source_relation___fivetran_id [RUN]
13:36:39 2 of 3 START test not_null_stg_facebook_ads__creative_history__fivetran_synced . [RUN]
13:36:39 3 of 3 START test not_null_stg_facebook_ads__creative_history_creative_id ...... [RUN]
13:36:41 2 of 3 PASS not_null_stg_facebook_ads__creative_history__fivetran_synced ....... [PASS in 1.66s]
13:36:41 3 of 3 PASS not_null_stg_facebook_ads__creative_history_creative_id ............ [PASS in 1.86s]
13:36:43 1 of 3 PASS dbt_utils_unique_combination_of_columns_stg_facebook_ads__creative_history_source_relation___fivetran_id [PASS in 3.22s]
13:36:43
13:36:43 Finished running 3 tests in 0 hours 0 minutes and 5.01 seconds (5.01s).
13:36:43
13:36:43 Completed successfully
13:36:43
13:36:43 Done. PASS=3 WARN=0 ERROR=0 SKIP=0 TOTAL=3
amazing, let's go that route to keep the test nice and simple. if you're still open to contributing a PR, i'd be happy to prioritze reviewing it! if not, the team can pick this up next sprint 🤠
Is there an existing issue for this?
Describe the issue
I have added the
ad_reporting
package to my project anddbt run
was successful for all models. However, I'm getting an error when testing the results because of apparent duplicates instg_facebook_ads__creative_history
. This is the only test that is failing.The output of this model is filtered by the two directly downstream (
facebook_ads__url_tags
andint_facebook_ads__creative_history
), which use the is_most_recent_record column to remove older rows.Temporary workaround
Using the package name and full identifier in my project file let me disable the test and pass CI:
Fix
Adding a
where
clause in the test config prevents the error:Let me know if this an acceptable solution and I will open a PR to make the change 🙂
Relevant error log or model output
Expected behavior
The tests pass or accurately reflect an error that needs to be fixed.
dbt Project configurations
Package versions
What database are you using dbt with?
bigquery
dbt Version
1.6.9
Additional Context
No response
Are you willing to open a PR to help address this issue?