[Bug] Update tests: -> data_tests

fivetran / dbt_google_ads

Fivetran data transformations for Google Ads built using dbt.

https://fivetran.github.io/dbt_google_ads/

Apache License 2.0

14 stars 29 forks source link

[Bug] Update tests: -> data_tests #65

Open kolbynelson opened 4 months ago

kolbynelson commented 4 months ago

Is there an existing issue for this?

[X] I have searched the existing issues

Describe the issue

On dbt version 1.8+ the tests: config results in a warning and needs to be replaced with the data_tests: config.

Relevant error log or model output

[WARNING]: Deprecated functionality
The `tests` config has been renamed to `data_tests`. Please see https://docs.getdbt.com/docs/build/data-tests#new-data_tests-syntax for more information.

Expected behavior

Do not warn.

dbt Project configurations

version: '1.8.1'

Package versions

packages:

package: fivetran/google_ads version: [">=0.10.0", "<0.11.0"]
package: fivetran/sage_intacct version: [">=0.4.0", "<0.5.0"]

What database are you using dbt with?

snowflake

dbt Version

version: '1.8.1'

Additional Context

This is especially an issue when combined with the --warn-error flag. In order for dbt Cloud to send email notifications about test warnings we use the command dbt --warn-error test command. The rest of our tests do not run due to this deprecation warning in the package.

Are you willing to open a PR to help address this issue?

[ ] Yes.
[ ] Yes, but I will need assistance and will schedule time during our office hours for guidance
[x] No.

fivetran-joemarkiewicz commented 4 months ago

Hi @kolbynelson thanks for raising this issue.

Updating the tests to data_tests is something that is on our radar to update across the Fivetran dbt packages in the near future. However, we will want to make this update to all our packages at the same time as this update will make our packages not compatibile with dbt-core<1.8.0 versions 😰.

That being said, the issue you are describing sounds like an unexpected behavior of dbt-core. I would expect while the tests config is still accepted, you should not be seeing this type of compilation failure which causes all your other tests to not run. I just opened an issue in dbt-core to see if this is an expected behavior. I will follow along there and determine if there are any immediate updates we need to make to our data model, or if this should be addressed in dbt-core.

Thanks!

fivetran-joemarkiewicz commented 4 months ago

@kolbynelson it looks like dbt Labs was able to respond to the above dbt-core issue I opened and are suggesting you use the following command as opposed to dbt --warn-error test.

dbt --warn-error-options '{"error": "all", "silence": ["TestsConfigDeprecation"]}' test

This should resolve the compilation you are seeing and allow you to continue leveraging this solution prior to the Fivetran dbt packages being updated to support the new data_tests naming convention. Let me know if this helps!

fivetran-joemarkiewicz commented 3 months ago

@kolbynelson let me know if the above addresses your concerns. As there is not something we will address here for the moment in the dbt package I will mark this as won't do. However, I will keep this open as we will plan to make this update in the future months. Thanks!

kolbynelson commented 3 months ago

Joe,

Thanks for looking into this and getting back to me. I tried the command given but it produces an error. We are actually using dbt Cloud, so this may be a difference in functionality? Does that command also support the --exclude flag?

Thanks, Kolby

From: Joe Markiewicz @.> Sent: Wednesday, June 12, 2024 6:44:25 AM To: fivetran/dbt_google_ads @.> Cc: Nelson, Kolby @.>; Mention @.> Subject: Re: [fivetran/dbt_google_ads] [Bug] Update tests: -> data_tests (Issue #65)

@kolbynelsonhttps://urldefense.com/v3/__https://github.com/kolbynelson__;!!Cki1qOQyyykDL2Q!LahQjfPnzLbUc2rpRabjnKYg9DXdWXNF2a_USTPywQglC6fxY7cnSPdKr3A-Pd8HrDCL7dmE2yQYwl-MDWFPsCZLwafgmEJpzw$ let me know if the above addresses your concerns. As there is not something we will address here for the moment in the dbt package I will mark this as won't do. However, I will keep this open as we will plan to make this update in the future months. Thanks!

— Reply to this email directly, view it on GitHubhttps://urldefense.com/v3/__https://github.com/fivetran/dbt_google_ads/issues/65*issuecomment-2163049397__;Iw!!Cki1qOQyyykDL2Q!LahQjfPnzLbUc2rpRabjnKYg9DXdWXNF2a_USTPywQglC6fxY7cnSPdKr3A-Pd8HrDCL7dmE2yQYwl-MDWFPsCZLwacFODUlEw$, or unsubscribehttps://urldefense.com/v3/__https://github.com/notifications/unsubscribe-auth/BDRS7ROONLT5N4XCK473P43ZHBGDFAVCNFSM6AAAAABI647XEOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNRTGA2DSMZZG4__;!!Cki1qOQyyykDL2Q!LahQjfPnzLbUc2rpRabjnKYg9DXdWXNF2a_USTPywQglC6fxY7cnSPdKr3A-Pd8HrDCL7dmE2yQYwl-MDWFPsCZLwad6JhvakQ$. You are receiving this because you were mentioned.Message ID: @.***>

fivetran-joemarkiewicz commented 3 months ago

Hi @kolbynelson,

I am sorry to hear that didn't work. Unfortunately I am not entirely familiar with the nuances of dbt Cloud in this context. I would recommend reviewing the suggestions from the issue I opened on dbt Core. It looks like there are a few other options you can take here as suggested by dbt Labs. If this doesn't work, I would advise commenting in the issue I created on dbt Labs to see if the folks there are able to provide guidance on possible next steps.

kolbynelson commented 3 months ago

Joe,

I will review the issue and resources provided and follow up there if necessary.

Thanks for your help, Kolby

From: Joe Markiewicz @.> Sent: Wednesday, June 12, 2024 8:54:33 AM To: fivetran/dbt_google_ads @.> Cc: Nelson, Kolby @.>; Mention @.> Subject: Re: [fivetran/dbt_google_ads] [Bug] Update tests: -> data_tests (Issue #65)

Hi @kolbynelsonhttps://urldefense.com/v3/__https://github.com/kolbynelson__;!!Cki1qOQyyykDL2Q!LuBlr-Q-JS5hqmmZTotIEeRR1HWQeJW3zblO4L3nnYMQ02ZCGfrfLWI2vhx0V11Ba_PahnkEB8eJ6j3y4xmK89o0yalEU4J5hQ$,

I am sorry to hear that didn't work. Unfortunately I am not entirely familiar with the nuances of dbt Cloud in this context. I would recommend reviewing the suggestions from the issue I opened on dbt Corehttps://urldefense.com/v3/__https://github.com/dbt-labs/dbt-core/issues/10274__;!!Cki1qOQyyykDL2Q!LuBlr-Q-JS5hqmmZTotIEeRR1HWQeJW3zblO4L3nnYMQ02ZCGfrfLWI2vhx0V11Ba_PahnkEB8eJ6j3y4xmK89o0yanbemY4iw$. It looks like there are a few other optionshttps://urldefense.com/v3/__https://github.com/dbt-labs/dbt-core/issues/10274*issuecomment-2155582205__;Iw!!Cki1qOQyyykDL2Q!LuBlr-Q-JS5hqmmZTotIEeRR1HWQeJW3zblO4L3nnYMQ02ZCGfrfLWI2vhx0V11Ba_PahnkEB8eJ6j3y4xmK89o0yangrc5S3g$ you can take here as suggested by dbt Labs. If this doesn't work, I would advise commenting in the issue I created on dbt Labs to see if the folks there are able to provide guidance on possible next steps.

— Reply to this email directly, view it on GitHubhttps://urldefense.com/v3/__https://github.com/fivetran/dbt_google_ads/issues/65*issuecomment-2163393538__;Iw!!Cki1qOQyyykDL2Q!LuBlr-Q-JS5hqmmZTotIEeRR1HWQeJW3zblO4L3nnYMQ02ZCGfrfLWI2vhx0V11Ba_PahnkEB8eJ6j3y4xmK89o0yallwCHwaA$, or unsubscribehttps://urldefense.com/v3/__https://github.com/notifications/unsubscribe-auth/BDRS7ROWN6HFP5KUQADQL5LZHBVLHAVCNFSM6AAAAABI647XEOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNRTGM4TGNJTHA__;!!Cki1qOQyyykDL2Q!LuBlr-Q-JS5hqmmZTotIEeRR1HWQeJW3zblO4L3nnYMQ02ZCGfrfLWI2vhx0V11Ba_PahnkEB8eJ6j3y4xmK89o0yanBnSHpOQ$. You are receiving this because you were mentioned.Message ID: @.***>