Closed bgeils closed 1 year ago
hey there! so it seems like discount_id
is a deprecated field that we had in our own data, but hasn't been populated in a long time. we should definitely remove this field and use code
instead for any tests in stg_shopify.yml (i think we already use code
for joins downstream)
are you only getting testing related errors, or anything else?
hey @bgeils i think i've addressed the problematic spots here
i have a working branch if you'd like to test it out! 🤞
packages:
- git: https://github.com/fivetran/dbt_shopify_source.git
revision: bugfix/abandoned-checkout-discount-id
another customer with this issue just tested the above branch out and confirmed that the error was resolved on their end at least!
actually it'd be great if you could test this out as well when ya have a chance @bgeils
@fivetran-jamie I'm having this same issue and I'd like to try testing this fix. I'm not clear how to do that. I tried dropping the package info below into my packages.yml file but I still had the error compiling the discount table. I suspect it's because I'm using the fivetran/dbt_shopify package which includes and include for the existing release with the bug. [Sorry if I'm breaking some git protocol etiquette here - I'm new to this]
hey @bgeils i think i've addressed the problematic spots here
i have a working branch if you'd like to test it out! 🤞
packages: - git: https://github.com/fivetran/dbt_shopify_source.git revision: bugfix/abandoned-checkout-discount-id
@brian-equilife hi there! yeah so you should comment out the the fivetran/dbt_shopify
package in your packages.yml, as you're right, there'll be conflicting staging versions otherwise. so, replace the shopify lines with
- git: https://github.com/fivetran/dbt_shopify_source.git
revision: bugfix/abandoned-checkout-discount-id
and run dbt clean
(if you have a clean-paths
config in your dbt_project.yml
, make sure it includes dbt_packages
instead of dbt_modules
) and then dbt run
(note, this will only run the staging models, not the downstream end models).
and you're totally not breaking any sort of protocol, glad you jumped in!
@brian-equilife following up to Jamie's post, I created a branch of our transform package fivetran/dbt_shopify
that points to the above staging branch if that's easier to test/confirm that the changes work downstream. You can use the below in your packages yml instead of the above:
packages:
- git: https://github.com/fivetran/dbt_shopify.git
revision: test/shopify_source_fix
Thank you!
Thank you @fivetran-jamie and @fivetran-catfritz This fix worked like a charm and I no longer have any errors with the shopify__discounts table. I really appreciate your detailed instructions.
this is in the latest release! if you're using the shopify transform package, it'll automatically pick up the changes in the source here
Is there an existing issue for this?
Describe the issue
I am attempting to run the Shopify dbt pipeline and get errors due to a missing field in a source table. The missing field is discount_id from the source table: abandoned_checkout_discount_code
It is an expected input column noted here -- and necessary to successfully run the discount pipeline path. https://github.com/fivetran/dbt_shopify_source/blob/main/macros/get_abandoned_checkout_discount_code_columns.sql
It is also missing from the ERD documentation: https://docs.google.com/presentation/d/1wSWI7SbY4NMtyRLWdg2Z4LW3-SRCF8K7McN0VzLjh3w/edit#slide=id.g1ba00f96b0c_2_0
Screenshot of missing column in FT schema.
Relevant error log or model output
No response
Expected behavior
The field to exist in the schema/database and subsequently the data to be ingested.
dbt Project configurations
NA
Package versions
packages:
What database are you using dbt with?
bigquery
dbt Version
NA (prior to dbt pipeline)
Additional Context
No response
Are you willing to open a PR to help address this issue?