search

fivetran / dbt_shopify_source

Fivetran's Shopify source dbt package
https://fivetran.github.io/dbt_shopify_source/
Apache License 2.0
29 stars 23 forks source link

[Feature] Shopify test table warning #83

Closed TomaszE closed 2 months ago

TomaszE commented 3 months ago

Is there an existing feature request for this?

Describe the Feature

Hi Guys, we are running on most recent shopify package version , and we have run in to test warining on 1 table : accepted_values_stg_shopifyfulfillment_event_statusattempted_deliverydeliveredfailurein_transitout_for_deliveryready_for_pickup__picked_uplabel_printed__label_purchased__confirmed

the outcome of the test is just one additional fullfilment status that is missing on your list which is delayed would it be possible to add this status to your accepted values list ? 

log-level-accepted_values_stg_shopify__fulfillment_event_status__attempted_delivery__delivered__failure__in_transit__out_for_delivery__ready_for_pickup__picked_up__label_printed__label_purchased__confirmed
Summary
Details

Download Logs
11:17:39 320 of 661 START test accepted_values_stg_shopify__fulfillment_event_status__attempted_delivery__delivered__failure__in_transit__out_for_delivery__ready_for_pickup__picked_up__label_printed__label_purchased__confirmed  [RUN]
11:17:42 320 of 661 WARN 1 accepted_values_stg_shopify__fulfillment_event_status__attempted_delivery__delivered__failure__in_transit__out_for_delivery__ready_for_pickup__picked_up__label_printed__label_purchased__confirmed  [WARN 1 in 2.91s]
11:30:21 Warning in test accepted_values_stg_shopify__fulfillment_event_status__attempted_delivery__delivered__failure__in_transit__out_for_delivery__ready_for_pickup__picked_up__label_printed__label_purchased__confirmed (models/stg_shopify.yml)
11:30:21 Got 1 result, configured to warn if != 0
11:30:21   compiled Code at target/compiled/shopify_source/models/stg_shopify.yml/accepted_values_stg_shopify__f_dedeafd665959576232d68017a86f88f.sql

Describe alternatives you've considered

for first one, as I said, just add status delayed on list,

Are you interested in contributing this feature?

Anything else?

No response

fivetran-jamie commented 3 months ago

Hey there @TomaszE -- yes we can certainly add delayed to the accepted_values

We have this warning in place because we pivot out these values downstream in the shopify__daily_shop transform model in a way that is slightly hard-coded

        {% for status in ['attempted_delivery', 'delivered', 'failure', 'in_transit', 'out_for_delivery', 'ready_for_pickup', 'picked_up', 'label_printed', 'label_purchased', 'confirmed']%}
        , count(distinct case when lower(status) = '{{ status }}' then fulfillment_id end) as count_fulfillment_{{ status }}
        {% endfor %}

So we'll also want to add it to this line in dbt_shopify as well

This is very straightforward update and we can fold it into an upcoming sprint!

shreveasaurus commented 3 months ago

@fivetran-jamie I hope you don't mind but I went ahead and opened a couple PRs to accommodate this new value as I was seeing the same error in one of the environments I'm working in. Thanks for calling out the dbt_shopify package needed to be updated as well!

https://github.com/fivetran/dbt_shopify/pull/81 https://github.com/fivetran/dbt_shopify_source/pull/84

fivetran-jamie commented 3 months ago

i certainly don't mind, this is awesome šŸ™Œ going to review your PRs so we can get them out next week!

fivetran-jamie commented 2 months ago

this is live in v0.12.1! thanks @TomaszE for opening and @shreveasaurus for the PR šŸ˜¤