fivetran / dbt_shopify_holistic_reporting

Fivetran's Shopify Holistic Reporting dbt package
https://fivetran.github.io/dbt_shopify_holistic_reporting
Apache License 2.0
6 stars 8 forks source link

[Bug] Orders model incremental logic does not capture updated orders #8

Closed fivetran-jamie closed 1 year ago

fivetran-jamie commented 1 year ago

Is there an existing issue for this?

Describe the issue

in https://github.com/fivetran/dbt_shopify_holistic_reporting/blob/main/models/shopify_holistic_reporting__orders_attribution.sql, we use the created_timestamp fields of orders to determine if the order is new enough to be included in an incremental batch.

with orders as (

    select *
    from {{ ref('shopify__orders') }}

    -- just grab new orders
    {% if is_incremental() %}
        where created_timestamp >= (select max(created_timestamp) from {{ this }})
    {% endif %}

should be


    select *
    from {{ ref('shopify__orders') }}

    -- just grab new orders
    {% if is_incremental() %}
        where updated_timestamp >= (select max(updated_timestamp) from {{ this }})
    {% endif %}

i think

Relevant error log or model output

No response

Expected behavior

updated orders should be re-transformed in this model so that they are up-to-date and not being lost along the way from shopify__orders

dbt Project configurations

-

Package versions

-

What database are you using dbt with?

bigquery

dbt Version

-

Additional Context

No response

Are you willing to open a PR to help address this issue?

fivetran-joemarkiewicz commented 1 year ago

Closing this issue as PR #9 addressed this bug.