fivetran / dbt_mixpanel

Fivetran's Mixpanel dbt package
https://fivetran.github.io/dbt_mixpanel/
Apache License 2.0
6 stars 10 forks source link

[Bug] Breaking change in stg_mixpanel___events.sql event table source #44

Open chiefdatawizard opened 9 months ago

chiefdatawizard commented 9 months ago

Is there an existing issue for this?

Describe the issue

The introduction of the hardcoded source table in commit 1b66f95 [https://github.com/fivetran/dbt_mixpanel/commit/1b66f95022f7e37c78863441b1fa66b48f36f8d8#diff-06f26e4fc8545509412ed16924c7ffacb49b0057ad1f6bb83d75ba1568197428] for events introduced issues for users that used the variable event_table in their dbt_project.yml config.

image

Relevant error log or model output

No response

Expected behavior

The source event table can be determined by the global var event_table in the dbt_project.yml

dbt Project configurations

vars:
  event_table: "{{ ref('stg_mixpanel_union_old_new') }}"
  mixpanel__event_frequency_limit: 1000

Package versions

packages:
  - package: fivetran/mixpanel
    version: [">=0.8.0", "<2.0.0"]

What database are you using dbt with?

snowflake

dbt Version

1.6

Additional Context

No response

Are you willing to open a PR to help address this issue?

fivetran-catfritz commented 9 months ago

Hi @chiefdatawizard thanks for bringing this up! The switch to the hard-coded source was necessary to take advantage of the dbt_utils.star macro, however you can still specify a custom source event table via the identifier variable. For your case. I'd use something like:

vars:
  mixpanel_event_identifier: stg_mixpanel_union_old_new
  mixpanel_schema: <name of the schema for stg_mixpanel_union_old_new>

Please let me know if that works for you. I'll also mention that we do use minor version updates to indicate breaking changes, so we recommend limiting version ranges to patch updates such as version: [">=0.8.0", "<0.9.0"] to avoid breaking updates being automatically loaded.

fivetran-catfritz commented 9 months ago

Hi @chiefdatawizard following up to my last message, I thought about it some more and realize that while the identifier will point to the alternative table, the downside is dbt won't treat it as an upstream model and know to run it first. I'll discuss this with our internal team for ideas on a remedy.

chiefdatawizard commented 9 months ago

Hi @chiefdatawizard thanks for bringing this up! The switch to the hard-coded source was necessary to take advantage of the dbt_utils.star macro, however you can still specify a custom source event table via the identifier variable. For your case. I'd use something like:

vars:
  mixpanel_event_identifier: stg_mixpanel_union_old_new
  mixpanel_schema: <name of the schema for stg_mixpanel_union_old_new>

Please let me know if that works for you. I'll also mention that we do use minor version updates to indicate breaking changes, so we recommend limiting version ranges to patch updates such as version: [">=0.8.0", "<0.9.0"] to avoid breaking updates being automatically loaded.

Hi @fivetran-catfritz, thank you for your reply. We will look into this, to see if this is a viable option for us.

fivetran-catfritz commented 8 months ago

Hi @chiefdatawizard just checking in, did my suggestion work for you?

chiefdatawizard commented 8 months ago

Hi! Unfortunately it is still on our to-do. But when we got an answer I will let you guys know.

fivetran-catfritz commented 7 months ago

Hi @chiefdatawizard let us know if you've been able to try this out. We'll come back to this if so.