Velir / dbt-ga4

dbt Package for modeling raw data exported by Google Analytics 4. BigQuery support, only.
MIT License
324 stars 135 forks source link

feature: add property_id #321

Closed FloDevelops closed 5 months ago

FloDevelops commented 6 months ago

Description & motivation

This PR adds a new property_id column to the base_ga4__events model which is derived from either the name of the imported dataset or the property_ids dbt variable (depending on whether this is a multi-property or single-property project).

Note that the default behavior of dbt is to ignore new columns during incremental runs. This behavior can be overridden with the on_schema_change configuration https://docs.getdbt.com/docs/build/incremental-models#what-if-the-columns-of-my-incremental-model-change

Checklist

adamribaudo-velir commented 6 months ago

The stream ID is a more robust method of determining where the data came from given that 1 property can have multiple streams. Can you describe a use case where the property ID provides value above and beyond the stream ID?

FloDevelops commented 6 months ago

The property ID simplifies analyzes by property ID and name (with a mapping seed) regardless of the number of streams for each property.

It also enables to build URLs to the GA4 UI.

adamribaudo-velir commented 5 months ago

@FloDevelops have you tested this on a property with intraday tables? I get the following error:

13:23:54  Completed with 1 error and 0 warnings:
13:23:54
13:23:54    Database Error in model base_ga4__events (models\staging\base\base_ga4__events.sql)
  Bad int64 value: intraday_20240604237690264
  compiled Code at target\run\ga4\models\staging\base\base_ga4__events.sql
FloDevelops commented 5 months ago

@adamribaudo-velir Indeed I did not test on intraday tables. I just fixed that now.

adamribaudo-velir commented 5 months ago

This looks good. Just be aware that the default behavior when adding a new column to an incremental model is to ignore the change until a full refresh. Users can update their config to append the new column

https://docs.getdbt.com/docs/build/incremental-models