mozilla / bigquery-etl

Bigquery ETL
https://mozilla.github.io/bigquery-etl
Mozilla Public License 2.0
253 stars 100 forks source link

Add 1 more column to downloads_v2 #5861

Closed kwindau closed 3 months ago

kwindau commented 3 months ago

Checklist for reviewer:

For modifications to schemas in restricted namespaces (see CODEOWNERS):

┆Issue is synchronized with this Jira Task

dataops-ci-bot commented 3 months ago

Integration report for "Merge branch 'main' into add-col-to-downloads"

sql.diff

Click to expand! ```diff diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-marketing-prod/ga_derived/www_site_downloads_v2/query.sql /tmp/workspace/generated-sql/sql/moz-fx-data-marketing-prod/ga_derived/www_site_downloads_v2/query.sql --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-marketing-prod/ga_derived/www_site_downloads_v2/query.sql 2024-06-27 20:42:08.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-marketing-prod/ga_derived/www_site_downloads_v2/query.sql 2024-06-27 20:42:08.000000000 +0000 @@ -16,6 +16,7 @@ campaign, ad_content, browser, + campaign_from_event_params, --note: the 2 columns are the same because in GA4, there is no logic saying you can only count 1 download per session, unlike GA3 COUNTIF( --prior to and including 2/16/24 @@ -64,6 +65,16 @@ FROM UNNEST(event_params) WHERE + key = 'campaign' + LIMIT + 1 + ).string_value AS campaign_from_event_params, + ( + SELECT + `value` + FROM + UNNEST(event_params) + WHERE key = 'product' LIMIT 1 @@ -103,7 +114,8 @@ medium, campaign, ad_content, - browser + browser, + campaign_from_event_params ) SELECT `date`, @@ -121,6 +133,7 @@ medium, campaign, ad_content, + campaign_from_event_params, browser, download_events, download_events AS downloads, diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-marketing-prod/ga_derived/www_site_downloads_v2/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-marketing-prod/ga_derived/www_site_downloads_v2/schema.yaml --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-marketing-prod/ga_derived/www_site_downloads_v2/schema.yaml 2024-06-27 20:42:08.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-marketing-prod/ga_derived/www_site_downloads_v2/schema.yaml 2024-06-27 20:42:08.000000000 +0000 @@ -64,6 +64,10 @@ type: STRING description: Browser - mode: NULLABLE + name: campaign_from_event_params + type: STRING + description: Campaign - Parsed from nested event_params key = campaign +- mode: NULLABLE name: download_events type: INT64 description: Download Events - Number of Firefox Desktop Downloads ```

Link to full diff

dataops-ci-bot commented 3 months ago

Integration report for "Merge branch 'main' into add-col-to-downloads"

sql.diff

Click to expand! ```diff diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-marketing-prod/ga_derived/www_site_downloads_v2/query.sql /tmp/workspace/generated-sql/sql/moz-fx-data-marketing-prod/ga_derived/www_site_downloads_v2/query.sql --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-marketing-prod/ga_derived/www_site_downloads_v2/query.sql 2024-06-27 20:54:47.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-marketing-prod/ga_derived/www_site_downloads_v2/query.sql 2024-06-27 20:54:54.000000000 +0000 @@ -16,6 +16,7 @@ campaign, ad_content, browser, + campaign_from_event_params, --note: the 2 columns are the same because in GA4, there is no logic saying you can only count 1 download per session, unlike GA3 COUNTIF( --prior to and including 2/16/24 @@ -64,6 +65,16 @@ FROM UNNEST(event_params) WHERE + key = 'campaign' + LIMIT + 1 + ).string_value AS campaign_from_event_params, + ( + SELECT + `value` + FROM + UNNEST(event_params) + WHERE key = 'product' LIMIT 1 @@ -103,7 +114,8 @@ medium, campaign, ad_content, - browser + browser, + campaign_from_event_params ) SELECT `date`, @@ -121,6 +133,7 @@ medium, campaign, ad_content, + campaign_from_event_params, browser, download_events, download_events AS downloads, diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-marketing-prod/ga_derived/www_site_downloads_v2/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-marketing-prod/ga_derived/www_site_downloads_v2/schema.yaml --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-marketing-prod/ga_derived/www_site_downloads_v2/schema.yaml 2024-06-27 20:54:47.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-marketing-prod/ga_derived/www_site_downloads_v2/schema.yaml 2024-06-27 20:54:54.000000000 +0000 @@ -64,6 +64,10 @@ type: STRING description: Browser - mode: NULLABLE + name: campaign_from_event_params + type: STRING + description: Campaign - Parsed from nested event_params key = campaign +- mode: NULLABLE name: download_events type: INT64 description: Download Events - Number of Firefox Desktop Downloads ```

Link to full diff