mozilla / bigquery-etl

Bigquery ETL
https://mozilla.github.io/bigquery-etl
Mozilla Public License 2.0
241 stars 98 forks source link

Bug 1889144 - Add unhashed user id to fxa_delete_events #5864

Closed akkomar closed 3 days ago

akkomar commented 3 days ago

Checklist for reviewer:

For modifications to schemas in restricted namespaces (see CODEOWNERS):

┆Issue is synchronized with this Jira Task

dataops-ci-bot commented 3 days ago

Integration report for "Bug 1889144 - Add unhashed user id to fxa_delete_events"

sql.diff

Click to expand! ```diff diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_accounts/fxa_delete_events/view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_accounts/fxa_delete_events/view.sql --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_accounts/fxa_delete_events/view.sql 2024-06-28 10:50:12.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_accounts/fxa_delete_events/view.sql 2024-06-28 10:49:54.000000000 +0000 @@ -4,13 +4,15 @@ SELECT submission_timestamp, user_id, - hmac_user_id + hmac_user_id, + CAST(NULL AS STRING) AS user_id_unhashed FROM `moz-fx-data-shared-prod.firefox_accounts_derived.fxa_delete_events_v1` UNION ALL SELECT submission_timestamp, user_id, - hmac_user_id + hmac_user_id, + user_id_unhashed FROM `moz-fx-data-shared-prod.firefox_accounts_derived.fxa_delete_events_v2` diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_accounts_derived/fxa_delete_events_v2/query.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_accounts_derived/fxa_delete_events_v2/query.sql --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_accounts_derived/fxa_delete_events_v2/query.sql 2024-06-28 10:50:12.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_accounts_derived/fxa_delete_events_v2/query.sql 2024-06-28 10:49:54.000000000 +0000 @@ -19,6 +19,7 @@ CAST(jsonPayload.fields.uid AS BYTES) ) ) AS hmac_user_id, + jsonPayload.fields.uid AS user_id_unhashed, FROM `moz-fx-fxa-prod.gke_fxa_prod_log.stderr` WHERE diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_accounts_derived/fxa_delete_events_v2/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_accounts_derived/fxa_delete_events_v2/schema.yaml --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_accounts_derived/fxa_delete_events_v2/schema.yaml 2024-06-28 10:50:12.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_accounts_derived/fxa_delete_events_v2/schema.yaml 2024-06-28 10:49:54.000000000 +0000 @@ -5,6 +5,11 @@ - name: user_id type: STRING mode: NULLABLE + description: Hashed user_id - name: hmac_user_id type: STRING mode: NULLABLE +- name: user_id_unhashed + type: STRING + mode: NULLABLE + description: Unhashed user_id ```

Link to full diff