AD-310 Hardcode normalized_os as iOS in contextual_services event_aggregates tables

curtismorales commented 1 month ago

iOS suggest pings are missing normalized_os (comes through as NULL) so we need to hardcode it.

Checklist for reviewer:

[ ] Commits should reference a bug or github issue, if relevant (if a bug is referenced, the pull request should include the bug number in the title).
[ ] If the PR comes from a fork, trigger integration CI tests by running the Push to upstream workflow and provide the <username>:<branch> of the fork as parameter. The parameter will also show up in the logs of the manual-trigger-required-for-fork CI task together with more detailed instructions.
[ ] If adding a new field to a query, ensure that the schema and dependent downstream schemas have been updated.
[ ] When adding a new derived dataset, ensure that data is not available already (fully or partially) and recommend extending an existing dataset in favor of creating new ones. Data can be available in the bigquery-etl repository, looker-hub or in looker-spoke-default.

For modifications to schemas in restricted namespaces (see CODEOWNERS):

[ ] Follow the change control procedure

┆Issue is synchronized with this Jira Task

dataops-ci-bot commented 1 month ago

Integration report for "Hardcode normalized_os as iOS in event_aggregates"

`sql.diff`

Click to expand!

```diff diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/contextual_services_derived/event_aggregates_suggest_v1/query.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/contextual_services_derived/event_aggregates_suggest_v1/query.sql --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/contextual_services_derived/event_aggregates_suggest_v1/query.sql 2024-06-03 23:58:57.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/contextual_services_derived/event_aggregates_suggest_v1/query.sql 2024-06-03 23:58:51.000000000 +0000 @@ -134,7 +134,9 @@ 'phone' AS form_factor, normalized_country_code AS country, metrics.string.fx_suggest_advertiser AS advertiser, - SPLIT(metadata.user_agent.os, ' ')[SAFE_OFFSET(0)] AS normalized_os, + -- This is now hardcoded, we can use the derived `normalized_os` once + -- https://bugzilla.mozilla.org/show_bug.cgi?id=1773722 is fixed + 'iOS' AS normalized_os, client_info.app_channel AS release_channel, metrics.quantity.fx_suggest_position AS position, -- Only remote settings is in use on mobile diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/contextual_services_derived/event_aggregates_v1/query.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/contextual_services_derived/event_aggregates_v1/query.sql --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/contextual_services_derived/event_aggregates_v1/query.sql 2024-06-03 23:58:57.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/contextual_services_derived/event_aggregates_v1/query.sql 2024-06-03 23:58:51.000000000 +0000 @@ -158,7 +158,9 @@ 'remote settings' AS provider, -- Only standard suggestions are in use on mobile 'firefox-suggest' AS match_type, - SPLIT(metadata.user_agent.os, ' ')[SAFE_OFFSET(0)] AS normalized_os, + -- This is now hardcoded, we can use the derived `normalized_os` once + -- https://bugzilla.mozilla.org/show_bug.cgi?id=1773722 is fixed + 'iOS' AS normalized_os, -- This is the opt-in for Merino, not in use on mobile CAST(NULL AS BOOLEAN) AS suggest_data_sharing_enabled, blocks.query_type, ```

Link to full diff

dataops-ci-bot commented 1 month ago

Integration report for "Merge branch 'main' into fix-ios-os-in-event-aggregates"

`sql.diff`

Click to expand!

```diff diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/contextual_services_derived/event_aggregates_suggest_v1/query.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/contextual_services_derived/event_aggregates_suggest_v1/query.sql --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/contextual_services_derived/event_aggregates_suggest_v1/query.sql 2024-06-04 17:38:46.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/contextual_services_derived/event_aggregates_suggest_v1/query.sql 2024-06-04 17:38:36.000000000 +0000 @@ -134,7 +134,9 @@ 'phone' AS form_factor, normalized_country_code AS country, metrics.string.fx_suggest_advertiser AS advertiser, - SPLIT(metadata.user_agent.os, ' ')[SAFE_OFFSET(0)] AS normalized_os, + -- This is now hardcoded, we can use the derived `normalized_os` once + -- https://bugzilla.mozilla.org/show_bug.cgi?id=1773722 is fixed + 'iOS' AS normalized_os, client_info.app_channel AS release_channel, metrics.quantity.fx_suggest_position AS position, -- Only remote settings is in use on mobile diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/contextual_services_derived/event_aggregates_v1/query.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/contextual_services_derived/event_aggregates_v1/query.sql --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/contextual_services_derived/event_aggregates_v1/query.sql 2024-06-04 17:38:46.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/contextual_services_derived/event_aggregates_v1/query.sql 2024-06-04 17:38:36.000000000 +0000 @@ -158,7 +158,9 @@ 'remote settings' AS provider, -- Only standard suggestions are in use on mobile 'firefox-suggest' AS match_type, - SPLIT(metadata.user_agent.os, ' ')[SAFE_OFFSET(0)] AS normalized_os, + -- This is now hardcoded, we can use the derived `normalized_os` once + -- https://bugzilla.mozilla.org/show_bug.cgi?id=1773722 is fixed + 'iOS' AS normalized_os, -- This is the opt-in for Merino, not in use on mobile CAST(NULL AS BOOLEAN) AS suggest_data_sharing_enabled, blocks.query_type, ```

Link to full diff

dataops-ci-bot commented 1 month ago

Integration report for "Merge branch 'main' into fix-ios-os-in-event-aggregates"

`sql.diff`

Click to expand!

```diff diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/contextual_services_derived/event_aggregates_suggest_v1/query.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/contextual_services_derived/event_aggregates_suggest_v1/query.sql --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/contextual_services_derived/event_aggregates_suggest_v1/query.sql 2024-06-05 14:25:43.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/contextual_services_derived/event_aggregates_suggest_v1/query.sql 2024-06-05 14:25:35.000000000 +0000 @@ -134,7 +134,9 @@ 'phone' AS form_factor, normalized_country_code AS country, metrics.string.fx_suggest_advertiser AS advertiser, - SPLIT(metadata.user_agent.os, ' ')[SAFE_OFFSET(0)] AS normalized_os, + -- This is now hardcoded, we can use the derived `normalized_os` once + -- https://bugzilla.mozilla.org/show_bug.cgi?id=1773722 is fixed + 'iOS' AS normalized_os, client_info.app_channel AS release_channel, metrics.quantity.fx_suggest_position AS position, -- Only remote settings is in use on mobile diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/contextual_services_derived/event_aggregates_v1/query.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/contextual_services_derived/event_aggregates_v1/query.sql --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/contextual_services_derived/event_aggregates_v1/query.sql 2024-06-05 14:25:43.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/contextual_services_derived/event_aggregates_v1/query.sql 2024-06-05 14:25:35.000000000 +0000 @@ -158,7 +158,9 @@ 'remote settings' AS provider, -- Only standard suggestions are in use on mobile 'firefox-suggest' AS match_type, - SPLIT(metadata.user_agent.os, ' ')[SAFE_OFFSET(0)] AS normalized_os, + -- This is now hardcoded, we can use the derived `normalized_os` once + -- https://bugzilla.mozilla.org/show_bug.cgi?id=1773722 is fixed + 'iOS' AS normalized_os, -- This is the opt-in for Merino, not in use on mobile CAST(NULL AS BOOLEAN) AS suggest_data_sharing_enabled, blocks.query_type, ```

Link to full diff

mozilla / bigquery-etl