mozilla / bigquery-etl

Bigquery ETL
https://mozilla.github.io/bigquery-etl
Mozilla Public License 2.0
243 stars 98 forks source link

feat: make changes to generate retention and engagement for focus and klar products also #5710

Closed kik-kik closed 1 month ago

kik-kik commented 1 month ago

feat: make changes to generate retention and engagement for focus and klar products also

The current version only generates retention and engagement artifacts for firefox_ios and fenix. This change aims to introduce these artifacts to the following mobile products:


Checklist for reviewer:

For modifications to schemas in restricted namespaces (see CODEOWNERS):

┆Issue is synchronized with this Jira Task

dataops-ci-bot commented 1 month ago

Integration report for "feat: make changes to generate retention and engagement for focus and klar products also"

sql.diff

Click to expand! ```diff Only in /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop/use_counters: schema.yaml Only in /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop/user_characteristics: schema.yaml Only in /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_background_update/use_counters: schema.yaml Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android: engagement Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android: engagement_clients Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android: retention Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android: retention_clients Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android_derived: engagement_v1 Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android_derived: retention_v1 Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_ios: engagement Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_ios: engagement_clients Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_ios: retention Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_ios: retention_clients Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_ios_derived: engagement_v1 Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_ios_derived: retention_v1 Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/klar_android: engagement Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/klar_android: engagement_clients Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/klar_android: retention Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/klar_android: retention_clients Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/klar_android_derived: engagement_v1 Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/klar_android_derived: retention_v1 Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/klar_ios: engagement Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/klar_ios: engagement_clients Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/klar_ios: retention Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/klar_ios: retention_clients Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/klar_ios_derived: engagement_v1 Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/klar_ios_derived: retention_v1 Only in /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/org_mozilla_fenix/use_counters: schema.yaml Only in /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/org_mozilla_fenix_nightly/use_counters: schema.yaml Only in /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/org_mozilla_fennec_aurora/use_counters: schema.yaml Only in /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/org_mozilla_firefox/use_counters: schema.yaml Only in /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/org_mozilla_firefox_beta/use_counters: schema.yaml Only in /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/org_mozilla_focus/use_counters: schema.yaml Only in /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/org_mozilla_focus_beta/use_counters: schema.yaml Only in /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/org_mozilla_focus_nightly/use_counters: schema.yaml Only in /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/org_mozilla_klar/use_counters: schema.yaml Only in /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/pine/use_counters: schema.yaml diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_backend_derived/event_monitoring_live_v1/materialized_view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_backend_derived/event_monitoring_live_v1/materialized_view.sql --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_backend_derived/event_monitoring_live_v1/materialized_view.sql 2024-06-03 10:44:10.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_backend_derived/event_monitoring_live_v1/materialized_view.sql 2024-06-03 10:46:15.000000000 +0000 @@ -60,7 +60,7 @@ LEFT JOIN UNNEST(event.extra) AS event_extra WHERE - DATE(submission_timestamp) >= "2024-05-31" + DATE(submission_timestamp) >= "2024-06-03" GROUP BY submission_date, window_start, diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_frontend_derived/event_monitoring_live_v1/materialized_view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_frontend_derived/event_monitoring_live_v1/materialized_view.sql --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_frontend_derived/event_monitoring_live_v1/materialized_view.sql 2024-06-03 10:44:11.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_frontend_derived/event_monitoring_live_v1/materialized_view.sql 2024-06-03 10:46:15.000000000 +0000 @@ -60,7 +60,7 @@ LEFT JOIN UNNEST(event.extra) AS event_extra WHERE - DATE(submission_timestamp) >= "2024-05-31" + DATE(submission_timestamp) >= "2024-06-03" GROUP BY submission_date, window_start, diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/bedrock_derived/event_monitoring_live_v1/materialized_view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/bedrock_derived/event_monitoring_live_v1/materialized_view.sql --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/bedrock_derived/event_monitoring_live_v1/materialized_view.sql 2024-06-03 10:44:10.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/bedrock_derived/event_monitoring_live_v1/materialized_view.sql 2024-06-03 10:46:16.000000000 +0000 @@ -50,7 +50,7 @@ client_info.app_display_version AS version, ping_info FROM - `moz-fx-data-shared-prod.bedrock_live.events_v1` + `moz-fx-data-shared-prod.bedrock_live.interaction_v1` UNION ALL SELECT submission_timestamp, @@ -60,7 +60,7 @@ client_info.app_display_version AS version, ping_info FROM - `moz-fx-data-shared-prod.bedrock_live.interaction_v1` + `moz-fx-data-shared-prod.bedrock_live.non_interaction_v1` UNION ALL SELECT submission_timestamp, @@ -70,7 +70,7 @@ client_info.app_display_version AS version, ping_info FROM - `moz-fx-data-shared-prod.bedrock_live.non_interaction_v1` + `moz-fx-data-shared-prod.bedrock_live.events_v1` ) CROSS JOIN UNNEST(events) AS event, @@ -80,7 +80,7 @@ LEFT JOIN UNNEST(event.extra) AS event_extra WHERE - DATE(submission_timestamp) >= "2024-05-31" + DATE(submission_timestamp) >= "2024-06-03" GROUP BY submission_date, window_start, diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates/schema.yaml --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates/schema.yaml 2024-06-03 10:43:28.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates/schema.yaml 2024-06-03 10:53:03.000000000 +0000 @@ -1,49 +1,49 @@ fields: -- mode: NULLABLE - name: submission_date +- name: submission_date type: DATE -- mode: NULLABLE - name: source + mode: NULLABLE +- name: source type: STRING -- mode: NULLABLE - name: event_type + mode: NULLABLE +- name: event_type type: STRING -- mode: NULLABLE - name: form_factor + mode: NULLABLE +- name: form_factor type: STRING -- mode: NULLABLE - name: country + mode: NULLABLE +- name: country type: STRING -- mode: NULLABLE - name: subdivision1 + mode: NULLABLE +- name: subdivision1 type: STRING -- mode: NULLABLE - name: advertiser + mode: NULLABLE +- name: advertiser type: STRING -- mode: NULLABLE - name: release_channel + mode: NULLABLE +- name: release_channel type: STRING -- mode: NULLABLE - name: position + mode: NULLABLE +- name: position type: INTEGER -- mode: NULLABLE - name: provider + mode: NULLABLE +- name: provider type: STRING -- mode: NULLABLE - name: match_type + mode: NULLABLE +- name: match_type type: STRING -- mode: NULLABLE - name: normalized_os + mode: NULLABLE +- name: normalized_os type: STRING -- mode: NULLABLE - name: suggest_data_sharing_enabled + mode: NULLABLE +- name: suggest_data_sharing_enabled type: BOOLEAN -- mode: NULLABLE - name: event_count + mode: NULLABLE +- name: event_count type: INTEGER -- mode: NULLABLE - name: user_count + mode: NULLABLE +- name: user_count type: INTEGER -- mode: NULLABLE - name: query_type + mode: NULLABLE +- name: query_type type: STRING + mode: NULLABLE diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates_suggest/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates_suggest/schema.yaml --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates_suggest/schema.yaml 2024-06-03 10:43:28.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates_suggest/schema.yaml 2024-06-03 10:53:03.000000000 +0000 @@ -1,40 +1,40 @@ fields: -- mode: NULLABLE - name: submission_date +- name: submission_date type: DATE -- mode: NULLABLE - name: form_factor + mode: NULLABLE +- name: form_factor type: STRING -- mode: NULLABLE - name: country + mode: NULLABLE +- name: country type: STRING -- mode: NULLABLE - name: advertiser + mode: NULLABLE +- name: advertiser type: STRING -- mode: NULLABLE - name: normalized_os + mode: NULLABLE +- name: normalized_os type: STRING -- mode: NULLABLE - name: release_channel + mode: NULLABLE +- name: release_channel type: STRING -- mode: NULLABLE - name: position + mode: NULLABLE +- name: position type: INTEGER -- mode: NULLABLE - name: provider + mode: NULLABLE +- name: provider type: STRING -- mode: NULLABLE - name: match_type + mode: NULLABLE +- name: match_type type: STRING -- mode: NULLABLE - name: suggest_data_sharing_enabled + mode: NULLABLE +- name: suggest_data_sharing_enabled type: BOOLEAN -- mode: NULLABLE - name: impression_count + mode: NULLABLE +- name: impression_count type: INTEGER -- mode: NULLABLE - name: click_count + mode: NULLABLE +- name: click_count type: INTEGER -- mode: NULLABLE - name: query_type + mode: NULLABLE +- name: query_type type: STRING + mode: NULLABLE diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/debug_ping_view_derived/event_monitoring_live_v1/materialized_view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/debug_ping_view_derived/event_monitoring_live_v1/materialized_view.sql --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/debug_ping_view_derived/event_monitoring_live_v1/materialized_view.sql 2024-06-03 10:44:10.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/debug_ping_view_derived/event_monitoring_live_v1/materialized_view.sql 2024-06-03 10:46:16.000000000 +0000 @@ -60,7 +60,7 @@ LEFT JOIN UNNEST(event.extra) AS event_extra WHERE - DATE(submission_timestamp) >= "2024-05-31" + DATE(submission_timestamp) >= "2024-06-03" GROUP BY submission_date, window_start, diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/engagement_clients/view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/engagement_clients/view.sql --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/engagement_clients/view.sql 2024-06-03 10:44:09.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/engagement_clients/view.sql 2024-06-03 10:46:14.000000000 +0000 @@ -26,19 +26,19 @@ client_id, sample_id, channel AS normalized_channel, + NULLIF(play_store_attribution_campaign, "") AS play_store_attribution_campaign, + NULLIF(play_store_attribution_medium, "") AS play_store_attribution_medium, + NULLIF(play_store_attribution_source, "") AS play_store_attribution_source, + NULLIF(meta_attribution_app, "") AS meta_attribution_app, + NULLIF(install_source, "") AS install_source, NULLIF(adjust_ad_group, "") AS adjust_ad_group, - NULLIF(adjust_creative, "") AS adjust_creative, - NULLIF(adjust_network, "") AS adjust_network, CASE WHEN adjust_network IN ('Google Organic Search', 'Organic') THEN 'Organic' ELSE NULLIF(adjust_campaign, "") END AS adjust_campaign, - NULLIF(play_store_attribution_campaign, "") AS play_store_attribution_campaign, - NULLIF(play_store_attribution_medium, "") AS play_store_attribution_medium, - NULLIF(play_store_attribution_source, "") AS play_store_attribution_source, - NULLIF(meta_attribution_app, "") AS meta_attribution_app, - NULLIF(install_source, "") AS install_source, + NULLIF(adjust_creative, "") AS adjust_creative, + NULLIF(adjust_network, "") AS adjust_network, FROM `moz-fx-data-shared-prod.fenix_derived.firefox_android_clients_v1` ) @@ -57,15 +57,15 @@ is_wau, is_mau, is_mobile, - attribution.adjust_ad_group, - attribution.adjust_campaign, - attribution.adjust_creative, - attribution.adjust_network, attribution.play_store_attribution_campaign, attribution.play_store_attribution_medium, attribution.play_store_attribution_source, attribution.meta_attribution_app, attribution.install_source, + attribution.adjust_ad_group, + attribution.adjust_campaign, + attribution.adjust_creative, + attribution.adjust_network, CASE WHEN active_users.submission_date = first_seen_date THEN 'new_profile' diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_clients/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_clients/schema.yaml --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_clients/schema.yaml 2024-06-03 10:43:28.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_clients/schema.yaml 2024-06-03 10:52:28.000000000 +0000 @@ -26,6 +26,9 @@ - name: adjust_network type: STRING mode: NULLABLE +- name: install_source + type: STRING + mode: NULLABLE - name: retained_week_2 type: BOOLEAN mode: NULLABLE diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_week_4/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_week_4/schema.yaml --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_week_4/schema.yaml 2024-06-03 10:43:28.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_week_4/schema.yaml 2024-06-03 10:52:28.000000000 +0000 @@ -48,6 +48,10 @@ description: 'The type of source of a client installation. ' +- name: install_source + type: STRING + mode: NULLABLE + description: null - name: new_profiles type: INTEGER mode: NULLABLE diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/retention_clients/view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/retention_clients/view.sql --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/retention_clients/view.sql 2024-06-03 10:44:09.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/retention_clients/view.sql 2024-06-03 10:46:14.000000000 +0000 @@ -22,19 +22,19 @@ client_id, sample_id, channel AS normalized_channel, + NULLIF(play_store_attribution_campaign, "") AS play_store_attribution_campaign, + NULLIF(play_store_attribution_medium, "") AS play_store_attribution_medium, + NULLIF(play_store_attribution_source, "") AS play_store_attribution_source, + NULLIF(meta_attribution_app, "") AS meta_attribution_app, + NULLIF(install_source, "") AS install_source, NULLIF(adjust_ad_group, "") AS adjust_ad_group, - NULLIF(adjust_creative, "") AS adjust_creative, - NULLIF(adjust_network, "") AS adjust_network, CASE WHEN adjust_network IN ('Google Organic Search', 'Organic') THEN 'Organic' ELSE NULLIF(adjust_campaign, "") END AS adjust_campaign, - NULLIF(play_store_attribution_campaign, "") AS play_store_attribution_campaign, - NULLIF(play_store_attribution_medium, "") AS play_store_attribution_medium, - NULLIF(play_store_attribution_source, "") AS play_store_attribution_source, - NULLIF(meta_attribution_app, "") AS meta_attribution_app, - NULLIF(install_source, "") AS install_source, + NULLIF(adjust_creative, "") AS adjust_creative, + NULLIF(adjust_network, "") AS adjust_network, FROM `moz-fx-data-shared-prod.fenix_derived.firefox_android_clients_v1` ) @@ -56,6 +56,10 @@ attribution.play_store_attribution_source, attribution.meta_attribution_app, attribution.install_source, + attribution.adjust_ad_group, + attribution.adjust_campaign, + attribution.adjust_creative, + attribution.adjust_network, -- ping sent retention active_users.retention_seen.day_27.active_on_metric_date AS ping_sent_metric_date, ( @@ -79,10 +83,6 @@ -- Looking back at 27 days to support the official definition of repeat_profile (someone active between days 2 and 28): AND BIT_COUNT(mozfun.bits28.range(active_users.days_active_bits, -26, 27)) > 0 ) AS repeat_profile, - attribution.adjust_ad_group, - attribution.adjust_campaign, - attribution.adjust_creative, - attribution.adjust_network, active_users.days_seen_bits, active_users.days_active_bits, CASE diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/engagement_v1/query.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/engagement_v1/query.sql --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/engagement_v1/query.sql 2024-06-03 10:44:11.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/engagement_v1/query.sql 2024-06-03 10:46:14.000000000 +0000 @@ -8,15 +8,15 @@ country, locale, is_mobile, - adjust_ad_group, - adjust_campaign, - adjust_creative, - adjust_network, play_store_attribution_campaign, play_store_attribution_medium, play_store_attribution_source, meta_attribution_app, install_source, + adjust_ad_group, + adjust_campaign, + adjust_creative, + adjust_network, COUNTIF(is_dau) AS dau, COUNTIF(is_wau) AS wau, COUNTIF(is_mau) AS mau @@ -33,12 +33,12 @@ country, locale, is_mobile, - adjust_ad_group, - adjust_campaign, - adjust_creative, - adjust_network, play_store_attribution_campaign, play_store_attribution_medium, play_store_attribution_source, meta_attribution_app, - install_source + install_source, + adjust_ad_group, + adjust_campaign, + adjust_creative, + adjust_network diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/engagement_v1/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/engagement_v1/schema.yaml --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/engagement_v1/schema.yaml 2024-06-03 10:44:11.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/engagement_v1/schema.yaml 2024-06-03 10:46:14.000000000 +0000 @@ -39,50 +39,50 @@ mode: NULLABLE description: Indicates if this specific entry is used towards calculating mobile DAU. -- name: adjust_ad_group +- name: play_store_attribution_campaign type: STRING mode: NULLABLE - description: Adjust Ad Group the profile is attributed to. + description: Play store campaign the profile is attributed to. -- name: adjust_campaign +- name: play_store_attribution_medium type: STRING mode: NULLABLE - description: Adjust Campaign the profile is attributed to. + description: Play store Medium the profile is attributed to. -- name: adjust_creative +- name: play_store_attribution_source type: STRING mode: NULLABLE - description: Adjust Creative the profile is attributed to. + description: Play store source the profile is attributed to. -- name: adjust_network +- name: meta_attribution_app type: STRING mode: NULLABLE - description: Adjust Network the profile is attributed to. + description: Facebook app linked to paid marketing. -- name: play_store_attribution_campaign +- name: install_source type: STRING mode: NULLABLE - description: Play store campaign the profile is attributed to. + description: The source of a profile installation. -- name: play_store_attribution_medium +- name: adjust_ad_group type: STRING mode: NULLABLE - description: Play store Medium the profile is attributed to. + description: Adjust Ad Group the profile is attributed to. -- name: play_store_attribution_source +- name: adjust_campaign type: STRING mode: NULLABLE - description: Play store source the profile is attributed to. + description: Adjust Campaign the profile is attributed to. -- name: meta_attribution_app +- name: adjust_creative type: STRING mode: NULLABLE - description: Facebook app linked to paid marketing. + description: Adjust Creative the profile is attributed to. -- name: install_source +- name: adjust_network type: STRING mode: NULLABLE - description: The source of a profile installation. + description: Adjust Network the profile is attributed to. - name: dau type: INTEGER diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/retention_v1/query.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/retention_v1/query.sql --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/retention_v1/query.sql 2024-06-03 10:44:11.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/retention_v1/query.sql 2024-06-03 10:46:14.000000000 +0000 @@ -8,15 +8,15 @@ app_version, locale, is_mobile, - adjust_ad_group, - adjust_campaign, - adjust_creative, - adjust_network, play_store_attribution_campaign, play_store_attribution_medium, play_store_attribution_source, meta_attribution_app, install_source, + adjust_ad_group, + adjust_campaign, + adjust_creative, + adjust_network, COUNTIF(ping_sent_metric_date) AS ping_sent_metric_date, COUNTIF(ping_sent_week_4) AS ping_sent_week_4, COUNTIF(active_metric_date) AS active_metric_date, @@ -38,12 +38,12 @@ app_version, locale, is_mobile, - adjust_ad_group, - adjust_campaign, - adjust_creative, - adjust_network, play_store_attribution_campaign, play_store_attribution_medium, play_store_attribution_source, meta_attribution_app, - install_source + install_source, + adjust_ad_group, + adjust_campaign, + adjust_creative, + adjust_network diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/retention_v1/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/retention_v1/schema.yaml --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/retention_v1/schema.yaml 2024-06-03 10:44:11.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/retention_v1/schema.yaml 2024-06-03 10:46:14.000000000 +0000 @@ -1,5 +1,4 @@ fields: - - mode: NULLABLE name: metric_date type: DATE @@ -40,50 +39,50 @@ mode: NULLABLE description: Indicates if this specific entry is used towards calculating mobile DAU. -- name: adjust_ad_group +- name: play_store_attribution_campaign type: STRING mode: NULLABLE - description: Adjust Ad Group the profile is attributed to. + description: Play store campaign the profile is attributed to. -- name: adjust_campaign +- name: play_store_attribution_medium type: STRING mode: NULLABLE - description: Adjust Campaign the profile is attributed to. + description: Play store Medium the profile is attributed to. -- name: adjust_creative +- name: play_store_attribution_source type: STRING mode: NULLABLE - description: Adjust Creative the profile is attributed to. + description: Play store source the profile is attributed to. -- name: adjust_network +- name: meta_attribution_app type: STRING mode: NULLABLE - description: Adjust Network the profile is attributed to. + description: Facebook app linked to paid marketing. -- name: play_store_attribution_campaign +- name: install_source type: STRING mode: NULLABLE - description: Play store campaign the profile is attributed to. + description: The source of a profile installation. -- name: play_store_attribution_medium +- name: adjust_ad_group type: STRING mode: NULLABLE - description: Play store Medium the profile is attributed to. + description: Adjust Ad Group the profile is attributed to. -- name: play_store_attribution_source +- name: adjust_campaign type: STRING mode: NULLABLE - description: Play store source the profile is attributed to. + description: Adjust Campaign the profile is attributed to. -- name: meta_attribution_app +- name: adjust_creative type: STRING mode: NULLABLE - description: Facebook app linked to paid marketing. + description: Adjust Creative the profile is attributed to. -- name: install_source +- name: adjust_network type: STRING mode: NULLABLE - description: The source of a profile installation. + description: Adjust Network the profile is attributed to. - name: ping_sent_metric_date type: INTEGER diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop/metrics/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop/metrics/schema.yaml --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop/metrics/schema.yaml 2024-06-03 10:44:11.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop/metrics/schema.yaml 2024-06-03 10:45:16.000000000 +0000 @@ -2538,7 +2538,7 @@ mode: NULLABLE description: 'Counts how often we purge trackers. Giving a high level overview about the effectivness of bounce tracking protection. Allows determining error - rate with failed/success label. When in dry mode, we count the purges that + rate with failure/success label. When in dry mode, we count the purges that would have happened under the dry label. ' diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop/use_counters/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop/use_counters/schema.yaml --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop/use_counters/schema.yaml 2024-06-03 10:44:11.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop/use_counters/schema.yaml 1970-01-01 00:00:00.000000000 +0000 @@ -1,18056 +0,0 @@ -fields: -- name: additional_properties - type: STRING - mode: NULLABLE - description: A JSON string containing any payload properties not present in the - schema -- name: client_info - type: RECORD - mode: NULLABLE - fields: - - name: android_sdk_version - type: STRING - mode: NULLABLE - description: The optional Android specific SDK version of the software running - on this hardware device. - - name: app_build - type: STRING - mode: NULLABLE - description: The build identifier generated by the CI system (e.g. "1234/A"). - For language bindings that provide automatic detection for this value, (e.g. - Android/Kotlin), in the unlikely event that the build identifier can not be - retrieved from the OS, it is set to "inaccessible". For other language bindings, - if the value was not provided through configuration, this metric gets set to - `Unknown`. - - name: app_channel - type: STRING - mode: NULLABLE - description: The channel the application is being distributed on. - - name: app_display_version - type: STRING - mode: NULLABLE - description: The user visible version string (e.g. "1.0.3"). In the unlikely - event that the display version can not be retrieved, it is set to "inaccessible". - - name: architecture - type: STRING - mode: NULLABLE - description: The architecture of the device, (e.g. "arm", "x86"). - - name: build_date - type: STRING - mode: NULLABLE - description: The date & time the application was built - - name: client_id - type: STRING - mode: NULLABLE - description: A UUID uniquely identifying the client. - - name: device_manufacturer - type: STRING - mode: NULLABLE - description: The manufacturer of the device the application is running on. Not - set if the device manufacturer can't be determined (e.g. on Desktop). - - name: device_model - type: STRING - mode: NULLABLE - description: The model of the device the application is running on. On Android, - this is Build.MODEL, the user-visible marketing name, like "Pixel 2 XL". Not - set if the device model can't be determined (e.g. on Desktop). - - name: first_run_date - type: STRING - mode: NULLABLE - description: The date of the first run of the application. - - name: locale - type: STRING - mode: NULLABLE - description: The locale of the application during initialization (e.g. "es-ES"). - If the locale can't be determined on the system, the value is ["und"](https://unicode.org/reports/tr35/#Unknown_or_Invalid_Identifiers), - to indicate "undetermined". - - name: os - type: STRING - mode: NULLABLE - description: 'The name of the operating system. Possible values: Android, iOS, - Linux, Darwin, Windows, FreeBSD, NetBSD, OpenBSD, Solaris, unknown' - - name: os_version - type: STRING - mode: NULLABLE - description: The user-visible version of the operating system (e.g. "1.2.3"). - If the version detection fails, this metric gets set to `Unknown`. - - name: telemetry_sdk_build - type: STRING - mode: NULLABLE - description: The version of the Glean SDK - - name: windows_build_number - type: INTEGER - mode: NULLABLE - description: The optional Windows build number, reported by Windows (e.g. 22000) - and not set for other platforms - - name: session_count - type: INTEGER - mode: NULLABLE - description: An optional running counter of the number of sessions for a client. - - name: session_id - type: STRING - mode: NULLABLE - description: An optional UUID uniquely identifying the client's current session. -- name: document_id - type: STRING - mode: NULLABLE - description: The document ID specified in the URI when the client sent this message -- name: events - type: RECORD - mode: REPEATED - fields: - - name: category - type: STRING - mode: NULLABLE - - name: extra - type: RECORD - mode: REPEATED - fields: - - name: key - type: STRING - mode: NULLABLE - - name: value - type: STRING - mode: NULLABLE - - name: name - type: STRING - mode: NULLABLE - - name: timestamp - type: INTEGER - mode: NULLABLE -- name: metadata - type: RECORD - mode: NULLABLE - fields: - - name: geo - type: RECORD - mode: NULLABLE - fields: - - name: city - type: STRING - mode: NULLABLE - - name: country - type: STRING - mode: NULLABLE - description: An ISO 3166-1 alpha-2 country code - - name: db_version - type: STRING - mode: NULLABLE - description: The specific geo database version used for this lookup - - name: subdivision1 - type: STRING - mode: NULLABLE - description: First major country subdivision, typically a state, province, or - county - - name: subdivision2 - type: STRING - mode: NULLABLE - description: Second major country subdivision; not applicable for most countries - description: Results of a geographic lookup based on the client's IP address - - name: header - type: RECORD - mode: NULLABLE - fields: - - name: date - type: STRING - mode: NULLABLE - description: Date HTTP header - - name: dnt - type: STRING - mode: NULLABLE - description: DNT (Do Not Track) HTTP header - - name: x_debug_id - type: STRING - mode: NULLABLE - description: X-Debug-Id HTTP header - - name: x_foxsec_ip_reputation - type: STRING - mode: NULLABLE - description: X-Foxsec-IP-Reputation header - - name: x_lb_tags - type: STRING - mode: NULLABLE - description: X-LB-Tags HTTP header - - name: x_pingsender_version - type: STRING - mode: NULLABLE - description: X-PingSender-Version HTTP header - - name: x_source_tags - type: STRING - mode: NULLABLE - description: X-Source-Tags HTTP header - - name: x_telemetry_agent - type: STRING - mode: NULLABLE - description: X-Telemetry-Agent HTTP header - - name: parsed_date - type: TIMESTAMP - mode: NULLABLE - - name: parsed_x_source_tags - type: STRING - mode: REPEATED - - name: parsed_x_lb_tags - type: RECORD - mode: NULLABLE - fields: - - name: tls_version - type: STRING - mode: NULLABLE - - name: tls_cipher_hex - type: STRING - mode: NULLABLE - description: Headers included in the client's HTTP request - - name: isp - type: RECORD - mode: NULLABLE - fields: - - name: db_version - type: STRING - mode: NULLABLE - description: The specific geo ISP database version used for this lookup - - name: name - type: STRING - mode: NULLABLE - description: The name of the ISP associated with the client's IP address - - name: organization - type: STRING - mode: NULLABLE - description: The name of a specific business entity associated with the client's - IP address when available; otherwise the ISP name - description: Results of ISP lookup based on the client's IP address - - name: user_agent - type: RECORD - mode: NULLABLE - fields: - - name: browser - type: STRING - mode: NULLABLE - - name: os - type: STRING - mode: NULLABLE - - name: version - type: STRING - mode: NULLABLE - description: Parsed components of the client's user agent string -- name: metrics - type: RECORD - mode: NULLABLE - fields: - - name: counter - type: RECORD - mode: NULLABLE - fields: - - name: use_counter_content_documents_destroyed - type: INTEGER - mode: NULLABLE - description: 'A count of how many content documents were destroyed. Used to - turn document use counters'' counts into rates. Excludes documents for which - we do not count use counters (See `Document::ShouldIncludeInTelemetry`). - - ' - - name: use_counter_css_doc_alignment_baseline - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the (unknown, counted) CSS property alignment-baseline. - Compare against `use.counter.content_documents_destroyed` to calculate the - rate. - - ' - - name: use_counter_css_doc_background_repeat_x - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the (unknown, counted) CSS property background-repeat-x. - Compare against `use.counter.content_documents_destroyed` to calculate the - rate. - - ' - - name: use_counter_css_doc_background_repeat_y - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the (unknown, counted) CSS property background-repeat-y. - Compare against `use.counter.content_documents_destroyed` to calculate the - rate. - - ' - - name: use_counter_css_doc_baseline_shift - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the (unknown, counted) CSS property baseline-shift. - Compare against `use.counter.content_documents_destroyed` to calculate the - rate. - - ' - - name: use_counter_css_doc_buffered_rendering - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the (unknown, counted) CSS property buffered-rendering. - Compare against `use.counter.content_documents_destroyed` to calculate the - rate. - - ' - - name: use_counter_css_doc_color_rendering - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the (unknown, counted) CSS property color-rendering. - Compare against `use.counter.content_documents_destroyed` to calculate the - rate. - - ' - - name: use_counter_css_doc_css_accent_color - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property accent-color. Compare - against `use.counter.content_documents_destroyed` to calculate the rate. - - ' - - name: use_counter_css_doc_css_align_content - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property align-content. Compare - against `use.counter.content_documents_destroyed` to calculate the rate. - - ' - - name: use_counter_css_doc_css_align_items - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property align-items. Compare - against `use.counter.content_documents_destroyed` to calculate the rate. - - ' - - name: use_counter_css_doc_css_align_self - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property align-self. Compare against - `use.counter.content_documents_destroyed` to calculate the rate. - - ' - - name: use_counter_css_doc_css_align_tracks - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property align-tracks. Compare - against `use.counter.content_documents_destroyed` to calculate the rate. - - ' - - name: use_counter_css_doc_css_all - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property all. Compare against - `use.counter.content_documents_destroyed` to calculate the rate. - - ' - - name: use_counter_css_doc_css_animation - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property animation. Compare against - `use.counter.content_documents_destroyed` to calculate the rate. - - ' - - name: use_counter_css_doc_css_animation_composition - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property animation-composition. - Compare against `use.counter.content_documents_destroyed` to calculate the - rate. - - ' - - name: use_counter_css_doc_css_animation_delay - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property animation-delay. Compare - against `use.counter.content_documents_destroyed` to calculate the rate. - - ' - - name: use_counter_css_doc_css_animation_direction - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property animation-direction. - Compare against `use.counter.content_documents_destroyed` to calculate the - rate. - - ' - - name: use_counter_css_doc_css_animation_duration - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property animation-duration. Compare - against `use.counter.content_documents_destroyed` to calculate the rate. - - ' - - name: use_counter_css_doc_css_animation_fill_mode - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property animation-fill-mode. - Compare against `use.counter.content_documents_destroyed` to calculate the - rate. - - ' - - name: use_counter_css_doc_css_animation_iteration_count - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property animation-iteration-count. - Compare against `use.counter.content_documents_destroyed` to calculate the - rate. - - ' - - name: use_counter_css_doc_css_animation_name - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property animation-name. Compare - against `use.counter.content_documents_destroyed` to calculate the rate. - - ' - - name: use_counter_css_doc_css_animation_play_state - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property animation-play-state. - Compare against `use.counter.content_documents_destroyed` to calculate the - rate. - - ' - - name: use_counter_css_doc_css_animation_timeline - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property animation-timeline. Compare - against `use.counter.content_documents_destroyed` to calculate the rate. - - ' - - name: use_counter_css_doc_css_animation_timing_function - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property animation-timing-function. - Compare against `use.counter.content_documents_destroyed` to calculate the - rate. - - ' - - name: use_counter_css_doc_css_appearance - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property appearance. Compare against - `use.counter.content_documents_destroyed` to calculate the rate. - - ' - - name: use_counter_css_doc_css_aspect_ratio - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property aspect-ratio. Compare - against `use.counter.content_documents_destroyed` to calculate the rate. - - ' - - name: use_counter_css_doc_css_backdrop_filter - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property backdrop-filter. Compare - against `use.counter.content_documents_destroyed` to calculate the rate. - - ' - - name: use_counter_css_doc_css_backface_visibility - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property backface-visibility. - Compare against `use.counter.content_documents_destroyed` to calculate the - rate. - - ' - - name: use_counter_css_doc_css_background - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property background. Compare against - `use.counter.content_documents_destroyed` to calculate the rate. - - ' - - name: use_counter_css_doc_css_background_attachment - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property background-attachment. - Compare against `use.counter.content_documents_destroyed` to calculate the - rate. - - ' - - name: use_counter_css_doc_css_background_blend_mode - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property background-blend-mode. - Compare against `use.counter.content_documents_destroyed` to calculate the - rate. - - ' - - name: use_counter_css_doc_css_background_clip - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property background-clip. Compare - against `use.counter.content_documents_destroyed` to calculate the rate. - - ' - - name: use_counter_css_doc_css_background_color - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property background-color. Compare - against `use.counter.content_documents_destroyed` to calculate the rate. - - ' - - name: use_counter_css_doc_css_background_image - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property background-image. Compare - against `use.counter.content_documents_destroyed` to calculate the rate. - - ' - - name: use_counter_css_doc_css_background_origin - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property background-origin. Compare - against `use.counter.content_documents_destroyed` to calculate the rate. - - ' - - name: use_counter_css_doc_css_background_position - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property background-position. - Compare against `use.counter.content_documents_destroyed` to calculate the - rate. - - ' - - name: use_counter_css_doc_css_background_position_x - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property background-position-x. - Compare against `use.counter.content_documents_destroyed` to calculate the - rate. - - ' - - name: use_counter_css_doc_css_background_position_y - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property background-position-y. - Compare against `use.counter.content_documents_destroyed` to calculate the - rate. - - ' - - name: use_counter_css_doc_css_background_repeat - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property background-repeat. Compare - against `use.counter.content_documents_destroyed` to calculate the rate. - - ' - - name: use_counter_css_doc_css_background_size - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property background-size. Compare - against `use.counter.content_documents_destroyed` to calculate the rate. - - ' - - name: use_counter_css_doc_css_baseline_source - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property baseline-source. Compare - against `use.counter.content_documents_destroyed` to calculate the rate. - - ' - - name: use_counter_css_doc_css_block_size - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property block-size. Compare against - `use.counter.content_documents_destroyed` to calculate the rate. - - ' - - name: use_counter_css_doc_css_border - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border. Compare against - `use.counter.content_documents_destroyed` to calculate the rate. - - ' - - name: use_counter_css_doc_css_border_block - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-block. Compare - against `use.counter.content_documents_destroyed` to calculate the rate. - - ' - - name: use_counter_css_doc_css_border_block_color - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-block-color. Compare - against `use.counter.content_documents_destroyed` to calculate the rate. - - ' - - name: use_counter_css_doc_css_border_block_end - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-block-end. Compare - against `use.counter.content_documents_destroyed` to calculate the rate. - - ' - - name: use_counter_css_doc_css_border_block_end_color - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-block-end-color. - Compare against `use.counter.content_documents_destroyed` to calculate the - rate. - - ' - - name: use_counter_css_doc_css_border_block_end_style - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-block-end-style. - Compare against `use.counter.content_documents_destroyed` to calculate the - rate. - - ' - - name: use_counter_css_doc_css_border_block_end_width - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-block-end-width. - Compare against `use.counter.content_documents_destroyed` to calculate the - rate. - - ' - - name: use_counter_css_doc_css_border_block_start - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-block-start. Compare - against `use.counter.content_documents_destroyed` to calculate the rate. - - ' - - name: use_counter_css_doc_css_border_block_start_color - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-block-start-color. - Compare against `use.counter.content_documents_destroyed` to calculate the - rate. - - ' - - name: use_counter_css_doc_css_border_block_start_style - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-block-start-style. - Compare against `use.counter.content_documents_destroyed` to calculate the - rate. - - ' - - name: use_counter_css_doc_css_border_block_start_width - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-block-start-width. - Compare against `use.counter.content_documents_destroyed` to calculate the - rate. - - ' - - name: use_counter_css_doc_css_border_block_style - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-block-style. Compare - against `use.counter.content_documents_destroyed` to calculate the rate. - - ' - - name: use_counter_css_doc_css_border_block_width - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-block-width. Compare - against `use.counter.content_documents_destroyed` to calculate the rate. - - ' - - name: use_counter_css_doc_css_border_bottom - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-bottom. Compare - against `use.counter.content_documents_destroyed` to calculate the rate. - - ' - - name: use_counter_css_doc_css_border_bottom_color - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-bottom-color. - Compare against `use.counter.content_documents_destroyed` to calculate the - rate. - - ' - - name: use_counter_css_doc_css_border_bottom_left_radius - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-bottom-left-radius. - Compare against `use.counter.content_documents_destroyed` to calculate the - rate. - - ' - - name: use_counter_css_doc_css_border_bottom_right_radius - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-bottom-right-radius. - Compare against `use.counter.content_documents_destroyed` to calculate the - rate. - - ' - - name: use_counter_css_doc_css_border_bottom_style - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-bottom-style. - Compare against `use.counter.content_documents_destroyed` to calculate the - rate. - - ' - - name: use_counter_css_doc_css_border_bottom_width - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-bottom-width. - Compare against `use.counter.content_documents_destroyed` to calculate the - rate. - - ' - - name: use_counter_css_doc_css_border_collapse - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-collapse. Compare - against `use.counter.content_documents_destroyed` to calculate the rate. - - ' - - name: use_counter_css_doc_css_border_color - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-color. Compare - against `use.counter.content_documents_destroyed` to calculate the rate. - - ' - - name: use_counter_css_doc_css_border_end_end_radius - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-end-end-radius. - Compare against `use.counter.content_documents_destroyed` to calculate the - rate. - - ' - - name: use_counter_css_doc_css_border_end_start_radius - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-end-start-radius. - Compare against `use.counter.content_documents_destroyed` to calculate the - rate. - - ' - - name: use_counter_css_doc_css_border_image - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-image. Compare - against `use.counter.content_documents_destroyed` to calculate the rate. - - ' - - name: use_counter_css_doc_css_border_image_outset - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-image-outset. - Compare against `use.counter.content_documents_destroyed` to calculate the - rate. - - ' - - name: use_counter_css_doc_css_border_image_repeat - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-image-repeat. - Compare against `use.counter.content_documents_destroyed` to calculate the - rate. - - ' - - name: use_counter_css_doc_css_border_image_slice - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-image-slice. Compare - against `use.counter.content_documents_destroyed` to calculate the rate. - - ' - - name: use_counter_css_doc_css_border_image_source - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-image-source. - Compare against `use.counter.content_documents_destroyed` to calculate the - rate. - - ' - - name: use_counter_css_doc_css_border_image_width - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-image-width. Compare - against `use.counter.content_documents_destroyed` to calculate the rate. - - ' - - name: use_counter_css_doc_css_border_inline - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-inline. Compare - against `use.counter.content_documents_destroyed` to calculate the rate. - - ' - - name: use_counter_css_doc_css_border_inline_color - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-inline-color. - Compare against `use.counter.content_documents_destroyed` to calculate the - rate. - - ' - - name: use_counter_css_doc_css_border_inline_end - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-inline-end. Compare - against `use.counter.content_documents_destroyed` to calculate the rate. - - ' - - name: use_counter_css_doc_css_border_inline_end_color - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-inline-end-color. - Compare against `use.counter.content_documents_destroyed` to calculate the - rate. - - ' - - name: use_counter_css_doc_css_border_inline_end_style - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-inline-end-style. - Compare against `use.counter.content_documents_destroyed` to calculate the - rate. - - ' - - name: use_counter_css_doc_css_border_inline_end_width - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-inline-end-width. - Compare against `use.counter.content_documents_destroyed` to calculate the - rate. - - ' - - name: use_counter_css_doc_css_border_inline_start - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-inline-start. - Compare against `use.counter.content_documents_destroyed` to calculate the - rate. - - ' - - name: use_counter_css_doc_css_border_inline_start_color - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-inline-start-color. - Compare against `use.counter.content_documents_destroyed` to calculate the - rate. - - ' - - name: use_counter_css_doc_css_border_inline_start_style - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-inline-start-style. - Compare against `use.counter.content_documents_destroyed` to calculate the - rate. - - ' - - name: use_counter_css_doc_css_border_inline_start_width - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-inline-start-width. - Compare against `use.counter.content_documents_destroyed` to calculate the - rate. - - ' - - name: use_counter_css_doc_css_border_inline_style - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-inline-style. - Compare against `use.counter.content_documents_destroyed` to calculate the - rate. - - ' - - name: use_counter_css_doc_css_border_inline_width - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-inline-width. - Compare against `use.counter.content_documents_destroyed` to calculate the - rate. - - ' - - name: use_counter_css_doc_css_border_left - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-left. Compare - against `use.counter.content_documents_destroyed` to calculate the rate. - - ' - - name: use_counter_css_doc_css_border_left_color - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-left-color. Compare - against `use.counter.content_documents_destroyed` to calculate the rate. - - ' - - name: use_counter_css_doc_css_border_left_style - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-left-style. Compare - against `use.counter.content_documents_destroyed` to calculate the rate. - - ' - - name: use_counter_css_doc_css_border_left_width - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-left-width. Compare - against `use.counter.content_documents_destroyed` to calculate the rate. - - ' - - name: use_counter_css_doc_css_border_radius - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-radius. Compare - against `use.counter.content_documents_destroyed` to calculate the rate. - - ' - - name: use_counter_css_doc_css_border_right - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-right. Compare - against `use.counter.content_documents_destroyed` to calculate the rate. - - ' - - name: use_counter_css_doc_css_border_right_color - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-right-color. Compare - against `use.counter.content_documents_destroyed` to calculate the rate. - - ' - - name: use_counter_css_doc_css_border_right_style - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-right-style. Compare - against `use.counter.content_documents_destroyed` to calculate the rate. - - ' - - name: use_counter_css_doc_css_border_right_width - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-right-width. Compare - against `use.counter.content_documents_destroyed` to calculate the rate. - - ' - - name: use_counter_css_doc_css_border_spacing - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-spacing. Compare - against `use.counter.content_documents_destroyed` to calculate the rate. - - ' - - name: use_counter_css_doc_css_border_start_end_radius - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-start-end-radius. - Compare against `use.counter.content_documents_destroyed` to calculate the - rate. - - ' - - name: use_counter_css_doc_css_border_start_start_radius - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-start-start-radius. - Compare against `use.counter.content_documents_destroyed` to calculate the - rate. - - ' - - name: use_counter_css_doc_css_border_style - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-style. Compare - against `use.counter.content_documents_destroyed` to calculate the rate. - - ' - - name: use_counter_css_doc_css_border_top - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-top. Compare against - `use.counter.content_documents_destroyed` to calculate the rate. - - ' - - name: use_counter_css_doc_css_border_top_color - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-top-color. Compare - against `use.counter.content_documents_destroyed` to calculate the rate. - - ' - - name: use_counter_css_doc_css_border_top_left_radius - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-top-left-radius. - Compare against `use.counter.content_documents_destroyed` to calculate the - rate. - - ' - - name: use_counter_css_doc_css_border_top_right_radius - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-top-right-radius. - Compare against `use.counter.content_documents_destroyed` to calculate the - rate. - - ' - - name: use_counter_css_doc_css_border_top_style - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-top-style. Compare - against `use.counter.content_documents_dest ```

⚠️ Only part of the diff is displayed.

Link to full diff

dataops-ci-bot commented 1 month ago

Integration report for "feat: add is_init() to mobile kpi generator queries"

sql.diff

Click to expand! ```diff Only in /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop/use_counters: schema.yaml Only in /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop/user_characteristics: schema.yaml Only in /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_background_update/use_counters: schema.yaml Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android: engagement Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android: engagement_clients Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android: retention Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android: retention_clients Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android_derived: engagement_v1 Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android_derived: retention_v1 Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_ios: engagement Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_ios: engagement_clients Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_ios: retention Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_ios: retention_clients Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_ios_derived: engagement_v1 Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_ios_derived: retention_v1 Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/klar_android: engagement Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/klar_android: engagement_clients Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/klar_android: retention Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/klar_android: retention_clients Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/klar_android_derived: engagement_v1 Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/klar_android_derived: retention_v1 Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/klar_ios: engagement Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/klar_ios: engagement_clients Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/klar_ios: retention Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/klar_ios: retention_clients Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/klar_ios_derived: engagement_v1 Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/klar_ios_derived: retention_v1 Only in /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/org_mozilla_fenix/use_counters: schema.yaml Only in /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/org_mozilla_fenix_nightly/use_counters: schema.yaml Only in /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/org_mozilla_fennec_aurora/use_counters: schema.yaml Only in /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/org_mozilla_firefox/use_counters: schema.yaml Only in /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/org_mozilla_firefox_beta/use_counters: schema.yaml Only in /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/org_mozilla_focus/use_counters: schema.yaml Only in /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/org_mozilla_focus_beta/use_counters: schema.yaml Only in /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/org_mozilla_focus_nightly/use_counters: schema.yaml Only in /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/org_mozilla_klar/use_counters: schema.yaml Only in /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/pine/use_counters: schema.yaml diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_backend_derived/event_monitoring_live_v1/materialized_view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_backend_derived/event_monitoring_live_v1/materialized_view.sql --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_backend_derived/event_monitoring_live_v1/materialized_view.sql 2024-06-03 14:00:50.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_backend_derived/event_monitoring_live_v1/materialized_view.sql 2024-06-03 14:02:30.000000000 +0000 @@ -60,7 +60,7 @@ LEFT JOIN UNNEST(event.extra) AS event_extra WHERE - DATE(submission_timestamp) >= "2024-05-31" + DATE(submission_timestamp) >= "2024-06-03" GROUP BY submission_date, window_start, diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_frontend_derived/event_monitoring_live_v1/materialized_view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_frontend_derived/event_monitoring_live_v1/materialized_view.sql --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_frontend_derived/event_monitoring_live_v1/materialized_view.sql 2024-06-03 14:00:51.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_frontend_derived/event_monitoring_live_v1/materialized_view.sql 2024-06-03 14:02:30.000000000 +0000 @@ -60,7 +60,7 @@ LEFT JOIN UNNEST(event.extra) AS event_extra WHERE - DATE(submission_timestamp) >= "2024-05-31" + DATE(submission_timestamp) >= "2024-06-03" GROUP BY submission_date, window_start, diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/bedrock_derived/event_monitoring_live_v1/materialized_view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/bedrock_derived/event_monitoring_live_v1/materialized_view.sql --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/bedrock_derived/event_monitoring_live_v1/materialized_view.sql 2024-06-03 14:00:51.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/bedrock_derived/event_monitoring_live_v1/materialized_view.sql 2024-06-03 14:02:30.000000000 +0000 @@ -50,7 +50,7 @@ client_info.app_display_version AS version, ping_info FROM - `moz-fx-data-shared-prod.bedrock_live.events_v1` + `moz-fx-data-shared-prod.bedrock_live.non_interaction_v1` UNION ALL SELECT submission_timestamp, @@ -60,7 +60,7 @@ client_info.app_display_version AS version, ping_info FROM - `moz-fx-data-shared-prod.bedrock_live.interaction_v1` + `moz-fx-data-shared-prod.bedrock_live.events_v1` UNION ALL SELECT submission_timestamp, @@ -70,7 +70,7 @@ client_info.app_display_version AS version, ping_info FROM - `moz-fx-data-shared-prod.bedrock_live.non_interaction_v1` + `moz-fx-data-shared-prod.bedrock_live.interaction_v1` ) CROSS JOIN UNNEST(events) AS event, @@ -80,7 +80,7 @@ LEFT JOIN UNNEST(event.extra) AS event_extra WHERE - DATE(submission_timestamp) >= "2024-05-31" + DATE(submission_timestamp) >= "2024-06-03" GROUP BY submission_date, window_start, diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates/schema.yaml --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates/schema.yaml 2024-06-03 14:00:09.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates/schema.yaml 2024-06-03 14:09:52.000000000 +0000 @@ -1,49 +1,49 @@ fields: -- mode: NULLABLE - name: submission_date +- name: submission_date type: DATE -- mode: NULLABLE - name: source + mode: NULLABLE +- name: source type: STRING -- mode: NULLABLE - name: event_type + mode: NULLABLE +- name: event_type type: STRING -- mode: NULLABLE - name: form_factor + mode: NULLABLE +- name: form_factor type: STRING -- mode: NULLABLE - name: country + mode: NULLABLE +- name: country type: STRING -- mode: NULLABLE - name: subdivision1 + mode: NULLABLE +- name: subdivision1 type: STRING -- mode: NULLABLE - name: advertiser + mode: NULLABLE +- name: advertiser type: STRING -- mode: NULLABLE - name: release_channel + mode: NULLABLE +- name: release_channel type: STRING -- mode: NULLABLE - name: position + mode: NULLABLE +- name: position type: INTEGER -- mode: NULLABLE - name: provider + mode: NULLABLE +- name: provider type: STRING -- mode: NULLABLE - name: match_type + mode: NULLABLE +- name: match_type type: STRING -- mode: NULLABLE - name: normalized_os + mode: NULLABLE +- name: normalized_os type: STRING -- mode: NULLABLE - name: suggest_data_sharing_enabled + mode: NULLABLE +- name: suggest_data_sharing_enabled type: BOOLEAN -- mode: NULLABLE - name: event_count + mode: NULLABLE +- name: event_count type: INTEGER -- mode: NULLABLE - name: user_count + mode: NULLABLE +- name: user_count type: INTEGER -- mode: NULLABLE - name: query_type + mode: NULLABLE +- name: query_type type: STRING + mode: NULLABLE diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates_suggest/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates_suggest/schema.yaml --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates_suggest/schema.yaml 2024-06-03 14:00:09.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates_suggest/schema.yaml 2024-06-03 14:09:52.000000000 +0000 @@ -1,40 +1,40 @@ fields: -- mode: NULLABLE - name: submission_date +- name: submission_date type: DATE -- mode: NULLABLE - name: form_factor + mode: NULLABLE +- name: form_factor type: STRING -- mode: NULLABLE - name: country + mode: NULLABLE +- name: country type: STRING -- mode: NULLABLE - name: advertiser + mode: NULLABLE +- name: advertiser type: STRING -- mode: NULLABLE - name: normalized_os + mode: NULLABLE +- name: normalized_os type: STRING -- mode: NULLABLE - name: release_channel + mode: NULLABLE +- name: release_channel type: STRING -- mode: NULLABLE - name: position + mode: NULLABLE +- name: position type: INTEGER -- mode: NULLABLE - name: provider + mode: NULLABLE +- name: provider type: STRING -- mode: NULLABLE - name: match_type + mode: NULLABLE +- name: match_type type: STRING -- mode: NULLABLE - name: suggest_data_sharing_enabled + mode: NULLABLE +- name: suggest_data_sharing_enabled type: BOOLEAN -- mode: NULLABLE - name: impression_count + mode: NULLABLE +- name: impression_count type: INTEGER -- mode: NULLABLE - name: click_count + mode: NULLABLE +- name: click_count type: INTEGER -- mode: NULLABLE - name: query_type + mode: NULLABLE +- name: query_type type: STRING + mode: NULLABLE diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/debug_ping_view_derived/event_monitoring_live_v1/materialized_view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/debug_ping_view_derived/event_monitoring_live_v1/materialized_view.sql --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/debug_ping_view_derived/event_monitoring_live_v1/materialized_view.sql 2024-06-03 14:00:51.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/debug_ping_view_derived/event_monitoring_live_v1/materialized_view.sql 2024-06-03 14:02:30.000000000 +0000 @@ -60,7 +60,7 @@ LEFT JOIN UNNEST(event.extra) AS event_extra WHERE - DATE(submission_timestamp) >= "2024-05-31" + DATE(submission_timestamp) >= "2024-06-03" GROUP BY submission_date, window_start, diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/engagement_clients/view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/engagement_clients/view.sql --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/engagement_clients/view.sql 2024-06-03 14:00:51.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/engagement_clients/view.sql 2024-06-03 14:02:29.000000000 +0000 @@ -26,19 +26,19 @@ client_id, sample_id, channel AS normalized_channel, + NULLIF(play_store_attribution_campaign, "") AS play_store_attribution_campaign, + NULLIF(play_store_attribution_medium, "") AS play_store_attribution_medium, + NULLIF(play_store_attribution_source, "") AS play_store_attribution_source, + NULLIF(meta_attribution_app, "") AS meta_attribution_app, + NULLIF(install_source, "") AS install_source, NULLIF(adjust_ad_group, "") AS adjust_ad_group, - NULLIF(adjust_creative, "") AS adjust_creative, - NULLIF(adjust_network, "") AS adjust_network, CASE WHEN adjust_network IN ('Google Organic Search', 'Organic') THEN 'Organic' ELSE NULLIF(adjust_campaign, "") END AS adjust_campaign, - NULLIF(play_store_attribution_campaign, "") AS play_store_attribution_campaign, - NULLIF(play_store_attribution_medium, "") AS play_store_attribution_medium, - NULLIF(play_store_attribution_source, "") AS play_store_attribution_source, - NULLIF(meta_attribution_app, "") AS meta_attribution_app, - NULLIF(install_source, "") AS install_source, + NULLIF(adjust_creative, "") AS adjust_creative, + NULLIF(adjust_network, "") AS adjust_network, FROM `moz-fx-data-shared-prod.fenix_derived.firefox_android_clients_v1` ) @@ -57,15 +57,15 @@ is_wau, is_mau, is_mobile, - attribution.adjust_ad_group, - attribution.adjust_campaign, - attribution.adjust_creative, - attribution.adjust_network, attribution.play_store_attribution_campaign, attribution.play_store_attribution_medium, attribution.play_store_attribution_source, attribution.meta_attribution_app, attribution.install_source, + attribution.adjust_ad_group, + attribution.adjust_campaign, + attribution.adjust_creative, + attribution.adjust_network, CASE WHEN active_users.submission_date = first_seen_date THEN 'new_profile' diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_clients/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_clients/schema.yaml --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_clients/schema.yaml 2024-06-03 14:00:09.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_clients/schema.yaml 2024-06-03 14:09:10.000000000 +0000 @@ -26,6 +26,9 @@ - name: adjust_network type: STRING mode: NULLABLE +- name: install_source + type: STRING + mode: NULLABLE - name: retained_week_2 type: BOOLEAN mode: NULLABLE diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_week_4/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_week_4/schema.yaml --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_week_4/schema.yaml 2024-06-03 14:00:09.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_week_4/schema.yaml 2024-06-03 14:09:10.000000000 +0000 @@ -48,6 +48,10 @@ description: 'The type of source of a client installation. ' +- name: install_source + type: STRING + mode: NULLABLE + description: null - name: new_profiles type: INTEGER mode: NULLABLE diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/retention_clients/view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/retention_clients/view.sql --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/retention_clients/view.sql 2024-06-03 14:00:51.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/retention_clients/view.sql 2024-06-03 14:02:29.000000000 +0000 @@ -22,19 +22,19 @@ client_id, sample_id, channel AS normalized_channel, + NULLIF(play_store_attribution_campaign, "") AS play_store_attribution_campaign, + NULLIF(play_store_attribution_medium, "") AS play_store_attribution_medium, + NULLIF(play_store_attribution_source, "") AS play_store_attribution_source, + NULLIF(meta_attribution_app, "") AS meta_attribution_app, + NULLIF(install_source, "") AS install_source, NULLIF(adjust_ad_group, "") AS adjust_ad_group, - NULLIF(adjust_creative, "") AS adjust_creative, - NULLIF(adjust_network, "") AS adjust_network, CASE WHEN adjust_network IN ('Google Organic Search', 'Organic') THEN 'Organic' ELSE NULLIF(adjust_campaign, "") END AS adjust_campaign, - NULLIF(play_store_attribution_campaign, "") AS play_store_attribution_campaign, - NULLIF(play_store_attribution_medium, "") AS play_store_attribution_medium, - NULLIF(play_store_attribution_source, "") AS play_store_attribution_source, - NULLIF(meta_attribution_app, "") AS meta_attribution_app, - NULLIF(install_source, "") AS install_source, + NULLIF(adjust_creative, "") AS adjust_creative, + NULLIF(adjust_network, "") AS adjust_network, FROM `moz-fx-data-shared-prod.fenix_derived.firefox_android_clients_v1` ) @@ -56,6 +56,10 @@ attribution.play_store_attribution_source, attribution.meta_attribution_app, attribution.install_source, + attribution.adjust_ad_group, + attribution.adjust_campaign, + attribution.adjust_creative, + attribution.adjust_network, -- ping sent retention active_users.retention_seen.day_27.active_on_metric_date AS ping_sent_metric_date, ( @@ -79,10 +83,6 @@ -- Looking back at 27 days to support the official definition of repeat_profile (someone active between days 2 and 28): AND BIT_COUNT(mozfun.bits28.range(active_users.days_active_bits, -26, 27)) > 0 ) AS repeat_profile, - attribution.adjust_ad_group, - attribution.adjust_campaign, - attribution.adjust_creative, - attribution.adjust_network, active_users.days_seen_bits, active_users.days_active_bits, CASE diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/engagement_v1/query.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/engagement_v1/query.sql --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/engagement_v1/query.sql 2024-06-03 14:00:50.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/engagement_v1/query.sql 2024-06-03 14:02:29.000000000 +0000 @@ -8,22 +8,26 @@ country, locale, is_mobile, - adjust_ad_group, - adjust_campaign, - adjust_creative, - adjust_network, play_store_attribution_campaign, play_store_attribution_medium, play_store_attribution_source, meta_attribution_app, install_source, + adjust_ad_group, + adjust_campaign, + adjust_creative, + adjust_network, COUNTIF(is_dau) AS dau, COUNTIF(is_wau) AS wau, COUNTIF(is_mau) AS mau FROM `moz-fx-data-shared-prod.fenix.engagement_clients` WHERE + {% if is_init() %} + submission_date < CURRENT_DATE + {% else %} submission_date = @submission_date + {% endif %} GROUP BY submission_date, first_seen_date, @@ -33,12 +37,12 @@ country, locale, is_mobile, - adjust_ad_group, - adjust_campaign, - adjust_creative, - adjust_network, play_store_attribution_campaign, play_store_attribution_medium, play_store_attribution_source, meta_attribution_app, - install_source + install_source, + adjust_ad_group, + adjust_campaign, + adjust_creative, + adjust_network diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/engagement_v1/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/engagement_v1/schema.yaml --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/engagement_v1/schema.yaml 2024-06-03 14:00:50.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/engagement_v1/schema.yaml 2024-06-03 14:02:29.000000000 +0000 @@ -39,50 +39,50 @@ mode: NULLABLE description: Indicates if this specific entry is used towards calculating mobile DAU. -- name: adjust_ad_group +- name: play_store_attribution_campaign type: STRING mode: NULLABLE - description: Adjust Ad Group the profile is attributed to. + description: Play store campaign the profile is attributed to. -- name: adjust_campaign +- name: play_store_attribution_medium type: STRING mode: NULLABLE - description: Adjust Campaign the profile is attributed to. + description: Play store Medium the profile is attributed to. -- name: adjust_creative +- name: play_store_attribution_source type: STRING mode: NULLABLE - description: Adjust Creative the profile is attributed to. + description: Play store source the profile is attributed to. -- name: adjust_network +- name: meta_attribution_app type: STRING mode: NULLABLE - description: Adjust Network the profile is attributed to. + description: Facebook app linked to paid marketing. -- name: play_store_attribution_campaign +- name: install_source type: STRING mode: NULLABLE - description: Play store campaign the profile is attributed to. + description: The source of a profile installation. -- name: play_store_attribution_medium +- name: adjust_ad_group type: STRING mode: NULLABLE - description: Play store Medium the profile is attributed to. + description: Adjust Ad Group the profile is attributed to. -- name: play_store_attribution_source +- name: adjust_campaign type: STRING mode: NULLABLE - description: Play store source the profile is attributed to. + description: Adjust Campaign the profile is attributed to. -- name: meta_attribution_app +- name: adjust_creative type: STRING mode: NULLABLE - description: Facebook app linked to paid marketing. + description: Adjust Creative the profile is attributed to. -- name: install_source +- name: adjust_network type: STRING mode: NULLABLE - description: The source of a profile installation. + description: Adjust Network the profile is attributed to. - name: dau type: INTEGER diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/retention_v1/query.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/retention_v1/query.sql --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/retention_v1/query.sql 2024-06-03 14:00:50.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/retention_v1/query.sql 2024-06-03 14:02:29.000000000 +0000 @@ -8,15 +8,15 @@ app_version, locale, is_mobile, - adjust_ad_group, - adjust_campaign, - adjust_creative, - adjust_network, play_store_attribution_campaign, play_store_attribution_medium, play_store_attribution_source, meta_attribution_app, install_source, + adjust_ad_group, + adjust_campaign, + adjust_creative, + adjust_network, COUNTIF(ping_sent_metric_date) AS ping_sent_metric_date, COUNTIF(ping_sent_week_4) AS ping_sent_week_4, COUNTIF(active_metric_date) AS active_metric_date, @@ -27,8 +27,13 @@ FROM `moz-fx-data-shared-prod.fenix.retention_clients` WHERE + {% if is_init() %} + metric_date < DATE_SUB(CURRENT_DATE, INTERVAL 27 DAY) + AND submission_date < CURRENT_DATE + {% else %} metric_date = DATE_SUB(@submission_date, INTERVAL 27 DAY) AND submission_date = @submission_date + {% endif %} GROUP BY metric_date, first_seen_date, @@ -38,12 +43,12 @@ app_version, locale, is_mobile, - adjust_ad_group, - adjust_campaign, - adjust_creative, - adjust_network, play_store_attribution_campaign, play_store_attribution_medium, play_store_attribution_source, meta_attribution_app, - install_source + install_source, + adjust_ad_group, + adjust_campaign, + adjust_creative, + adjust_network diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/retention_v1/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/retention_v1/schema.yaml --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/retention_v1/schema.yaml 2024-06-03 14:00:50.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/retention_v1/schema.yaml 2024-06-03 14:02:29.000000000 +0000 @@ -1,5 +1,4 @@ fields: - - mode: NULLABLE name: metric_date type: DATE @@ -40,50 +39,50 @@ mode: NULLABLE description: Indicates if this specific entry is used towards calculating mobile DAU. -- name: adjust_ad_group +- name: play_store_attribution_campaign type: STRING mode: NULLABLE - description: Adjust Ad Group the profile is attributed to. + description: Play store campaign the profile is attributed to. -- name: adjust_campaign +- name: play_store_attribution_medium type: STRING mode: NULLABLE - description: Adjust Campaign the profile is attributed to. + description: Play store Medium the profile is attributed to. -- name: adjust_creative +- name: play_store_attribution_source type: STRING mode: NULLABLE - description: Adjust Creative the profile is attributed to. + description: Play store source the profile is attributed to. -- name: adjust_network +- name: meta_attribution_app type: STRING mode: NULLABLE - description: Adjust Network the profile is attributed to. + description: Facebook app linked to paid marketing. -- name: play_store_attribution_campaign +- name: install_source type: STRING mode: NULLABLE - description: Play store campaign the profile is attributed to. + description: The source of a profile installation. -- name: play_store_attribution_medium +- name: adjust_ad_group type: STRING mode: NULLABLE - description: Play store Medium the profile is attributed to. + description: Adjust Ad Group the profile is attributed to. -- name: play_store_attribution_source +- name: adjust_campaign type: STRING mode: NULLABLE - description: Play store source the profile is attributed to. + description: Adjust Campaign the profile is attributed to. -- name: meta_attribution_app +- name: adjust_creative type: STRING mode: NULLABLE - description: Facebook app linked to paid marketing. + description: Adjust Creative the profile is attributed to. -- name: install_source +- name: adjust_network type: STRING mode: NULLABLE - description: The source of a profile installation. + description: Adjust Network the profile is attributed to. - name: ping_sent_metric_date type: INTEGER diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop/metrics/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop/metrics/schema.yaml --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop/metrics/schema.yaml 2024-06-03 14:00:51.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop/metrics/schema.yaml 2024-06-03 14:01:37.000000000 +0000 @@ -2538,7 +2538,7 @@ mode: NULLABLE description: 'Counts how often we purge trackers. Giving a high level overview about the effectivness of bounce tracking protection. Allows determining error - rate with failed/success label. When in dry mode, we count the purges that + rate with failure/success label. When in dry mode, we count the purges that would have happened under the dry label. ' diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop/use_counters/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop/use_counters/schema.yaml --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop/use_counters/schema.yaml 2024-06-03 14:00:51.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop/use_counters/schema.yaml 1970-01-01 00:00:00.000000000 +0000 @@ -1,18056 +0,0 @@ -fields: -- name: additional_properties - type: STRING - mode: NULLABLE - description: A JSON string containing any payload properties not present in the - schema -- name: client_info - type: RECORD - mode: NULLABLE - fields: - - name: android_sdk_version - type: STRING - mode: NULLABLE - description: The optional Android specific SDK version of the software running - on this hardware device. - - name: app_build - type: STRING - mode: NULLABLE - description: The build identifier generated by the CI system (e.g. "1234/A"). - For language bindings that provide automatic detection for this value, (e.g. - Android/Kotlin), in the unlikely event that the build identifier can not be - retrieved from the OS, it is set to "inaccessible". For other language bindings, - if the value was not provided through configuration, this metric gets set to - `Unknown`. - - name: app_channel - type: STRING - mode: NULLABLE - description: The channel the application is being distributed on. - - name: app_display_version - type: STRING - mode: NULLABLE - description: The user visible version string (e.g. "1.0.3"). In the unlikely - event that the display version can not be retrieved, it is set to "inaccessible". - - name: architecture - type: STRING - mode: NULLABLE - description: The architecture of the device, (e.g. "arm", "x86"). - - name: build_date - type: STRING - mode: NULLABLE - description: The date & time the application was built - - name: client_id - type: STRING - mode: NULLABLE - description: A UUID uniquely identifying the client. - - name: device_manufacturer - type: STRING - mode: NULLABLE - description: The manufacturer of the device the application is running on. Not - set if the device manufacturer can't be determined (e.g. on Desktop). - - name: device_model - type: STRING - mode: NULLABLE - description: The model of the device the application is running on. On Android, - this is Build.MODEL, the user-visible marketing name, like "Pixel 2 XL". Not - set if the device model can't be determined (e.g. on Desktop). - - name: first_run_date - type: STRING - mode: NULLABLE - description: The date of the first run of the application. - - name: locale - type: STRING - mode: NULLABLE - description: The locale of the application during initialization (e.g. "es-ES"). - If the locale can't be determined on the system, the value is ["und"](https://unicode.org/reports/tr35/#Unknown_or_Invalid_Identifiers), - to indicate "undetermined". - - name: os - type: STRING - mode: NULLABLE - description: 'The name of the operating system. Possible values: Android, iOS, - Linux, Darwin, Windows, FreeBSD, NetBSD, OpenBSD, Solaris, unknown' - - name: os_version - type: STRING - mode: NULLABLE - description: The user-visible version of the operating system (e.g. "1.2.3"). - If the version detection fails, this metric gets set to `Unknown`. - - name: telemetry_sdk_build - type: STRING - mode: NULLABLE - description: The version of the Glean SDK - - name: windows_build_number - type: INTEGER - mode: NULLABLE - description: The optional Windows build number, reported by Windows (e.g. 22000) - and not set for other platforms - - name: session_count - type: INTEGER - mode: NULLABLE - description: An optional running counter of the number of sessions for a client. - - name: session_id - type: STRING - mode: NULLABLE - description: An optional UUID uniquely identifying the client's current session. -- name: document_id - type: STRING - mode: NULLABLE - description: The document ID specified in the URI when the client sent this message -- name: events - type: RECORD - mode: REPEATED - fields: - - name: category - type: STRING - mode: NULLABLE - - name: extra - type: RECORD - mode: REPEATED - fields: - - name: key - type: STRING - mode: NULLABLE - - name: value - type: STRING - mode: NULLABLE - - name: name - type: STRING - mode: NULLABLE - - name: timestamp - type: INTEGER - mode: NULLABLE -- name: metadata - type: RECORD - mode: NULLABLE - fields: - - name: geo - type: RECORD - mode: NULLABLE - fields: - - name: city - type: STRING - mode: NULLABLE - - name: country - type: STRING - mode: NULLABLE - description: An ISO 3166-1 alpha-2 country code - - name: db_version - type: STRING - mode: NULLABLE - description: The specific geo database version used for this lookup - - name: subdivision1 - type: STRING - mode: NULLABLE - description: First major country subdivision, typically a state, province, or - county - - name: subdivision2 - type: STRING - mode: NULLABLE - description: Second major country subdivision; not applicable for most countries - description: Results of a geographic lookup based on the client's IP address - - name: header - type: RECORD - mode: NULLABLE - fields: - - name: date - type: STRING - mode: NULLABLE - description: Date HTTP header - - name: dnt - type: STRING - mode: NULLABLE - description: DNT (Do Not Track) HTTP header - - name: x_debug_id - type: STRING - mode: NULLABLE - description: X-Debug-Id HTTP header - - name: x_foxsec_ip_reputation - type: STRING - mode: NULLABLE - description: X-Foxsec-IP-Reputation header - - name: x_lb_tags - type: STRING - mode: NULLABLE - description: X-LB-Tags HTTP header - - name: x_pingsender_version - type: STRING - mode: NULLABLE - description: X-PingSender-Version HTTP header - - name: x_source_tags - type: STRING - mode: NULLABLE - description: X-Source-Tags HTTP header - - name: x_telemetry_agent - type: STRING - mode: NULLABLE - description: X-Telemetry-Agent HTTP header - - name: parsed_date - type: TIMESTAMP - mode: NULLABLE - - name: parsed_x_source_tags - type: STRING - mode: REPEATED - - name: parsed_x_lb_tags - type: RECORD - mode: NULLABLE - fields: - - name: tls_version - type: STRING - mode: NULLABLE - - name: tls_cipher_hex - type: STRING - mode: NULLABLE - description: Headers included in the client's HTTP request - - name: isp - type: RECORD - mode: NULLABLE - fields: - - name: db_version - type: STRING - mode: NULLABLE - description: The specific geo ISP database version used for this lookup - - name: name - type: STRING - mode: NULLABLE - description: The name of the ISP associated with the client's IP address - - name: organization - type: STRING - mode: NULLABLE - description: The name of a specific business entity associated with the client's - IP address when available; otherwise the ISP name - description: Results of ISP lookup based on the client's IP address - - name: user_agent - type: RECORD - mode: NULLABLE - fields: - - name: browser - type: STRING - mode: NULLABLE - - name: os - type: STRING - mode: NULLABLE - - name: version - type: STRING - mode: NULLABLE - description: Parsed components of the client's user agent string -- name: metrics - type: RECORD - mode: NULLABLE - fields: - - name: counter - type: RECORD - mode: NULLABLE - fields: - - name: use_counter_content_documents_destroyed - type: INTEGER - mode: NULLABLE - description: 'A count of how many content documents were destroyed. Used to - turn document use counters'' counts into rates. Excludes documents for which - we do not count use counters (See `Document::ShouldIncludeInTelemetry`). - - ' - - name: use_counter_css_doc_alignment_baseline - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the (unknown, counted) CSS property alignment-baseline. - Compare against `use.counter.content_documents_destroyed` to calculate the - rate. - - ' - - name: use_counter_css_doc_background_repeat_x - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the (unknown, counted) CSS property background-repeat-x. - Compare against `use.counter.content_documents_destroyed` to calculate the - rate. - - ' - - name: use_counter_css_doc_background_repeat_y - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the (unknown, counted) CSS property background-repeat-y. - Compare against `use.counter.content_documents_destroyed` to calculate the - rate. - - ' - - name: use_counter_css_doc_baseline_shift - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the (unknown, counted) CSS property baseline-shift. - Compare against `use.counter.content_documents_destroyed` to calculate the - rate. - - ' - - name: use_counter_css_doc_buffered_rendering - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the (unknown, counted) CSS property buffered-rendering. - Compare against `use.counter.content_documents_destroyed` to calculate the - rate. - - ' - - name: use_counter_css_doc_color_rendering - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the (unknown, counted) CSS property color-rendering. - Compare against `use.counter.content_documents_destroyed` to calculate the - rate. - - ' - - name: use_counter_css_doc_css_accent_color - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property accent-color. Compare - against `use.counter.content_documents_destroyed` to calculate the rate. - - ' - - name: use_counter_css_doc_css_align_content - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property align-content. Compare - against `use.counter.content_documents_destroyed` to calculate the rate. - - ' - - name: use_counter_css_doc_css_align_items - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property align-items. Compare - against `use.counter.content_documents_destroyed` to calculate the rate. - - ' - - name: use_counter_css_doc_css_align_self - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property align-self. Compare against - `use.counter.content_documents_destroyed` to calculate the rate. - - ' - - name: use_counter_css_doc_css_align_tracks - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property align-tracks. Compare - against `use.counter.content_documents_destroyed` to calculate the rate. - - ' - - name: use_counter_css_doc_css_all - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property all. Compare against - `use.counter.content_documents_destroyed` to calculate the rate. - - ' - - name: use_counter_css_doc_css_animation - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property animation. Compare against - `use.counter.content_documents_destroyed` to calculate the rate. - - ' - - name: use_counter_css_doc_css_animation_composition - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property animation-composition. - Compare against `use.counter.content_documents_destroyed` to calculate the - rate. - - ' - - name: use_counter_css_doc_css_animation_delay - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property animation-delay. Compare - against `use.counter.content_documents_destroyed` to calculate the rate. - - ' - - name: use_counter_css_doc_css_animation_direction - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property animation-direction. - Compare against `use.counter.content_documents_destroyed` to calculate the - rate. - - ' - - name: use_counter_css_doc_css_animation_duration - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property animation-duration. Compare - against `use.counter.content_documents_destroyed` to calculate the rate. - - ' - - name: use_counter_css_doc_css_animation_fill_mode - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property animation-fill-mode. - Compare against `use.counter.content_documents_destroyed` to calculate the - rate. - - ' - - name: use_counter_css_doc_css_animation_iteration_count - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property animation-iteration-count. - Compare against `use.counter.content_documents_destroyed` to calculate the - rate. - - ' - - name: use_counter_css_doc_css_animation_name - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property animation-name. Compare - against `use.counter.content_documents_destroyed` to calculate the rate. - - ' - - name: use_counter_css_doc_css_animation_play_state - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property animation-play-state. - Compare against `use.counter.content_documents_destroyed` to calculate the - rate. - - ' - - name: use_counter_css_doc_css_animation_timeline - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property animation-timeline. Compare - against `use.counter.content_documents_destroyed` to calculate the rate. - - ' - - name: use_counter_css_doc_css_animation_timing_function - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property animation-timing-function. - Compare against `use.counter.content_documents_destroyed` to calculate the - rate. - - ' - - name: use_counter_css_doc_css_appearance - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property appearance. Compare against - `use.counter.content_documents_destroyed` to calculate the rate. - - ' - - name: use_counter_css_doc_css_aspect_ratio - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property aspect-ratio. Compare - against `use.counter.content_documents_destroyed` to calculate the rate. - - ' - - name: use_counter_css_doc_css_backdrop_filter - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property backdrop-filter. Compare - against `use.counter.content_documents_destroyed` to calculate the rate. - - ' - - name: use_counter_css_doc_css_backface_visibility - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property backface-visibility. - Compare against `use.counter.content_documents_destroyed` to calculate the - rate. - - ' - - name: use_counter_css_doc_css_background - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property background. Compare against - `use.counter.content_documents_destroyed` to calculate the rate. - - ' - - name: use_counter_css_doc_css_background_attachment - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property background-attachment. - Compare against `use.counter.content_documents_destroyed` to calculate the - rate. - - ' - - name: use_counter_css_doc_css_background_blend_mode - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property background-blend-mode. - Compare against `use.counter.content_documents_destroyed` to calculate the - rate. - - ' - - name: use_counter_css_doc_css_background_clip - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property background-clip. Compare - against `use.counter.content_documents_destroyed` to calculate the rate. - - ' - - name: use_counter_css_doc_css_background_color - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property background-color. Compare - against `use.counter.content_documents_destroyed` to calculate the rate. - - ' - - name: use_counter_css_doc_css_background_image - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property background-image. Compare - against `use.counter.content_documents_destroyed` to calculate the rate. - - ' - - name: use_counter_css_doc_css_background_origin - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property background-origin. Compare - against `use.counter.content_documents_destroyed` to calculate the rate. - - ' - - name: use_counter_css_doc_css_background_position - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property background-position. - Compare against `use.counter.content_documents_destroyed` to calculate the - rate. - - ' - - name: use_counter_css_doc_css_background_position_x - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property background-position-x. - Compare against `use.counter.content_documents_destroyed` to calculate the - rate. - - ' - - name: use_counter_css_doc_css_background_position_y - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property background-position-y. - Compare against `use.counter.content_documents_destroyed` to calculate the - rate. - - ' - - name: use_counter_css_doc_css_background_repeat - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property background-repeat. Compare - against `use.counter.content_documents_destroyed` to calculate the rate. - - ' - - name: use_counter_css_doc_css_background_size - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property background-size. Compare - against `use.counter.content_documents_destroyed` to calculate the rate. - - ' - - name: use_counter_css_doc_css_baseline_source - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property baseline-source. Compare - against `use.counter.content_documents_destroyed` to calculate the rate. - - ' - - name: use_counter_css_doc_css_block_size - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property block-size. Compare against - `use.counter.content_documents_destroyed` to calculate the rate. - - ' - - name: use_counter_css_doc_css_border - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border. Compare against - `use.counter.content_documents_destroyed` to calculate the rate. - - ' - - name: use_counter_css_doc_css_border_block - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-block. Compare - against `use.counter.content_documents_destroyed` to calculate the rate. - - ' - - name: use_counter_css_doc_css_border_block_color - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-block-color. Compare - against `use.counter.content_documents_destroyed` to calculate the rate. - - ' - - name: use_counter_css_doc_css_border_block_end - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-block-end. Compare - against `use.counter.content_documents_destroyed` to calculate the rate. - - ' - - name: use_counter_css_doc_css_border_block_end_color - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-block-end-color. - Compare against `use.counter.content_documents_destroyed` to calculate the - rate. - - ' - - name: use_counter_css_doc_css_border_block_end_style - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-block-end-style. - Compare against `use.counter.content_documents_destroyed` to calculate the - rate. - - ' - - name: use_counter_css_doc_css_border_block_end_width - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-block-end-width. - Compare against `use.counter.content_documents_destroyed` to calculate the - rate. - - ' - - name: use_counter_css_doc_css_border_block_start - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-block-start. Compare - against `use.counter.content_documents_destroyed` to calculate the rate. - - ' - - name: use_counter_css_doc_css_border_block_start_color - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-block-start-color. - Compare against `use.counter.content_documents_destroyed` to calculate the - rate. - - ' - - name: use_counter_css_doc_css_border_block_start_style - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-block-start-style. - Compare against `use.counter.content_documents_destroyed` to calculate the - rate. - - ' - - name: use_counter_css_doc_css_border_block_start_width - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-block-start-width. - Compare against `use.counter.content_documents_destroyed` to calculate the - rate. - - ' - - name: use_counter_css_doc_css_border_block_style - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-block-style. Compare - against `use.counter.content_documents_destroyed` to calculate the rate. - - ' - - name: use_counter_css_doc_css_border_block_width - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-block-width. Compare - against `use.counter.content_documents_destroyed` to calculate the rate. - - ' - - name: use_counter_css_doc_css_border_bottom - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-bottom. Compare - against `use.counter.content_documents_destroyed` to calculate the rate. - - ' - - name: use_counter_css_doc_css_border_bottom_color - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-bottom-color. - Compare against `use.counter.content_documents_destroyed` to calculate the - rate. - - ' - - name: use_counter_css_doc_css_border_bottom_left_radius - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-bottom-left-radius. - Compare against `use.counter.content_documents_destroyed` to calculate the - rate. - - ' - - name: use_counter_css_doc_css_border_bottom_right_radius - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-bottom-right-radius. - Compare against `use.counter.content_documents_destroyed` to calculate the - rate. - - ' - - name: use_counter_css_doc_css_border_bottom_style - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-bottom-style. - Compare against `use.counter.content_documents_destroyed` to calculate the - rate. - - ' - - name: use_counter_css_doc_css_border_bottom_width - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-bottom-width. - Compare against `use.counter.content_documents_destroyed` to calculate the - rate. - - ' - - name: use_counter_css_doc_css_border_collapse - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-collapse. Compare - against `use.counter.content_documents_destroyed` to calculate the rate. - - ' - - name: use_counter_css_doc_css_border_color - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-color. Compare - against `use.counter.content_documents_destroyed` to calculate the rate. - - ' - - name: use_counter_css_doc_css_border_end_end_radius - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-end-end-radius. - Compare against `use.counter.content_documents_destroyed` to calculate the - rate. - - ' - - name: use_counter_css_doc_css_border_end_start_radius - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-end-start-radius. - Compare against `use.counter.content_documents_destroyed` to calculate the - rate. - - ' - - name: use_counter_css_doc_css_border_image - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-image. Compare - against `use.counter.content_documents_destroyed` to calculate the rate. - - ' - - name: use_counter_css_doc_css_border_image_outset - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-image-outset. - Compare against `use.counter.content_documents_destroyed` to calculate the - rate. - - ' - - name: use_counter_css_doc_css_border_image_repeat - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-image-repeat. - Compare against `use.counter.content_documents_destroyed` to calculate the - rate. - - ' - - name: use_counter_css_doc_css_border_image_slice - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-image-slice. Compare - against `use.counter.content_documents_destroyed` to calculate the rate. - - ' - - name: use_counter_css_doc_css_border_image_source - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-image-source. - Compare against `use.counter.content_documents_destroyed` to calculate the - rate. - - ' - - name: use_counter_css_doc_css_border_image_width - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-image-width. Compare - against `use.counter.content_documents_destroyed` to calculate the rate. - - ' - - name: use_counter_css_doc_css_border_inline - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-inline. Compare - against `use.counter.content_documents_destroyed` to calculate the rate. - - ' - - name: use_counter_css_doc_css_border_inline_color - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-inline-color. - Compare against `use.counter.content_documents_destroyed` to calculate the - rate. - - ' - - name: use_counter_css_doc_css_border_inline_end - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-inline-end. Compare - against `use.counter.content_documents_destroyed` to calculate the rate. - - ' - - name: use_counter_css_doc_css_border_inline_end_color - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-inline-end-color. - Compare against `use.counter.content_documents_destroyed` to calculate the - rate. - - ' - - name: use_counter_css_doc_css_border_inline_end_style - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-inline-end-style. - Compare against `use.counter.content_documents_destroyed` to calculate the - rate. - - ' - - name: use_counter_css_doc_css_border_inline_end_width - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-inline-end-width. - Compare against `use.counter.content_documents_destroyed` to calculate the - rate. - - ' - - name: use_counter_css_doc_css_border_inline_start - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-inline-start. - Compare against `use.counter.content_documents_destroyed` to calculate the - rate. - - ' - - name: use_counter_css_doc_css_border_inline_start_color - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-inline-start-color. - Compare against `use.counter.content_documents_destroyed` to calculate the - rate. - - ' - - name: use_counter_css_doc_css_border_inline_start_style - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-inline-start-style. - Compare against `use.counter.content_documents_destroyed` to calculate the - rate. - - ' - - name: use_counter_css_doc_css_border_inline_start_width - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-inline-start-width. - Compare against `use.counter.content_documents_destroyed` to calculate the - rate. - - ' - - name: use_counter_css_doc_css_border_inline_style - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-inline-style. - Compare against `use.counter.content_documents_destroyed` to calculate the - rate. - - ' - - name: use_counter_css_doc_css_border_inline_width - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-inline-width. - Compare against `use.counter.content_documents_destroyed` to calculate the - rate. - - ' - - name: use_counter_css_doc_css_border_left - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-left. Compare - against `use.counter.content_documents_destroyed` to calculate the rate. - - ' - - name: use_counter_css_doc_css_border_left_color - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-left-color. Compare - against `use.counter.content_documents_destroyed` to calculate the rate. - - ' - - name: use_counter_css_doc_css_border_left_style - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-left-style. Compare - against `use.counter.content_documents_destroyed` to calculate the rate. - - ' - - name: use_counter_css_doc_css_border_left_width - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-left-width. Compare - against `use.counter.content_documents_destroyed` to calculate the rate. - - ' - - name: use_counter_css_doc_css_border_radius - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-radius. Compare - against `use.counter.content_documents_destroyed` to calculate the rate. - - ' - - name: use_counter_css_doc_css_border_right - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-right. Compare - against `use.counter.content_documents_destroyed` to calculate the rate. - - ' - - name: use_counter_css_doc_css_border_right_color - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-right-color. Compare - against `use.counter.content_documents_destroyed` to calculate the rate. - - ' - - name: use_counter_css_doc_css_border_right_style - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-right-style. Compare - against `use.counter.content_documents_destroyed` to calculate the rate. - - ' - - name: use_counter_css_doc_css_border_right_width - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-right-width. Compare - against `use.counter.content_documents_destroyed` to calculate the rate. - - ' - - name: use_counter_css_doc_css_border_spacing - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-spacing. Compare - against `use.counter.content_documents_destroyed` to calculate the rate. - - ' - - name: use_counter_css_doc_css_border_start_end_radius - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-start-end-radius. - Compare against `use.counter.content_documents_destroyed` to calculate the - rate. - - ' - - name: use_counter_css_doc_css_border_start_start_radius - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-start-start-radius. - Compare against `use.counter.content_documents_destroyed` to calculate the - rate. - - ' - - name: use_counter_css_doc_css_border_style - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-style. Compare - against `use.counter.content_documents_destroyed` to calculate the rate. - - ' - - name: use_counter_css_doc_css_border_top - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-top. Compare against - `use.counter.content_documents_destroyed` to calculate the rate. - - ' - - name: use_counter_css_doc_css_border_top_color - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-top-color. Compare - against `use.counter.content_documents_destroyed` to calculate the rate. - - ' - - name: use_counter_css_doc_css_border_top_left_radius - type: INTEGER - mode: NULLABLE - description: 'Whether a document used the CSS property border-top-left-radius. - Compare against ` ```

⚠️ Only part of the diff is displayed.

Link to full diff

dataops-ci-bot commented 1 month ago

Integration report for "fix: trailing comma in queries"

sql.diff

Click to expand! ```diff diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/dags/bqetl_analytics_aggregations.py /tmp/workspace/generated-sql/dags/bqetl_analytics_aggregations.py --- /tmp/workspace/main-generated-sql/dags/bqetl_analytics_aggregations.py 2024-06-03 15:29:02.000000000 +0000 +++ /tmp/workspace/generated-sql/dags/bqetl_analytics_aggregations.py 2024-06-03 15:41:12.000000000 +0000 @@ -190,6 +190,18 @@ pool="DATA_ENG_EXTERNALTASKSENSOR", ) + wait_for_search_derived__mobile_search_clients_daily__v1 = ExternalTaskSensor( + task_id="wait_for_search_derived__mobile_search_clients_daily__v1", + external_dag_id="bqetl_mobile_search", + external_task_id="search_derived__mobile_search_clients_daily__v1", + execution_delta=datetime.timedelta(seconds=8100), + check_existence=True, + mode="reschedule", + allowed_states=ALLOWED_STATES, + failed_states=FAILED_STATES, + pool="DATA_ENG_EXTERNALTASKSENSOR", + ) + wait_for_firefox_ios_derived__clients_last_seen_joined__v1 = ExternalTaskSensor( task_id="wait_for_firefox_ios_derived__clients_last_seen_joined__v1", external_dag_id="bqetl_glean_usage", @@ -848,6 +860,10 @@ wait_for_fenix_derived__clients_last_seen_joined__v1 ) + fenix_active_users_aggregates.set_upstream( + wait_for_search_derived__mobile_search_clients_daily__v1 + ) + firefox_desktop_active_users_aggregates.set_upstream( wait_for_checks__fail_telemetry_derived__clients_last_seen__v2 ) @@ -868,11 +884,19 @@ wait_for_firefox_ios_derived__clients_last_seen_joined__v1 ) + firefox_ios_active_users_aggregates.set_upstream( + wait_for_search_derived__mobile_search_clients_daily__v1 + ) + focus_android_active_users_aggregates.set_upstream( wait_for_focus_android_derived__clients_last_seen_joined__v1 ) focus_android_active_users_aggregates.set_upstream( + wait_for_search_derived__mobile_search_clients_daily__v1 + ) + + focus_android_active_users_aggregates.set_upstream( wait_for_telemetry_derived__core_clients_last_seen__v1 ) @@ -892,6 +916,10 @@ wait_for_focus_ios_derived__clients_last_seen_joined__v1 ) + focus_ios_active_users_aggregates.set_upstream( + wait_for_search_derived__mobile_search_clients_daily__v1 + ) + klar_ios_active_users_aggregates.set_upstream( wait_for_checks__fail_fenix_derived__firefox_android_clients__v1 ) @@ -908,6 +936,10 @@ wait_for_klar_ios_derived__clients_last_seen_joined__v1 ) + klar_ios_active_users_aggregates.set_upstream( + wait_for_search_derived__mobile_search_clients_daily__v1 + ) + telemetry_derived__cohort_daily_statistics__v1.set_upstream( wait_for_checks__fail_telemetry_derived__unified_metrics__v1 ) diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/dags/bqetl_desktop_mobile_search_monthly.py /tmp/workspace/generated-sql/dags/bqetl_desktop_mobile_search_monthly.py --- /tmp/workspace/main-generated-sql/dags/bqetl_desktop_mobile_search_monthly.py 2024-06-03 15:29:02.000000000 +0000 +++ /tmp/workspace/generated-sql/dags/bqetl_desktop_mobile_search_monthly.py 2024-06-03 15:41:12.000000000 +0000 @@ -50,6 +50,18 @@ tags=tags, ) as dag: + wait_for_search_derived__mobile_search_clients_daily__v1 = ExternalTaskSensor( + task_id="wait_for_search_derived__mobile_search_clients_daily__v1", + external_dag_id="bqetl_mobile_search", + external_task_id="search_derived__mobile_search_clients_daily__v1", + execution_delta=datetime.timedelta(days=2, seconds=10800), + check_existence=True, + mode="reschedule", + allowed_states=ALLOWED_STATES, + failed_states=FAILED_STATES, + pool="DATA_ENG_EXTERNALTASKSENSOR", + ) + wait_for_search_derived__search_clients_daily__v8 = ExternalTaskSensor( task_id="wait_for_search_derived__search_clients_daily__v8", external_dag_id="bqetl_search", @@ -75,5 +87,9 @@ ) search_derived__desktop_mobile_search_clients_monthly__v1.set_upstream( + wait_for_search_derived__mobile_search_clients_daily__v1 + ) + + search_derived__desktop_mobile_search_clients_monthly__v1.set_upstream( wait_for_search_derived__search_clients_daily__v8 ) diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/dags/bqetl_firefox_ios.py /tmp/workspace/generated-sql/dags/bqetl_firefox_ios.py --- /tmp/workspace/main-generated-sql/dags/bqetl_firefox_ios.py 2024-06-03 15:29:02.000000000 +0000 +++ /tmp/workspace/generated-sql/dags/bqetl_firefox_ios.py 2024-06-03 15:41:11.000000000 +0000 @@ -86,6 +86,18 @@ pool="DATA_ENG_EXTERNALTASKSENSOR", ) + wait_for_search_derived__mobile_search_clients_daily__v1 = ExternalTaskSensor( + task_id="wait_for_search_derived__mobile_search_clients_daily__v1", + external_dag_id="bqetl_mobile_search", + external_task_id="search_derived__mobile_search_clients_daily__v1", + execution_delta=datetime.timedelta(seconds=7200), + check_existence=True, + mode="reschedule", + allowed_states=ALLOWED_STATES, + failed_states=FAILED_STATES, + pool="DATA_ENG_EXTERNALTASKSENSOR", + ) + wait_for_org_mozilla_ios_fennec_derived__baseline_clients_daily__v1 = ExternalTaskSensor( task_id="wait_for_org_mozilla_ios_fennec_derived__baseline_clients_daily__v1", external_dag_id="bqetl_glean_usage", @@ -693,6 +705,10 @@ wait_for_copy_deduplicate_all ) + firefox_ios_derived__attributable_clients__v1.set_upstream( + wait_for_search_derived__mobile_search_clients_daily__v1 + ) + firefox_ios_derived__baseline_clients_yearly__v1.set_upstream( wait_for_org_mozilla_ios_fennec_derived__baseline_clients_daily__v1 ) @@ -725,6 +741,10 @@ wait_for_checks__fail_org_mozilla_ios_firefoxbeta_derived__baseline_clients_last_seen__v1 ) + firefox_ios_derived__clients_activation__v1.set_upstream( + wait_for_search_derived__mobile_search_clients_daily__v1 + ) + firefox_ios_derived__firefox_ios_clients__v1.set_upstream( wait_for_copy_deduplicate_all ) @@ -796,3 +816,7 @@ firefox_ios_derived__new_profile_activation__v2.set_upstream( wait_for_checks__fail_org_mozilla_ios_firefoxbeta_derived__baseline_clients_last_seen__v1 ) + + firefox_ios_derived__new_profile_activation__v2.set_upstream( + wait_for_search_derived__mobile_search_clients_daily__v1 + ) diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/dags/bqetl_mobile_activation.py /tmp/workspace/generated-sql/dags/bqetl_mobile_activation.py --- /tmp/workspace/main-generated-sql/dags/bqetl_mobile_activation.py 2024-06-03 15:29:02.000000000 +0000 +++ /tmp/workspace/generated-sql/dags/bqetl_mobile_activation.py 2024-06-03 15:41:12.000000000 +0000 @@ -182,6 +182,18 @@ pool="DATA_ENG_EXTERNALTASKSENSOR", ) + wait_for_search_derived__mobile_search_clients_daily__v1 = ExternalTaskSensor( + task_id="wait_for_search_derived__mobile_search_clients_daily__v1", + external_dag_id="bqetl_mobile_search", + external_task_id="search_derived__mobile_search_clients_daily__v1", + execution_delta=datetime.timedelta(days=-1, seconds=79200), + check_existence=True, + mode="reschedule", + allowed_states=ALLOWED_STATES, + failed_states=FAILED_STATES, + pool="DATA_ENG_EXTERNALTASKSENSOR", + ) + wait_for_checks__fail_org_mozilla_ios_fennec_derived__baseline_clients_last_seen__v1 = ExternalTaskSensor( task_id="wait_for_checks__fail_org_mozilla_ios_fennec_derived__baseline_clients_last_seen__v1", external_dag_id="bqetl_glean_usage", @@ -341,6 +353,10 @@ wait_for_org_mozilla_firefox_derived__baseline_clients_daily__v1 ) + fenix_derived__new_profile_activation__v1.set_upstream( + wait_for_search_derived__mobile_search_clients_daily__v1 + ) + firefox_ios_derived__new_profile_activation__v1.set_upstream( wait_for_checks__fail_org_mozilla_ios_fennec_derived__baseline_clients_last_seen__v1 ) @@ -364,3 +380,7 @@ firefox_ios_derived__new_profile_activation__v1.set_upstream( wait_for_org_mozilla_ios_firefoxbeta_derived__baseline_clients_daily__v1 ) + + firefox_ios_derived__new_profile_activation__v1.set_upstream( + wait_for_search_derived__mobile_search_clients_daily__v1 + ) diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/dags/bqetl_mobile_search.py /tmp/workspace/generated-sql/dags/bqetl_mobile_search.py --- /tmp/workspace/main-generated-sql/dags/bqetl_mobile_search.py 2024-06-03 15:29:02.000000000 +0000 +++ /tmp/workspace/generated-sql/dags/bqetl_mobile_search.py 2024-06-03 15:41:10.000000000 +0000 @@ -52,6 +52,18 @@ tags=tags, ) as dag: + wait_for_copy_deduplicate_all = ExternalTaskSensor( + task_id="wait_for_copy_deduplicate_all", + external_dag_id="copy_deduplicate", + external_task_id="copy_deduplicate_all", + execution_delta=datetime.timedelta(seconds=3600), + check_existence=True, + mode="reschedule", + allowed_states=ALLOWED_STATES, + failed_states=FAILED_STATES, + pool="DATA_ENG_EXTERNALTASKSENSOR", + ) + search_derived__mobile_search_aggregates__v1 = bigquery_etl_query( task_id="search_derived__mobile_search_aggregates__v1", destination_table="mobile_search_aggregates_v1", @@ -68,6 +80,90 @@ depends_on_past=False, ) + search_derived__mobile_search_clients_daily__v1 = bigquery_etl_query( + task_id="search_derived__mobile_search_clients_daily__v1", + destination_table="mobile_search_clients_daily_v1", + dataset_id="search_derived", + project_id="moz-fx-data-shared-prod", + owner="akomar@mozilla.com", + email=[ + "akomar@mozilla.com", + "anicholson@mozilla.com", + "cmorales@mozilla.com", + "telemetry-alerts@mozilla.com", + ], + date_partition_parameter="submission_date", + depends_on_past=False, + ) + + with TaskGroup( + "search_derived__mobile_search_clients_daily__v1_external", + ) as search_derived__mobile_search_clients_daily__v1_external: + ExternalTaskMarker( + task_id="bqetl_analytics_aggregations__wait_for_search_derived__mobile_search_clients_daily__v1", + external_dag_id="bqetl_analytics_aggregations", + external_task_id="wait_for_search_derived__mobile_search_clients_daily__v1", + execution_date="{{ (execution_date - macros.timedelta(days=-1, seconds=78300)).isoformat() }}", + ) + + ExternalTaskMarker( + task_id="bqetl_kpis_shredder__wait_for_search_derived__mobile_search_clients_daily__v1", + external_dag_id="bqetl_kpis_shredder", + external_task_id="wait_for_search_derived__mobile_search_clients_daily__v1", + ) + + ExternalTaskMarker( + task_id="bqetl_org_mozilla_firefox_derived__wait_for_search_derived__mobile_search_clients_daily__v1", + external_dag_id="bqetl_org_mozilla_firefox_derived", + external_task_id="wait_for_search_derived__mobile_search_clients_daily__v1", + ) + + ExternalTaskMarker( + task_id="bqetl_mobile_activation__wait_for_search_derived__mobile_search_clients_daily__v1", + external_dag_id="bqetl_mobile_activation", + external_task_id="wait_for_search_derived__mobile_search_clients_daily__v1", + execution_date="{{ (execution_date - macros.timedelta(seconds=7200)).isoformat() }}", + ) + + ExternalTaskMarker( + task_id="bqetl_firefox_ios__wait_for_search_derived__mobile_search_clients_daily__v1", + external_dag_id="bqetl_firefox_ios", + external_task_id="wait_for_search_derived__mobile_search_clients_daily__v1", + execution_date="{{ (execution_date - macros.timedelta(days=-1, seconds=79200)).isoformat() }}", + ) + + ExternalTaskMarker( + task_id="bqetl_review_checker__wait_for_search_derived__mobile_search_clients_daily__v1", + external_dag_id="bqetl_review_checker", + external_task_id="wait_for_search_derived__mobile_search_clients_daily__v1", + execution_date="{{ (execution_date - macros.timedelta(seconds=7200)).isoformat() }}", + ) + + ExternalTaskMarker( + task_id="bqetl_desktop_mobile_search_monthly__wait_for_search_derived__mobile_search_clients_daily__v1", + external_dag_id="bqetl_desktop_mobile_search_monthly", + external_task_id="wait_for_search_derived__mobile_search_clients_daily__v1", + execution_date="{{ (execution_date - macros.timedelta(days=-3, seconds=75600)).isoformat() }}", + ) + + ExternalTaskMarker( + task_id="bqetl_search_dashboard__wait_for_search_derived__mobile_search_clients_daily__v1", + external_dag_id="bqetl_search_dashboard", + external_task_id="wait_for_search_derived__mobile_search_clients_daily__v1", + execution_date="{{ (execution_date - macros.timedelta(days=-1, seconds=77400)).isoformat() }}", + ) + + ExternalTaskMarker( + task_id="bqetl_unified__wait_for_search_derived__mobile_search_clients_daily__v1", + external_dag_id="bqetl_unified", + external_task_id="wait_for_search_derived__mobile_search_clients_daily__v1", + execution_date="{{ (execution_date - macros.timedelta(days=-1, seconds=82800)).isoformat() }}", + ) + + search_derived__mobile_search_clients_daily__v1_external.set_upstream( + search_derived__mobile_search_clients_daily__v1 + ) + search_derived__mobile_search_clients_last_seen__v1 = bigquery_etl_query( task_id="search_derived__mobile_search_clients_last_seen__v1", destination_table="mobile_search_clients_last_seen_v1", @@ -83,3 +179,15 @@ date_partition_parameter="submission_date", depends_on_past=True, ) + + search_derived__mobile_search_aggregates__v1.set_upstream( + search_derived__mobile_search_clients_daily__v1 + ) + + search_derived__mobile_search_clients_daily__v1.set_upstream( + wait_for_copy_deduplicate_all + ) + + search_derived__mobile_search_clients_last_seen__v1.set_upstream( + search_derived__mobile_search_clients_daily__v1 + ) diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/dags/bqetl_org_mozilla_firefox_derived.py /tmp/workspace/generated-sql/dags/bqetl_org_mozilla_firefox_derived.py --- /tmp/workspace/main-generated-sql/dags/bqetl_org_mozilla_firefox_derived.py 2024-06-03 15:29:02.000000000 +0000 +++ /tmp/workspace/generated-sql/dags/bqetl_org_mozilla_firefox_derived.py 2024-06-03 15:41:10.000000000 +0000 @@ -139,6 +139,17 @@ pool="DATA_ENG_EXTERNALTASKSENSOR", ) + wait_for_search_derived__mobile_search_clients_daily__v1 = ExternalTaskSensor( + task_id="wait_for_search_derived__mobile_search_clients_daily__v1", + external_dag_id="bqetl_mobile_search", + external_task_id="search_derived__mobile_search_clients_daily__v1", + check_existence=True, + mode="reschedule", + allowed_states=ALLOWED_STATES, + failed_states=FAILED_STATES, + pool="DATA_ENG_EXTERNALTASKSENSOR", + ) + wait_for_checks__fail_firefox_ios_derived__baseline_clients_yearly__v1 = ExternalTaskSensor( task_id="wait_for_checks__fail_firefox_ios_derived__baseline_clients_yearly__v1", external_dag_id="bqetl_firefox_ios", @@ -520,12 +531,20 @@ wait_for_org_mozilla_firefox_derived__baseline_clients_daily__v1 ) + fenix_derived__attributable_clients__v1.set_upstream( + wait_for_search_derived__mobile_search_clients_daily__v1 + ) + fenix_derived__attributable_clients__v2.set_upstream(wait_for_copy_deduplicate_all) fenix_derived__attributable_clients__v2.set_upstream( wait_for_fenix_derived__new_profile_activation__v1 ) + fenix_derived__attributable_clients__v2.set_upstream( + wait_for_search_derived__mobile_search_clients_daily__v1 + ) + fenix_derived__client_adclicks_history__v1.set_upstream( fenix_derived__attributable_clients__v2 ) diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/dags/bqetl_review_checker.py /tmp/workspace/generated-sql/dags/bqetl_review_checker.py --- /tmp/workspace/main-generated-sql/dags/bqetl_review_checker.py 2024-06-03 15:29:02.000000000 +0000 +++ /tmp/workspace/generated-sql/dags/bqetl_review_checker.py 2024-06-03 15:41:12.000000000 +0000 @@ -112,6 +112,18 @@ pool="DATA_ENG_EXTERNALTASKSENSOR", ) + wait_for_search_derived__mobile_search_clients_daily__v1 = ExternalTaskSensor( + task_id="wait_for_search_derived__mobile_search_clients_daily__v1", + external_dag_id="bqetl_mobile_search", + external_task_id="search_derived__mobile_search_clients_daily__v1", + execution_delta=datetime.timedelta(days=-1, seconds=79200), + check_existence=True, + mode="reschedule", + allowed_states=ALLOWED_STATES, + failed_states=FAILED_STATES, + pool="DATA_ENG_EXTERNALTASKSENSOR", + ) + firefox_desktop_review_checker_clients__v1 = bigquery_etl_query( task_id="firefox_desktop_review_checker_clients__v1", destination_table="review_checker_clients_v1", @@ -249,6 +261,10 @@ wait_for_copy_deduplicate_all ) + org_mozilla_fenix_review_checker_clients__v1.set_upstream( + wait_for_search_derived__mobile_search_clients_daily__v1 + ) + org_mozilla_fenix_review_checker_events__v1.set_upstream( wait_for_copy_deduplicate_all ) @@ -257,6 +273,10 @@ wait_for_copy_deduplicate_all ) + org_mozilla_ios_firefox_review_checker_clients__v1.set_upstream( + wait_for_search_derived__mobile_search_clients_daily__v1 + ) + org_mozilla_ios_firefox_review_checker_events__v1.set_upstream( wait_for_copy_deduplicate_all ) diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/dags/bqetl_search_dashboard.py /tmp/workspace/generated-sql/dags/bqetl_search_dashboard.py --- /tmp/workspace/main-generated-sql/dags/bqetl_search_dashboard.py 2024-06-03 15:29:02.000000000 +0000 +++ /tmp/workspace/generated-sql/dags/bqetl_search_dashboard.py 2024-06-03 15:41:11.000000000 +0000 @@ -71,6 +71,18 @@ pool="DATA_ENG_EXTERNALTASKSENSOR", ) + wait_for_search_derived__mobile_search_clients_daily__v1 = ExternalTaskSensor( + task_id="wait_for_search_derived__mobile_search_clients_daily__v1", + external_dag_id="bqetl_mobile_search", + external_task_id="search_derived__mobile_search_clients_daily__v1", + execution_delta=datetime.timedelta(seconds=9000), + check_existence=True, + mode="reschedule", + allowed_states=ALLOWED_STATES, + failed_states=FAILED_STATES, + pool="DATA_ENG_EXTERNALTASKSENSOR", + ) + wait_for_checks__fail_fenix_derived__active_users_aggregates__v3 = ( ExternalTaskSensor( task_id="wait_for_checks__fail_fenix_derived__active_users_aggregates__v3", @@ -229,6 +241,10 @@ wait_for_search_derived__search_aggregates__v8 ) + search_derived__mobile_search_aggregates_for_searchreport__v1.set_upstream( + wait_for_search_derived__mobile_search_clients_daily__v1 + ) + search_derived__search_revenue_levers_daily__v1.set_upstream( wait_for_checks__fail_fenix_derived__active_users_aggregates__v3 ) @@ -254,5 +270,9 @@ ) search_derived__search_revenue_levers_daily__v1.set_upstream( + wait_for_search_derived__mobile_search_clients_daily__v1 + ) + + search_derived__search_revenue_levers_daily__v1.set_upstream( wait_for_search_derived__search_clients_daily__v8 ) diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/dags/bqetl_unified.py /tmp/workspace/generated-sql/dags/bqetl_unified.py --- /tmp/workspace/main-generated-sql/dags/bqetl_unified.py 2024-06-03 15:29:02.000000000 +0000 +++ /tmp/workspace/generated-sql/dags/bqetl_unified.py 2024-06-03 15:41:12.000000000 +0000 @@ -128,6 +128,18 @@ pool="DATA_ENG_EXTERNALTASKSENSOR", ) + wait_for_search_derived__mobile_search_clients_daily__v1 = ExternalTaskSensor( + task_id="wait_for_search_derived__mobile_search_clients_daily__v1", + external_dag_id="bqetl_mobile_search", + external_task_id="search_derived__mobile_search_clients_daily__v1", + execution_delta=datetime.timedelta(seconds=3600), + check_existence=True, + mode="reschedule", + allowed_states=ALLOWED_STATES, + failed_states=FAILED_STATES, + pool="DATA_ENG_EXTERNALTASKSENSOR", + ) + wait_for_telemetry_derived__core_clients_last_seen__v1 = ExternalTaskSensor( task_id="wait_for_telemetry_derived__core_clients_last_seen__v1", external_dag_id="bqetl_core", @@ -329,5 +341,9 @@ ) telemetry_derived__unified_metrics__v1.set_upstream( + wait_for_search_derived__mobile_search_clients_daily__v1 + ) + + telemetry_derived__unified_metrics__v1.set_upstream( wait_for_telemetry_derived__core_clients_last_seen__v1 ) Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android: engagement Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android: engagement_clients Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android: retention Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android: retention_clients Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android_derived: engagement_v1 Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android_derived: retention_v1 Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_ios: engagement Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_ios: engagement_clients Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_ios: retention Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_ios: retention_clients Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_ios_derived: engagement_v1 Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_ios_derived: retention_v1 Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/klar_android: engagement Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/klar_android: engagement_clients Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/klar_android: retention Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/klar_android: retention_clients Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/klar_android_derived: engagement_v1 Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/klar_android_derived: retention_v1 Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/klar_ios: engagement Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/klar_ios: engagement_clients Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/klar_ios: retention Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/klar_ios: retention_clients Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/klar_ios_derived: engagement_v1 Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/klar_ios_derived: retention_v1 Only in /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/org_mozilla_fenix_nightly/creditcards_sync: schema.yaml Only in /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/org_mozilla_firefox/sync: schema.yaml Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/search/mobile_search_clients_engines_sources_daily: schema.yaml diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/bedrock_derived/event_monitoring_live_v1/materialized_view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/bedrock_derived/event_monitoring_live_v1/materialized_view.sql --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/bedrock_derived/event_monitoring_live_v1/materialized_view.sql 2024-06-03 15:26:02.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/bedrock_derived/event_monitoring_live_v1/materialized_view.sql 2024-06-03 15:27:50.000000000 +0000 @@ -50,7 +50,7 @@ client_info.app_display_version AS version, ping_info FROM - `moz-fx-data-shared-prod.bedrock_live.events_v1` + `moz-fx-data-shared-prod.bedrock_live.interaction_v1` UNION ALL SELECT submission_timestamp, @@ -60,7 +60,7 @@ client_info.app_display_version AS version, ping_info FROM - `moz-fx-data-shared-prod.bedrock_live.interaction_v1` + `moz-fx-data-shared-prod.bedrock_live.non_interaction_v1` UNION ALL SELECT submission_timestamp, @@ -70,7 +70,7 @@ client_info.app_display_version AS version, ping_info FROM - `moz-fx-data-shared-prod.bedrock_live.non_interaction_v1` + `moz-fx-data-shared-prod.bedrock_live.events_v1` ) CROSS JOIN UNNEST(events) AS event, diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/braze_derived/subscriptions_map_v1/script.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/braze_derived/subscriptions_map_v1/script.sql --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/braze_derived/subscriptions_map_v1/script.sql 2024-06-03 15:25:25.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/braze_derived/subscriptions_map_v1/script.sql 2024-06-03 15:26:06.000000000 +0000 @@ -131,14 +131,6 @@ 'mozilla-builder' ), ( - 'mozilla-builders-application-2024', - 'Mozilla Builder Application', - 'd48a2578-4963-4ac9-9d4e-3005106a3606', - 'd985ba67-a7ad-47a3-8358-04d223a16079', - '6fdbce2c-0110-4bd5-94ce-ff6f4e2ca905', - 'mozilla-builders-application-2024' - ), - ( 'mozilla-innovation', 'Innovation Newsletter', '39fcaf11-2926-4366-b756-2c02e65cbce7', diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates/schema.yaml --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates/schema.yaml 2024-06-03 15:25:24.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates/schema.yaml 2024-06-03 15:33:20.000000000 +0000 @@ -1,49 +1,49 @@ fields: -- mode: NULLABLE - name: submission_date +- name: submission_date type: DATE -- mode: NULLABLE - name: source + mode: NULLABLE +- name: source type: STRING -- mode: NULLABLE - name: event_type + mode: NULLABLE +- name: event_type type: STRING -- mode: NULLABLE - name: form_factor + mode: NULLABLE +- name: form_factor type: STRING -- mode: NULLABLE - name: country + mode: NULLABLE +- name: country type: STRING -- mode: NULLABLE - name: subdivision1 + mode: NULLABLE +- name: subdivision1 type: STRING -- mode: NULLABLE - name: advertiser + mode: NULLABLE +- name: advertiser type: STRING -- mode: NULLABLE - name: release_channel + mode: NULLABLE +- name: release_channel type: STRING -- mode: NULLABLE - name: position + mode: NULLABLE +- name: position type: INTEGER -- mode: NULLABLE - name: provider + mode: NULLABLE +- name: provider type: STRING -- mode: NULLABLE - name: match_type + mode: NULLABLE +- name: match_type type: STRING -- mode: NULLABLE - name: normalized_os + mode: NULLABLE +- name: normalized_os type: STRING -- mode: NULLABLE - name: suggest_data_sharing_enabled + mode: NULLABLE +- name: suggest_data_sharing_enabled type: BOOLEAN -- mode: NULLABLE - name: event_count + mode: NULLABLE +- name: event_count type: INTEGER -- mode: NULLABLE - name: user_count + mode: NULLABLE +- name: user_count type: INTEGER -- mode: NULLABLE - name: query_type + mode: NULLABLE +- name: query_type type: STRING + mode: NULLABLE diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates_suggest/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates_suggest/schema.yaml --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates_suggest/schema.yaml 2024-06-03 15:25:24.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates_suggest/schema.yaml 2024-06-03 15:33:19.000000000 +0000 @@ -1,40 +1,40 @@ fields: -- mode: NULLABLE - name: submission_date +- name: submission_date type: DATE -- mode: NULLABLE - name: form_factor + mode: NULLABLE +- name: form_factor type: STRING -- mode: NULLABLE - name: country + mode: NULLABLE +- name: country type: STRING -- mode: NULLABLE - name: advertiser + mode: NULLABLE +- name: advertiser type: STRING -- mode: NULLABLE - name: normalized_os + mode: NULLABLE +- name: normalized_os type: STRING -- mode: NULLABLE - name: release_channel + mode: NULLABLE +- name: release_channel type: STRING -- mode: NULLABLE - name: position + mode: NULLABLE +- name: position type: INTEGER -- mode: NULLABLE - name: provider + mode: NULLABLE +- name: provider type: STRING -- mode: NULLABLE - name: match_type + mode: NULLABLE +- name: match_type type: STRING -- mode: NULLABLE - name: suggest_data_sharing_enabled + mode: NULLABLE +- name: suggest_data_sharing_enabled type: BOOLEAN -- mode: NULLABLE - name: impression_count + mode: NULLABLE +- name: impression_count type: INTEGER -- mode: NULLABLE - name: click_count + mode: NULLABLE +- name: click_count type: INTEGER -- mode: NULLABLE - name: query_type + mode: NULLABLE +- name: query_type type: STRING + mode: NULLABLE diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/engagement_clients/view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/engagement_clients/view.sql --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/engagement_clients/view.sql 2024-06-03 15:26:01.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/engagement_clients/view.sql 2024-06-03 15:27:50.000000000 +0000 @@ -26,19 +26,19 @@ client_id, sample_id, channel AS normalized_channel, + NULLIF(play_store_attribution_campaign, "") AS play_store_attribution_campaign, + NULLIF(play_store_attribution_medium, "") AS play_store_attribution_medium, + NULLIF(play_store_attribution_source, "") AS play_store_attribution_source, + NULLIF(meta_attribution_app, "") AS meta_attribution_app, + NULLIF(install_source, "") AS install_source, NULLIF(adjust_ad_group, "") AS adjust_ad_group, - NULLIF(adjust_creative, "") AS adjust_creative, - NULLIF(adjust_network, "") AS adjust_network, CASE WHEN adjust_network IN ('Google Organic Search', 'Organic') THEN 'Organic' ELSE NULLIF(adjust_campaign, "") END AS adjust_campaign, - NULLIF(play_store_attribution_campaign, "") AS play_store_attribution_campaign, - NULLIF(play_store_attribution_medium, "") AS play_store_attribution_medium, - NULLIF(play_store_attribution_source, "") AS play_store_attribution_source, - NULLIF(meta_attribution_app, "") AS meta_attribution_app, - NULLIF(install_source, "") AS install_source, + NULLIF(adjust_creative, "") AS adjust_creative, + NULLIF(adjust_network, "") AS adjust_network, FROM `moz-fx-data-shared-prod.fenix_derived.firefox_android_clients_v1` ) @@ -57,15 +57,15 @@ is_wau, is_mau, is_mobile, - attribution.adjust_ad_group, - attribution.adjust_campaign, - attribution.adjust_creative, - attribution.adjust_network, attribution.play_store_attribution_campaign, attribution.play_store_attribution_medium, attribution.play_store_attribution_source, attribution.meta_attribution_app, attribution.install_source, + attribution.adjust_ad_group, + attribution.adjust_campaign, + attribution.adjust_creative, + attribution.adjust_network, CASE WHEN active_users.submission_date = first_seen_date THEN 'new_profile' diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_clients/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_clients/schema.yaml --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_clients/schema.yaml 2024-06-03 15:25:24.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_clients/schema.yaml 2024-06-03 15:35:05.000000000 +0000 @@ -26,6 +26,9 @@ - name: adjust_network type: STRING mode: NULLABLE +- name: install_source + type: STRING + mode: NULLABLE - name: retained_week_2 type: BOOLEAN mode: NULLABLE diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_week_4/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_week_4/schema.yaml --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_week_4/schema.yaml 2024-06-03 15:25:24.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_week_4/schema.yaml 2024-06-03 15:35:04.000000000 +0000 @@ -48,6 +48,10 @@ description: 'The type of source of a client installation. ' +- name: install_source + type: STRING + mode: NULLABLE + description: null - name: new_profiles type: INTEGER mode: NULLABLE diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/retention_clients/view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/retention_clients/view.sql --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/retention_clients/view.sql 2024-06-03 15:26:01.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/retention_clients/view.sql 2024-06-03 15:27:50.000000000 +0000 @@ -22,19 +22,19 @@ client_id, sample_id, channel AS normalized_channel, + NULLIF(play_store_attribution_campaign, "") AS play_store_attribution_campaign, + NULLIF(play_store_attribution_medium, "") AS play_store_attribution_medium, + NULLIF(play_store_attribution_source, "") AS play_store_attribution_source, + NULLIF(meta_attribution_app, "") AS meta_attribution_app, + NULLIF(install_source, "") AS install_source, NULLIF(adjust_ad_group, "") AS adjust_ad_group, - NULLIF(adjust_creative, "") AS adjust_creative, - NULLIF(adjust_network, "") AS adjust_network, CASE WHEN adjust_network IN ('Google Organic Search', 'Organic') THEN 'Organic' ELSE NULLIF(adjust_campaign, "") END AS adjust_campaign, - NULLIF(play_store_attribution_campaign, "") AS play_store_attribution_campaign, - NULLIF(play_store_attribution_medium, "") AS play_store_attribution_medium, - NULLIF(play_store_attribution_source, "") AS play_store_attribution_source, - NULLIF(meta_attribution_app, "") AS meta_attribution_app, - NULLIF(install_source, "") AS install_source, + NULLIF(adjust_creative, "") AS adjust_creative, + NULLIF(adjust_network, "") AS adjust_network, FROM `moz-fx-data-shared-prod.fenix_derived.firefox_android_clients_v1` ) @@ -56,6 +56,10 @@ attribution.play_store_attribution_source, attribution.meta_attribution_app, attribution.install_source, + attribution.adjust_ad_group, + attribution.adjust_campaign, + attribution.adjust_creative, + attribution.adjust_network, -- ping sent retention active_users.retention_seen.day_27.active_on_metric_date AS ping_sent_metric_date, ( @@ -79,10 +83,6 @@ -- Looking back at 27 days to support the official definition of repeat_profile (someone active between days 2 and 28): AND BIT_COUNT(mozfun.bits28.range(active_users.days_active_bits, -26, 27)) > 0 ) AS repeat_profile, - attribution.adjust_ad_group, - attribution.adjust_campaign, - attribution.adjust_creative, - attribution.adjust_network, active_users.days_seen_bits, active_users.days_active_bits, CASE diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/engagement_v1/query.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/engagement_v1/query.sql --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/engagement_v1/query.sql 2024-06-03 15:26:03.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/engagement_v1/query.sql 2024-06-03 15:27:50.000000000 +0000 @@ -8,22 +8,26 @@ country, locale, is_mobile, - adjust_ad_group, - adjust_campaign, - adjust_creative, - adjust_network, play_store_attribution_campaign, play_store_attribution_medium, play_store_attribution_source, meta_attribution_app, install_source, + adjust_ad_group, + adjust_campaign, + adjust_creative, + adjust_network, COUNTIF(is_dau) AS dau, COUNTIF(is_wau) AS wau, - COUNTIF(is_mau) AS mau + COUNTIF(is_mau) AS mau, FROM `moz-fx-data-shared-prod.fenix.engagement_clients` WHERE + {% if is_init() %} + submission_date < CURRENT_DATE + {% else %} submission_date = @submission_date + {% endif %} GROUP BY submission_date, first_seen_date, @@ -33,12 +37,12 @@ country, locale, is_mobile, - adjust_ad_group, - adjust_campaign, - adjust_creative, - adjust_network, play_store_attribution_campaign, play_store_attribution_medium, play_store_attribution_source, meta_attribution_app, - install_source + install_source, + adjust_ad_group, + adjust_campaign, + adjust_creative, + adjust_network diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/engagement_v1/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/engagement_v1/schema.yaml --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/engagement_v1/schema.yaml 2024-06-03 15:26:03.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/engagement_v1/schema.yaml 2024-06-03 15:27:50.000000000 +0000 @@ -39,50 +39,50 @@ mode: NULLABLE description: Indicates if this specific entry is used towards calculating mobile DAU. -- name: adjust_ad_group +- name: play_store_attribution_campaign type: STRING mode: NULLABLE - description: Adjust Ad Group the profile is attributed to. + description: Play store campaign the profile is attributed to. -- name: adjust_campaign +- name: play_store_attribution_medium type: STRING mode: NULLABLE - description: Adjust Campaign the profile is attributed to. + description: Play store Medium the profile is attributed to. -- name: adjust_creative +- name: play_store_attribution_source type: STRING mode: NULLABLE - description: Adjust Creative the profile is attributed to. + description: Play store source the profile is attributed to. -- name: adjust_network +- name: meta_attribution_app type: STRING mode: NULLABLE - description: Adjust Network the profile is attributed to. + description: Facebook app linked to paid marketing. -- name: play_store_attribution_campaign +- name: install_source type: STRING mode: NULLABLE - description: Play store campaign the profile is attributed to. + description: The source of a profile installation. -- name: play_store_attribution_medium +- name: adjust_ad_group type: STRING mode: NULLABLE - description: Play store Medium the profile is attributed to. + description: Adjust Ad Group the profile is attributed to. -- name: play_store_attribution_source +- name: adjust_campaign type: STRING mode: NULLABLE - description: Play store source the profile is attributed to. + description: Adjust Campaign the profile is attributed to. -- name: meta_attribution_app +- name: adjust_creative type: STRING mode: NULLABLE - description: Facebook app linked to paid marketing. + description: Adjust Creative the profile is attributed to. -- name: install_source +- name: adjust_network type: STRING mode: NULLABLE - description: The source of a profile installation. + description: Adjust Network the profile is attributed to. - name: dau type: INTEGER diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/retention_v1/query.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/retention_v1/query.sql --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/retention_v1/query.sql 2024-06-03 15:26:03.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/retention_v1/query.sql 2024-06-03 15:27:50.000000000 +0000 @@ -8,15 +8,15 @@ app_version, locale, is_mobile, - adjust_ad_group, - adjust_campaign, - adjust_creative, - adjust_network, play_store_attribution_campaign, play_store_attribution_medium, play_store_attribution_source, meta_attribution_app, install_source, + adjust_ad_group, + adjust_campaign, + adjust_creative, + adjust_network, COUNTIF(ping_sent_metric_date) AS ping_sent_metric_date, COUNTIF(ping_sent_week_4) AS ping_sent_week_4, COUNTIF(active_metric_date) AS active_metric_date, @@ -27,8 +27,13 @@ FROM `moz-fx-data-shared-prod.fenix.retention_clients` WHERE + {% if is_init() %} + metric_date < DATE_SUB(CURRENT_DATE, INTERVAL 27 DAY) + AND submission_date < CURRENT_DATE + {% else %} metric_date = DATE_SUB(@submission_date, INTERVAL 27 DAY) AND submission_date = @submission_date + {% endif %} GROUP BY metric_date, first_seen_date, @@ -38,12 +43,12 @@ app_version, locale, is_mobile, - adjust_ad_group, - adjust_campaign, - adjust_creative, - adjust_network, play_store_attribution_campaign, play_store_attribution_medium, play_store_attribution_source, meta_attribution_app, - install_source + install_source, + adjust_ad_group, + adjust_campaign, + adjust_creative, + adjust_network diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/retention_v1/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/retention_v1/schema.yaml --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/retention_v1/schema.yaml 2024-06-03 15:26:03.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/retention_v1/schema.yaml 2024-06-03 15:27:50.000000000 +0000 @@ -1,5 +1,4 @@ fields: - - mode: NULLABLE name: metric_date type: DATE @@ -40,50 +39,50 @@ mode: NULLABLE description: Indicates if this specific entry is used towards calculating mobile DAU. -- name: adjust_ad_group +- name: play_store_attribution_campaign type: STRING mode: NULLABLE - description: Adjust Ad Group the profile is attributed to. + description: Play store campaign the profile is attributed to. -- name: adjust_campaign +- name: play_store_attribution_medium type: STRING mode: NULLABLE - description: Adjust Campaign the profile is attributed to. + description: Play store Medium the profile is attributed to. -- name: adjust_creative +- name: play_store_attribution_source type: STRING mode: NULLABLE - description: Adjust Creative the profile is attributed to. + description: Play store source the profile is attributed to. -- name: adjust_network +- name: meta_attribution_app type: STRING mode: NULLABLE - description: Adjust Network the profile is attributed to. + description: Facebook app linked to paid marketing. -- name: play_store_attribution_campaign +- name: install_source type: STRING mode: NULLABLE - description: Play store campaign the profile is attributed to. + description: The source of a profile installation. -- name: play_store_attribution_medium +- name: adjust_ad_group type: STRING mode: NULLABLE - description: Play store Medium the profile is attributed to. + description: Adjust Ad Group the profile is attributed to. -- name: play_store_attribution_source +- name: adjust_campaign type: STRING mode: NULLABLE - description: Play store source the profile is attributed to. + description: Adjust Campaign the profile is attributed to. -- name: meta_attribution_app +- name: adjust_creative type: STRING mode: NULLABLE - description: Facebook app linked to paid marketing. + description: Adjust Creative the profile is attributed to. -- name: install_source +- name: adjust_network type: STRING mode: NULLABLE - description: The source of a profile installation. + description: Adjust Network the profile is attributed to. - name: ping_sent_metric_date type: INTEGER diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_background_tasks_derived/event_monitoring_live_v1/materialized_view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_background_tasks_derived/event_monitoring_live_v1/materialized_view.sql --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_background_tasks_derived/event_monitoring_live_v1/materialized_view.sql 2024-06-03 15:26:03.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_background_tasks_derived/event_monitoring_live_v1/materialized_view.sql 2024-06-03 15:27:51.000000000 +0000 @@ -50,7 +50,7 @@ client_info.app_display_version AS version, ping_info FROM - `moz-fx-data-shared-prod.firefox_desktop_background_tasks_live.events_v1` + `moz-fx-data-shared-prod.firefox_desktop_background_tasks_live.background_tasks_v1` UNION ALL SELECT submission_timestamp, @@ -60,7 +60,7 @@ client_info.app_display_version AS version, ping_info FROM - `moz-fx-data-shared-prod.firefox_desktop_background_tasks_live.background_tasks_v1` + `moz-fx-data-shared-prod.firefox_desktop_background_tasks_live.events_v1` ) CROSS JOIN UNNEST(events) AS event, diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_derived/event_monitoring_live_v1/materialized_view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_derived/event_monitoring_live_v1/materialized_view.sql --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_derived/event_monitoring_live_v1/materialized_view.sql 2024-06-03 15:26:02.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_derived/event_monitoring_live_v1/materialized_view.sql 2024-06-03 15:27:51.000000000 +0000 @@ -50,7 +50,7 @@ client_info.app_display_version AS version, ping_info FROM - `moz-fx-data-shared-prod.firefox_desktop_live.newtab_v1` + `moz-fx-data-shared-prod.firefox_desktop_live.urlbar_potential_exposure_v1` UNION ALL SELECT submission_timestamp, @@ -60,7 +60,7 @@ client_info.app_display_version AS version, ping_info FROM - `moz-fx-data-shared-prod.firefox_desktop_live.urlbar_potential_exposure_v1` + `moz-fx-data-shared-prod.firefox_desktop_live.newtab_v1` UNION ALL SELECT submission_timestamp, @@ -70,7 +70,7 @@ client_info.app_display_version AS version, ping_info FROM - `moz-fx-data-shared-prod.firefox_desktop_live.events_v1` + `moz-fx-data-shared-prod.firefox_desktop_live.prototype_no_code_events_v1` UNION ALL SELECT submission_timestamp, @@ -80,7 +80,7 @@ client_info.app_display_version AS version, ping_info FROM - `moz-fx-data-shared-prod.firefox_desktop_live.prototype_no_code_events_v1` + `moz-fx-data-shared-prod.firefox_desktop_live.events_v1` ) CROSS JOIN UNNEST(events) AS event, diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_ios/engagement_clients/view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios/engagement_clients/view.sql --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_ios/engagement_clients/view.sql 2024-06-03 15:26:02.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios/engagement_clients/view.sql 2024-06-03 15:27:50.000000000 +0000 @@ -26,11 +26,11 @@ client_id, sample_id, channel AS normalized_channel, + is_suspicious_device_client, NULLIF(adjust_ad_group, "") AS adjust_ad_group, + NULLIF(adjust_campaign, "") AS adjust_campaign, NULLIF(adjust_creative, "") AS adjust_creative, NULLIF(adjust_network, "") AS adjust_network, - NULLIF(adjust_campaign, "") AS adjust_campaign, - is_suspicious_device_client FROM `moz-fx-data-shared-prod.firefox_ios_derived.firefox_ios_clients_v1` ) @@ -49,11 +49,11 @@ is_wau, is_mau, is_mobile, + attribution.is_suspicious_device_client, attribution.adjust_ad_group, attribution.adjust_campaign, attribution.adjust_creative, attribution.adjust_network, - attribution.is_suspicious_device_client, CASE WHEN active_users.submission_date = first_seen_date THEN 'new_profile' diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_ios/retention_clients/view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios/retention_clients/view.sql --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_ios/retention_clients/view.sql 2024-06-03 15:26:02.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios/retention_clients/view.sql 2024-06-03 15:27:50.000000000 +0000 @@ -22,11 +22,11 @@ client_id, sample_id, channel AS normalized_channel, + is_suspicious_device_client, NULLIF(adjust_ad_group, "") AS adjust_ad_group, + NULLIF(adjust_campaign, "") AS adjust_campaign, NULLIF(adjust_creative, "") AS adjust_creative, NULLIF(adjust_network, "") AS adjust_network, - NULLIF(adjust_campaign, "") AS adjust_campaign, - is_suspicious_device_client FROM `moz-fx-data-shared-prod.firefox_ios_derived.firefox_ios_clients_v1` ) @@ -44,6 +44,10 @@ clients_daily.isp, active_users.is_mobile, attribution.is_suspicious_device_client, + attribution.adjust_ad_group, + attribution.adjust_campaign, + attribution.adjust_creative, + attribution.adjust_network, -- ping sent retention active_users.retention_seen.day_27.active_on_metric_date AS ping_sent_metric_date, ( @@ -67,10 +71,6 @@ -- Looking back at 27 days to support the official definition of repeat_profile (someone active between days 2 and 28): AND BIT_COUNT(mozfun.bits28.range(active_users.days_active_bits, -26, 27)) > 0 ) AS repeat_profile, - attribution.adjust_ad_group, - attribution.adjust_campaign, - attribution.adjust_creative, - attribution.adjust_network, active_users.days_seen_bits, active_users.days_active_bits, CASE diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/engagement_v1/query.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/engagement_v1/query.sql --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/engagement_v1/query.sql 2024-06-03 15:26:03.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/engagement_v1/query.sql 2024-06-03 15:27:50.000000000 +0000 @@ -8,18 +8,22 @@ country, locale, is_mobile, + is_suspicious_device_client, adjust_ad_group, adjust_campaign, adjust_creative, adjust_network, - is_suspicious_device_client, COUNTIF(is_dau) AS dau, COUNTIF(is_wau) AS wau, - COUNTIF(is_mau) AS mau + COUNTIF(is_mau) AS mau, FROM `moz-fx-data-shared-prod.firefox_ios.engagement_clients` WHERE + {% if is_init() %} + submission_date < CURRENT_DATE + {% else %} submission_date = @submission_date + {% endif %} GROUP BY submission_date, first_seen_date, @@ -29,8 +33,8 @@ country, locale, is_mobile, + is_suspicious_device_client, adjust_ad_group, adjust_campaign, adjust_creative, - adjust_network, - is_suspicious_device_client + adjust_network diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/engagement_v1/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/engagement_v1/schema.yaml --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/engagement_v1/schema.yaml 2024-06-03 15:26:03.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/engagement_v1/schema.yaml 2024-06-03 15:27:50.000000000 +0000 @@ -39,6 +39,11 @@ mode: NULLABLE description: Indicates if this specific entry is used towards calculating mobile DAU. +- name: is_suspicious_device_client + type: BOOLEAN + mode: NULLABLE + description: Flag to identify suspicious device users, see bug-1846554 for more info. + - name: adjust_ad_group type: STRING mode: NULLABLE @@ -59,11 +64,6 @@ mode: NULLABLE description: Adjust Network the profile is attributed to. -- name: is_suspicious_device_client - type: BOOLEAN - mode: NULLABLE - description: Flag to identify suspicious device users, see bug-1846554 for more info. - - name: dau type: INTEGER mode: NULLABLE diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/retention_v1/query.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/retention_v1/query.sql --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/retention_v1/query.sql 2024-06-03 15:26:03.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/retention_v1/query.sql 2024-06-03 15:27:50.000000000 +0000 @@ -8,11 +8,11 @@ app_version, locale, is_mobile, + is_suspicious_device_client, adjust_ad_group, adjust_campaign, adjust_creative, adjust_network, - is_suspicious_device_client, COUNTIF(ping_sent_metric_date) AS ping_sent_metric_date, COUNTIF(ping_sent_week_4) AS ping_sent_week_4, COUNTIF(active_metric_date) AS active_metric_date, @@ -23,8 +23,13 @@ FROM `moz-fx-data-shared-prod.firefox_ios.retention_clients` WHERE + {% if is_init() %} + metric_date < DATE_SUB(CURRENT_DATE, INTERVAL 27 DAY) + AND submission_date < CURRENT_DATE + {% else %} metric_date = DATE_SUB(@submission_date, INTERVAL 27 DAY) AND submission_date = @submission_date + {% endif %} GROUP BY metric_date, first_seen_date, @@ -34,8 +39,8 @@ app_version, locale, is_mobile, + is_suspicious_device_client, adjust_ad_group, adjust_campaign, adjust_creative, - adjust_network, - is_suspicious_device_client + adjust_network diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/retention_v1/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/retention_v1/schema.yaml --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/retention_v1/schema.yaml 2024-06-03 15:26:03.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/retention_v1/schema.yaml 2024-06-03 15:27:50.000000000 +0000 @@ -1,5 +1,4 @@ fields: - - mode: NULLABLE name: metric_date type: DATE @@ -40,6 +39,11 @@ mode: NULLABLE description: Indicates if this specific entry is used towards calculating mobile DAU. +- name: is_suspicious_device_client + type: BOOLEAN + mode: NULLABLE + description: Flag to identify suspicious device users, see bug-1846554 for more info. + - name: adjust_ad_group type: STRING mode: NULLABLE @@ -60,11 +64,6 @@ mode: NULLABLE description: Adjust Network the profile is attributed to. -- name: is_suspicious_device_client - type: BOOLEAN - mode: NULLABLE - description: Flag to identify suspicious device users, see bug-1846554 for more info. - - name: ping_sent_metric_date type: INTEGER mode: NULLABLE diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android/engagement/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android/engagement/metadata.yaml --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android/engagement/metadata.yaml 1970-01-01 00:00:00.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android/engagement/metadata.yaml 2024-06-03 15:38:18.000000000 +0000 @@ -0,0 +1,13 @@ +friendly_name: Engagement +description: |- + Please provide a description for the query +owners: [] +labels: {} +bigquery: null +workgroup_access: +- role: roles/bigquery.dataViewer + members: + - workgroup:mozilla-confidential +references: + view.sql: + - moz-fx-data-shared-prod.focus_android_derived.engagement_v1 diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android/engagement/view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android/engagement/view.sql --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android/engagement/view.sql 1970-01-01 00:00:00.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android/engagement/view.sql 2024-06-03 15:27:50.000000000 +0000 @@ -0,0 +1,19 @@ +-- Query generated via `mobile_kpi_support_metrics` SQL generator. +CREATE OR REPLACE VIEW + `moz-fx-data-shared-prod.focus_android.engagement` +AS +SELECT + *, + CASE + WHEN first_seen_date = submission_date + THEN 'new_profile' + WHEN DATE_DIFF(submission_date, first_seen_date, DAY) + BETWEEN 1 + AND 27 + THEN 'repeat_user' + WHEN DATE_DIFF(submission_date, first_seen_date, DAY) >= 28 + THEN 'existing_user' + ELSE 'Unknown' + END AS lifecycle_stage, +FROM + `moz-fx-data-shared-prod.focus_android_derived.engagement_v1` diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android/engagement_clients/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android/engagement_clients/metadata.yaml --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android/engagement_clients/metadata.yaml 1970-01-01 00:00:00.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android/engagement_clients/metadata.yaml 2024-06-03 15:38:18.000000000 +0000 @@ -0,0 +1,13 @@ +friendly_name: Engagement Clients +description: |- + Please provide a description for the query +owners: [] +labels: {} +bigquery: null +workgroup_access: +- role: roles/bigquery.dataViewer + members: + - workgroup:mozilla-confidential +references: + view.sql: + - moz-fx-data-shared-prod.focus_android.active_users diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android/engagement_clients/view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android/engagement_clients/view.sql --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android/engagement_clients/view.sql 1970-01-01 00:00:00.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android/engagement_clients/view.sql 2024-06-03 15:27:50.000000000 +0000 @@ -0,0 +1,51 @@ +-- Query generated via `mobile_kpi_support_metrics` SQL generator. +CREATE OR REPLACE VIEW + `moz-fx-data-shared-prod.focus_android.engagement_clients` +AS +WITH active_users AS ( + SELECT + submission_date, + client_id, + sample_id, + first_seen_date, + app_name, + normalized_channel, + locale, + country, + isp, + app_display_version, + is_dau, + is_wau, + is_mau, + is_mobile, + FROM + `moz-fx-data-shared-prod.focus_android.active_users` +) +SELECT + submission_date, + client_id, + sample_id, + first_seen_date, + app_name, + normalized_channel, + app_display_version AS app_version, + locale, + country, + isp, + is_dau, + is_wau, + is_mau, + is_mobile, + CASE + WHEN active_users.submission_date = first_seen_date + THEN 'new_profile' + WHEN DATE_DIFF(active_users.submission_date, first_seen_date, DAY) + BETWEEN 1 + AND 27 + THEN 'repeat_user' + WHEN DATE_DIFF(active_users.submission_date, first_seen_date, DAY) >= 28 + THEN 'existing_user' + ELSE 'Unknown' + END AS lifecycle_stage, +FROM + active_users diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android/retention/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android/retention/metadata.yaml --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android/retention/metadata.yaml 1970-01-01 00:00:00.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android/retention/metadata.yaml 2024-06-03 15:38:18.000000000 +0000 @@ -0,0 +1,13 @@ +friendly_name: Retention +description: |- + Please provide a description for the query +owners: [] +labels: {} +bigquery: null +workgroup_access: +- role: roles/bigquery.dataViewer + members: + - workgroup:mozilla-confidential +references: + view.sql: + - moz-fx-data-shared-prod.focus_android_derived.retention_v1 diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android/retention/view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android/retention/view.sql --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android/retention/view.sql 1970-01-01 00:00:00.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android/retention/view.sql 2024-06-03 15:27:50.000000000 +0000 @@ -0,0 +1,19 @@ +-- Query generated via `mobile_kpi_support_metrics` SQL generator. +CREATE OR REPLACE VIEW + `moz-fx-data-shared-prod.focus_android.retention` +AS +SELECT + *, + CASE + WHEN first_seen_date = metric_date + THEN 'new_profile' + WHEN DATE_DIFF(metric_date, first_seen_date, DAY) + BETWEEN 1 + AND 27 + THEN 'repeat_user' + WHEN DATE_DIFF(metric_date, first_seen_date, DAY) >= 28 + THEN 'existing_user' + ELSE 'Unknown' + END AS lifecycle_stage, +FROM + `moz-fx-data-shared-prod.focus_android_derived.retention_v1` diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android/retention_clients/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android/retention_clients/metadata.yaml --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android/retention_clients/metadata.yaml 1970-01-01 00:00:00.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android/retention_clients/metadata.yaml 2024-06-03 15:38:18.000000000 +0000 @@ -0,0 +1,14 @@ +friendly_name: Retention Clients +description: |- + Please provide a description for the query +owners: [] +labels: { ```

⚠️ Only part of the diff is displayed.

Link to full diff

dataops-ci-bot commented 1 month ago

Integration report for "Merge branch 'main' into feat/update-mobile_kpi_generator-to-generate-all-templates-for-all-products"

sql.diff

Click to expand! ```diff Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android: engagement Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android: engagement_clients Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android: retention Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android: retention_clients Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android_derived: engagement_v1 Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android_derived: retention_v1 Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_ios: engagement Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_ios: engagement_clients Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_ios: retention Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_ios: retention_clients Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_ios_derived: engagement_v1 Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_ios_derived: retention_v1 Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/klar_android: engagement Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/klar_android: engagement_clients Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/klar_android: retention Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/klar_android: retention_clients Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/klar_android_derived: engagement_v1 Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/klar_android_derived: retention_v1 Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/klar_ios: engagement Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/klar_ios: engagement_clients Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/klar_ios: retention Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/klar_ios: retention_clients Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/klar_ios_derived: engagement_v1 Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/klar_ios_derived: retention_v1 diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/bedrock_derived/event_monitoring_live_v1/materialized_view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/bedrock_derived/event_monitoring_live_v1/materialized_view.sql --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/bedrock_derived/event_monitoring_live_v1/materialized_view.sql 2024-06-03 15:59:35.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/bedrock_derived/event_monitoring_live_v1/materialized_view.sql 2024-06-03 16:01:23.000000000 +0000 @@ -60,7 +60,7 @@ client_info.app_display_version AS version, ping_info FROM - `moz-fx-data-shared-prod.bedrock_live.interaction_v1` + `moz-fx-data-shared-prod.bedrock_live.non_interaction_v1` UNION ALL SELECT submission_timestamp, @@ -70,7 +70,7 @@ client_info.app_display_version AS version, ping_info FROM - `moz-fx-data-shared-prod.bedrock_live.non_interaction_v1` + `moz-fx-data-shared-prod.bedrock_live.interaction_v1` ) CROSS JOIN UNNEST(events) AS event, diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates/schema.yaml --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates/schema.yaml 2024-06-03 15:58:54.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates/schema.yaml 2024-06-03 16:07:19.000000000 +0000 @@ -1,49 +1,49 @@ fields: -- mode: NULLABLE - name: submission_date +- name: submission_date type: DATE -- mode: NULLABLE - name: source + mode: NULLABLE +- name: source type: STRING -- mode: NULLABLE - name: event_type + mode: NULLABLE +- name: event_type type: STRING -- mode: NULLABLE - name: form_factor + mode: NULLABLE +- name: form_factor type: STRING -- mode: NULLABLE - name: country + mode: NULLABLE +- name: country type: STRING -- mode: NULLABLE - name: subdivision1 + mode: NULLABLE +- name: subdivision1 type: STRING -- mode: NULLABLE - name: advertiser + mode: NULLABLE +- name: advertiser type: STRING -- mode: NULLABLE - name: release_channel + mode: NULLABLE +- name: release_channel type: STRING -- mode: NULLABLE - name: position + mode: NULLABLE +- name: position type: INTEGER -- mode: NULLABLE - name: provider + mode: NULLABLE +- name: provider type: STRING -- mode: NULLABLE - name: match_type + mode: NULLABLE +- name: match_type type: STRING -- mode: NULLABLE - name: normalized_os + mode: NULLABLE +- name: normalized_os type: STRING -- mode: NULLABLE - name: suggest_data_sharing_enabled + mode: NULLABLE +- name: suggest_data_sharing_enabled type: BOOLEAN -- mode: NULLABLE - name: event_count + mode: NULLABLE +- name: event_count type: INTEGER -- mode: NULLABLE - name: user_count + mode: NULLABLE +- name: user_count type: INTEGER -- mode: NULLABLE - name: query_type + mode: NULLABLE +- name: query_type type: STRING + mode: NULLABLE diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates_suggest/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates_suggest/schema.yaml --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates_suggest/schema.yaml 2024-06-03 15:58:54.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates_suggest/schema.yaml 2024-06-03 16:07:19.000000000 +0000 @@ -1,40 +1,40 @@ fields: -- mode: NULLABLE - name: submission_date +- name: submission_date type: DATE -- mode: NULLABLE - name: form_factor + mode: NULLABLE +- name: form_factor type: STRING -- mode: NULLABLE - name: country + mode: NULLABLE +- name: country type: STRING -- mode: NULLABLE - name: advertiser + mode: NULLABLE +- name: advertiser type: STRING -- mode: NULLABLE - name: normalized_os + mode: NULLABLE +- name: normalized_os type: STRING -- mode: NULLABLE - name: release_channel + mode: NULLABLE +- name: release_channel type: STRING -- mode: NULLABLE - name: position + mode: NULLABLE +- name: position type: INTEGER -- mode: NULLABLE - name: provider + mode: NULLABLE +- name: provider type: STRING -- mode: NULLABLE - name: match_type + mode: NULLABLE +- name: match_type type: STRING -- mode: NULLABLE - name: suggest_data_sharing_enabled + mode: NULLABLE +- name: suggest_data_sharing_enabled type: BOOLEAN -- mode: NULLABLE - name: impression_count + mode: NULLABLE +- name: impression_count type: INTEGER -- mode: NULLABLE - name: click_count + mode: NULLABLE +- name: click_count type: INTEGER -- mode: NULLABLE - name: query_type + mode: NULLABLE +- name: query_type type: STRING + mode: NULLABLE diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/engagement_clients/view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/engagement_clients/view.sql --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/engagement_clients/view.sql 2024-06-03 15:59:34.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/engagement_clients/view.sql 2024-06-03 16:01:23.000000000 +0000 @@ -26,19 +26,19 @@ client_id, sample_id, channel AS normalized_channel, + NULLIF(play_store_attribution_campaign, "") AS play_store_attribution_campaign, + NULLIF(play_store_attribution_medium, "") AS play_store_attribution_medium, + NULLIF(play_store_attribution_source, "") AS play_store_attribution_source, + NULLIF(meta_attribution_app, "") AS meta_attribution_app, + NULLIF(install_source, "") AS install_source, NULLIF(adjust_ad_group, "") AS adjust_ad_group, - NULLIF(adjust_creative, "") AS adjust_creative, - NULLIF(adjust_network, "") AS adjust_network, CASE WHEN adjust_network IN ('Google Organic Search', 'Organic') THEN 'Organic' ELSE NULLIF(adjust_campaign, "") END AS adjust_campaign, - NULLIF(play_store_attribution_campaign, "") AS play_store_attribution_campaign, - NULLIF(play_store_attribution_medium, "") AS play_store_attribution_medium, - NULLIF(play_store_attribution_source, "") AS play_store_attribution_source, - NULLIF(meta_attribution_app, "") AS meta_attribution_app, - NULLIF(install_source, "") AS install_source, + NULLIF(adjust_creative, "") AS adjust_creative, + NULLIF(adjust_network, "") AS adjust_network, FROM `moz-fx-data-shared-prod.fenix_derived.firefox_android_clients_v1` ) @@ -57,15 +57,15 @@ is_wau, is_mau, is_mobile, - attribution.adjust_ad_group, - attribution.adjust_campaign, - attribution.adjust_creative, - attribution.adjust_network, attribution.play_store_attribution_campaign, attribution.play_store_attribution_medium, attribution.play_store_attribution_source, attribution.meta_attribution_app, attribution.install_source, + attribution.adjust_ad_group, + attribution.adjust_campaign, + attribution.adjust_creative, + attribution.adjust_network, CASE WHEN active_users.submission_date = first_seen_date THEN 'new_profile' diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_clients/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_clients/schema.yaml --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_clients/schema.yaml 2024-06-03 15:58:54.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_clients/schema.yaml 2024-06-03 16:06:45.000000000 +0000 @@ -26,6 +26,9 @@ - name: adjust_network type: STRING mode: NULLABLE +- name: install_source + type: STRING + mode: NULLABLE - name: retained_week_2 type: BOOLEAN mode: NULLABLE diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_week_4/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_week_4/schema.yaml --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_week_4/schema.yaml 2024-06-03 15:58:54.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_week_4/schema.yaml 2024-06-03 16:06:46.000000000 +0000 @@ -48,6 +48,10 @@ description: 'The type of source of a client installation. ' +- name: install_source + type: STRING + mode: NULLABLE + description: null - name: new_profiles type: INTEGER mode: NULLABLE diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/retention_clients/view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/retention_clients/view.sql --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/retention_clients/view.sql 2024-06-03 15:59:34.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/retention_clients/view.sql 2024-06-03 16:01:22.000000000 +0000 @@ -22,19 +22,19 @@ client_id, sample_id, channel AS normalized_channel, + NULLIF(play_store_attribution_campaign, "") AS play_store_attribution_campaign, + NULLIF(play_store_attribution_medium, "") AS play_store_attribution_medium, + NULLIF(play_store_attribution_source, "") AS play_store_attribution_source, + NULLIF(meta_attribution_app, "") AS meta_attribution_app, + NULLIF(install_source, "") AS install_source, NULLIF(adjust_ad_group, "") AS adjust_ad_group, - NULLIF(adjust_creative, "") AS adjust_creative, - NULLIF(adjust_network, "") AS adjust_network, CASE WHEN adjust_network IN ('Google Organic Search', 'Organic') THEN 'Organic' ELSE NULLIF(adjust_campaign, "") END AS adjust_campaign, - NULLIF(play_store_attribution_campaign, "") AS play_store_attribution_campaign, - NULLIF(play_store_attribution_medium, "") AS play_store_attribution_medium, - NULLIF(play_store_attribution_source, "") AS play_store_attribution_source, - NULLIF(meta_attribution_app, "") AS meta_attribution_app, - NULLIF(install_source, "") AS install_source, + NULLIF(adjust_creative, "") AS adjust_creative, + NULLIF(adjust_network, "") AS adjust_network, FROM `moz-fx-data-shared-prod.fenix_derived.firefox_android_clients_v1` ) @@ -56,6 +56,10 @@ attribution.play_store_attribution_source, attribution.meta_attribution_app, attribution.install_source, + attribution.adjust_ad_group, + attribution.adjust_campaign, + attribution.adjust_creative, + attribution.adjust_network, -- ping sent retention active_users.retention_seen.day_27.active_on_metric_date AS ping_sent_metric_date, ( @@ -79,10 +83,6 @@ -- Looking back at 27 days to support the official definition of repeat_profile (someone active between days 2 and 28): AND BIT_COUNT(mozfun.bits28.range(active_users.days_active_bits, -26, 27)) > 0 ) AS repeat_profile, - attribution.adjust_ad_group, - attribution.adjust_campaign, - attribution.adjust_creative, - attribution.adjust_network, active_users.days_seen_bits, active_users.days_active_bits, CASE diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/engagement_v1/query.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/engagement_v1/query.sql --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/engagement_v1/query.sql 2024-06-03 15:59:33.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/engagement_v1/query.sql 2024-06-03 16:01:23.000000000 +0000 @@ -8,22 +8,26 @@ country, locale, is_mobile, - adjust_ad_group, - adjust_campaign, - adjust_creative, - adjust_network, play_store_attribution_campaign, play_store_attribution_medium, play_store_attribution_source, meta_attribution_app, install_source, + adjust_ad_group, + adjust_campaign, + adjust_creative, + adjust_network, COUNTIF(is_dau) AS dau, COUNTIF(is_wau) AS wau, - COUNTIF(is_mau) AS mau + COUNTIF(is_mau) AS mau, FROM `moz-fx-data-shared-prod.fenix.engagement_clients` WHERE + {% if is_init() %} + submission_date < CURRENT_DATE + {% else %} submission_date = @submission_date + {% endif %} GROUP BY submission_date, first_seen_date, @@ -33,12 +37,12 @@ country, locale, is_mobile, - adjust_ad_group, - adjust_campaign, - adjust_creative, - adjust_network, play_store_attribution_campaign, play_store_attribution_medium, play_store_attribution_source, meta_attribution_app, - install_source + install_source, + adjust_ad_group, + adjust_campaign, + adjust_creative, + adjust_network diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/engagement_v1/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/engagement_v1/schema.yaml --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/engagement_v1/schema.yaml 2024-06-03 15:59:33.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/engagement_v1/schema.yaml 2024-06-03 16:01:23.000000000 +0000 @@ -39,50 +39,50 @@ mode: NULLABLE description: Indicates if this specific entry is used towards calculating mobile DAU. -- name: adjust_ad_group +- name: play_store_attribution_campaign type: STRING mode: NULLABLE - description: Adjust Ad Group the profile is attributed to. + description: Play store campaign the profile is attributed to. -- name: adjust_campaign +- name: play_store_attribution_medium type: STRING mode: NULLABLE - description: Adjust Campaign the profile is attributed to. + description: Play store Medium the profile is attributed to. -- name: adjust_creative +- name: play_store_attribution_source type: STRING mode: NULLABLE - description: Adjust Creative the profile is attributed to. + description: Play store source the profile is attributed to. -- name: adjust_network +- name: meta_attribution_app type: STRING mode: NULLABLE - description: Adjust Network the profile is attributed to. + description: Facebook app linked to paid marketing. -- name: play_store_attribution_campaign +- name: install_source type: STRING mode: NULLABLE - description: Play store campaign the profile is attributed to. + description: The source of a profile installation. -- name: play_store_attribution_medium +- name: adjust_ad_group type: STRING mode: NULLABLE - description: Play store Medium the profile is attributed to. + description: Adjust Ad Group the profile is attributed to. -- name: play_store_attribution_source +- name: adjust_campaign type: STRING mode: NULLABLE - description: Play store source the profile is attributed to. + description: Adjust Campaign the profile is attributed to. -- name: meta_attribution_app +- name: adjust_creative type: STRING mode: NULLABLE - description: Facebook app linked to paid marketing. + description: Adjust Creative the profile is attributed to. -- name: install_source +- name: adjust_network type: STRING mode: NULLABLE - description: The source of a profile installation. + description: Adjust Network the profile is attributed to. - name: dau type: INTEGER diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/retention_v1/query.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/retention_v1/query.sql --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/retention_v1/query.sql 2024-06-03 15:59:33.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/retention_v1/query.sql 2024-06-03 16:01:23.000000000 +0000 @@ -8,15 +8,15 @@ app_version, locale, is_mobile, - adjust_ad_group, - adjust_campaign, - adjust_creative, - adjust_network, play_store_attribution_campaign, play_store_attribution_medium, play_store_attribution_source, meta_attribution_app, install_source, + adjust_ad_group, + adjust_campaign, + adjust_creative, + adjust_network, COUNTIF(ping_sent_metric_date) AS ping_sent_metric_date, COUNTIF(ping_sent_week_4) AS ping_sent_week_4, COUNTIF(active_metric_date) AS active_metric_date, @@ -27,8 +27,13 @@ FROM `moz-fx-data-shared-prod.fenix.retention_clients` WHERE + {% if is_init() %} + metric_date < DATE_SUB(CURRENT_DATE, INTERVAL 27 DAY) + AND submission_date < CURRENT_DATE + {% else %} metric_date = DATE_SUB(@submission_date, INTERVAL 27 DAY) AND submission_date = @submission_date + {% endif %} GROUP BY metric_date, first_seen_date, @@ -38,12 +43,12 @@ app_version, locale, is_mobile, - adjust_ad_group, - adjust_campaign, - adjust_creative, - adjust_network, play_store_attribution_campaign, play_store_attribution_medium, play_store_attribution_source, meta_attribution_app, - install_source + install_source, + adjust_ad_group, + adjust_campaign, + adjust_creative, + adjust_network diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/retention_v1/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/retention_v1/schema.yaml --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/retention_v1/schema.yaml 2024-06-03 15:59:33.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/retention_v1/schema.yaml 2024-06-03 16:01:23.000000000 +0000 @@ -1,5 +1,4 @@ fields: - - mode: NULLABLE name: metric_date type: DATE @@ -40,50 +39,50 @@ mode: NULLABLE description: Indicates if this specific entry is used towards calculating mobile DAU. -- name: adjust_ad_group +- name: play_store_attribution_campaign type: STRING mode: NULLABLE - description: Adjust Ad Group the profile is attributed to. + description: Play store campaign the profile is attributed to. -- name: adjust_campaign +- name: play_store_attribution_medium type: STRING mode: NULLABLE - description: Adjust Campaign the profile is attributed to. + description: Play store Medium the profile is attributed to. -- name: adjust_creative +- name: play_store_attribution_source type: STRING mode: NULLABLE - description: Adjust Creative the profile is attributed to. + description: Play store source the profile is attributed to. -- name: adjust_network +- name: meta_attribution_app type: STRING mode: NULLABLE - description: Adjust Network the profile is attributed to. + description: Facebook app linked to paid marketing. -- name: play_store_attribution_campaign +- name: install_source type: STRING mode: NULLABLE - description: Play store campaign the profile is attributed to. + description: The source of a profile installation. -- name: play_store_attribution_medium +- name: adjust_ad_group type: STRING mode: NULLABLE - description: Play store Medium the profile is attributed to. + description: Adjust Ad Group the profile is attributed to. -- name: play_store_attribution_source +- name: adjust_campaign type: STRING mode: NULLABLE - description: Play store source the profile is attributed to. + description: Adjust Campaign the profile is attributed to. -- name: meta_attribution_app +- name: adjust_creative type: STRING mode: NULLABLE - description: Facebook app linked to paid marketing. + description: Adjust Creative the profile is attributed to. -- name: install_source +- name: adjust_network type: STRING mode: NULLABLE - description: The source of a profile installation. + description: Adjust Network the profile is attributed to. - name: ping_sent_metric_date type: INTEGER diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_background_tasks_derived/event_monitoring_live_v1/materialized_view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_background_tasks_derived/event_monitoring_live_v1/materialized_view.sql --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_background_tasks_derived/event_monitoring_live_v1/materialized_view.sql 2024-06-03 15:59:33.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_background_tasks_derived/event_monitoring_live_v1/materialized_view.sql 2024-06-03 16:01:24.000000000 +0000 @@ -50,7 +50,7 @@ client_info.app_display_version AS version, ping_info FROM - `moz-fx-data-shared-prod.firefox_desktop_background_tasks_live.events_v1` + `moz-fx-data-shared-prod.firefox_desktop_background_tasks_live.background_tasks_v1` UNION ALL SELECT submission_timestamp, @@ -60,7 +60,7 @@ client_info.app_display_version AS version, ping_info FROM - `moz-fx-data-shared-prod.firefox_desktop_background_tasks_live.background_tasks_v1` + `moz-fx-data-shared-prod.firefox_desktop_background_tasks_live.events_v1` ) CROSS JOIN UNNEST(events) AS event, diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_derived/event_monitoring_live_v1/materialized_view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_derived/event_monitoring_live_v1/materialized_view.sql --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_derived/event_monitoring_live_v1/materialized_view.sql 2024-06-03 15:59:35.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_derived/event_monitoring_live_v1/materialized_view.sql 2024-06-03 16:01:24.000000000 +0000 @@ -60,7 +60,7 @@ client_info.app_display_version AS version, ping_info FROM - `moz-fx-data-shared-prod.firefox_desktop_live.urlbar_potential_exposure_v1` + `moz-fx-data-shared-prod.firefox_desktop_live.events_v1` UNION ALL SELECT submission_timestamp, @@ -70,7 +70,7 @@ client_info.app_display_version AS version, ping_info FROM - `moz-fx-data-shared-prod.firefox_desktop_live.events_v1` + `moz-fx-data-shared-prod.firefox_desktop_live.prototype_no_code_events_v1` UNION ALL SELECT submission_timestamp, @@ -80,7 +80,7 @@ client_info.app_display_version AS version, ping_info FROM - `moz-fx-data-shared-prod.firefox_desktop_live.prototype_no_code_events_v1` + `moz-fx-data-shared-prod.firefox_desktop_live.urlbar_potential_exposure_v1` ) CROSS JOIN UNNEST(events) AS event, diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_ios/engagement_clients/view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios/engagement_clients/view.sql --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_ios/engagement_clients/view.sql 2024-06-03 15:59:34.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios/engagement_clients/view.sql 2024-06-03 16:01:23.000000000 +0000 @@ -26,11 +26,11 @@ client_id, sample_id, channel AS normalized_channel, + is_suspicious_device_client, NULLIF(adjust_ad_group, "") AS adjust_ad_group, + NULLIF(adjust_campaign, "") AS adjust_campaign, NULLIF(adjust_creative, "") AS adjust_creative, NULLIF(adjust_network, "") AS adjust_network, - NULLIF(adjust_campaign, "") AS adjust_campaign, - is_suspicious_device_client FROM `moz-fx-data-shared-prod.firefox_ios_derived.firefox_ios_clients_v1` ) @@ -49,11 +49,11 @@ is_wau, is_mau, is_mobile, + attribution.is_suspicious_device_client, attribution.adjust_ad_group, attribution.adjust_campaign, attribution.adjust_creative, attribution.adjust_network, - attribution.is_suspicious_device_client, CASE WHEN active_users.submission_date = first_seen_date THEN 'new_profile' diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_ios/retention_clients/view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios/retention_clients/view.sql --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_ios/retention_clients/view.sql 2024-06-03 15:59:34.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios/retention_clients/view.sql 2024-06-03 16:01:22.000000000 +0000 @@ -22,11 +22,11 @@ client_id, sample_id, channel AS normalized_channel, + is_suspicious_device_client, NULLIF(adjust_ad_group, "") AS adjust_ad_group, + NULLIF(adjust_campaign, "") AS adjust_campaign, NULLIF(adjust_creative, "") AS adjust_creative, NULLIF(adjust_network, "") AS adjust_network, - NULLIF(adjust_campaign, "") AS adjust_campaign, - is_suspicious_device_client FROM `moz-fx-data-shared-prod.firefox_ios_derived.firefox_ios_clients_v1` ) @@ -44,6 +44,10 @@ clients_daily.isp, active_users.is_mobile, attribution.is_suspicious_device_client, + attribution.adjust_ad_group, + attribution.adjust_campaign, + attribution.adjust_creative, + attribution.adjust_network, -- ping sent retention active_users.retention_seen.day_27.active_on_metric_date AS ping_sent_metric_date, ( @@ -67,10 +71,6 @@ -- Looking back at 27 days to support the official definition of repeat_profile (someone active between days 2 and 28): AND BIT_COUNT(mozfun.bits28.range(active_users.days_active_bits, -26, 27)) > 0 ) AS repeat_profile, - attribution.adjust_ad_group, - attribution.adjust_campaign, - attribution.adjust_creative, - attribution.adjust_network, active_users.days_seen_bits, active_users.days_active_bits, CASE diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/engagement_v1/query.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/engagement_v1/query.sql --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/engagement_v1/query.sql 2024-06-03 15:59:33.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/engagement_v1/query.sql 2024-06-03 16:01:23.000000000 +0000 @@ -8,18 +8,22 @@ country, locale, is_mobile, + is_suspicious_device_client, adjust_ad_group, adjust_campaign, adjust_creative, adjust_network, - is_suspicious_device_client, COUNTIF(is_dau) AS dau, COUNTIF(is_wau) AS wau, - COUNTIF(is_mau) AS mau + COUNTIF(is_mau) AS mau, FROM `moz-fx-data-shared-prod.firefox_ios.engagement_clients` WHERE + {% if is_init() %} + submission_date < CURRENT_DATE + {% else %} submission_date = @submission_date + {% endif %} GROUP BY submission_date, first_seen_date, @@ -29,8 +33,8 @@ country, locale, is_mobile, + is_suspicious_device_client, adjust_ad_group, adjust_campaign, adjust_creative, - adjust_network, - is_suspicious_device_client + adjust_network diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/engagement_v1/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/engagement_v1/schema.yaml --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/engagement_v1/schema.yaml 2024-06-03 15:59:33.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/engagement_v1/schema.yaml 2024-06-03 16:01:23.000000000 +0000 @@ -39,6 +39,11 @@ mode: NULLABLE description: Indicates if this specific entry is used towards calculating mobile DAU. +- name: is_suspicious_device_client + type: BOOLEAN + mode: NULLABLE + description: Flag to identify suspicious device users, see bug-1846554 for more info. + - name: adjust_ad_group type: STRING mode: NULLABLE @@ -59,11 +64,6 @@ mode: NULLABLE description: Adjust Network the profile is attributed to. -- name: is_suspicious_device_client - type: BOOLEAN - mode: NULLABLE - description: Flag to identify suspicious device users, see bug-1846554 for more info. - - name: dau type: INTEGER mode: NULLABLE diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/retention_v1/query.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/retention_v1/query.sql --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/retention_v1/query.sql 2024-06-03 15:59:33.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/retention_v1/query.sql 2024-06-03 16:01:23.000000000 +0000 @@ -8,11 +8,11 @@ app_version, locale, is_mobile, + is_suspicious_device_client, adjust_ad_group, adjust_campaign, adjust_creative, adjust_network, - is_suspicious_device_client, COUNTIF(ping_sent_metric_date) AS ping_sent_metric_date, COUNTIF(ping_sent_week_4) AS ping_sent_week_4, COUNTIF(active_metric_date) AS active_metric_date, @@ -23,8 +23,13 @@ FROM `moz-fx-data-shared-prod.firefox_ios.retention_clients` WHERE + {% if is_init() %} + metric_date < DATE_SUB(CURRENT_DATE, INTERVAL 27 DAY) + AND submission_date < CURRENT_DATE + {% else %} metric_date = DATE_SUB(@submission_date, INTERVAL 27 DAY) AND submission_date = @submission_date + {% endif %} GROUP BY metric_date, first_seen_date, @@ -34,8 +39,8 @@ app_version, locale, is_mobile, + is_suspicious_device_client, adjust_ad_group, adjust_campaign, adjust_creative, - adjust_network, - is_suspicious_device_client + adjust_network diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/retention_v1/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/retention_v1/schema.yaml --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/retention_v1/schema.yaml 2024-06-03 15:59:33.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/retention_v1/schema.yaml 2024-06-03 16:01:23.000000000 +0000 @@ -1,5 +1,4 @@ fields: - - mode: NULLABLE name: metric_date type: DATE @@ -40,6 +39,11 @@ mode: NULLABLE description: Indicates if this specific entry is used towards calculating mobile DAU. +- name: is_suspicious_device_client + type: BOOLEAN + mode: NULLABLE + description: Flag to identify suspicious device users, see bug-1846554 for more info. + - name: adjust_ad_group type: STRING mode: NULLABLE @@ -60,11 +64,6 @@ mode: NULLABLE description: Adjust Network the profile is attributed to. -- name: is_suspicious_device_client - type: BOOLEAN - mode: NULLABLE - description: Flag to identify suspicious device users, see bug-1846554 for more info. - - name: ping_sent_metric_date type: INTEGER mode: NULLABLE diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android/engagement/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android/engagement/metadata.yaml --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android/engagement/metadata.yaml 1970-01-01 00:00:00.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android/engagement/metadata.yaml 2024-06-03 16:11:53.000000000 +0000 @@ -0,0 +1,13 @@ +friendly_name: Engagement +description: |- + Please provide a description for the query +owners: [] +labels: {} +bigquery: null +workgroup_access: +- role: roles/bigquery.dataViewer + members: + - workgroup:mozilla-confidential +references: + view.sql: + - moz-fx-data-shared-prod.focus_android_derived.engagement_v1 diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android/engagement/view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android/engagement/view.sql --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android/engagement/view.sql 1970-01-01 00:00:00.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android/engagement/view.sql 2024-06-03 16:01:23.000000000 +0000 @@ -0,0 +1,19 @@ +-- Query generated via `mobile_kpi_support_metrics` SQL generator. +CREATE OR REPLACE VIEW + `moz-fx-data-shared-prod.focus_android.engagement` +AS +SELECT + *, + CASE + WHEN first_seen_date = submission_date + THEN 'new_profile' + WHEN DATE_DIFF(submission_date, first_seen_date, DAY) + BETWEEN 1 + AND 27 + THEN 'repeat_user' + WHEN DATE_DIFF(submission_date, first_seen_date, DAY) >= 28 + THEN 'existing_user' + ELSE 'Unknown' + END AS lifecycle_stage, +FROM + `moz-fx-data-shared-prod.focus_android_derived.engagement_v1` diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android/engagement_clients/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android/engagement_clients/metadata.yaml --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android/engagement_clients/metadata.yaml 1970-01-01 00:00:00.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android/engagement_clients/metadata.yaml 2024-06-03 16:11:53.000000000 +0000 @@ -0,0 +1,13 @@ +friendly_name: Engagement Clients +description: |- + Please provide a description for the query +owners: [] +labels: {} +bigquery: null +workgroup_access: +- role: roles/bigquery.dataViewer + members: + - workgroup:mozilla-confidential +references: + view.sql: + - moz-fx-data-shared-prod.focus_android.active_users diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android/engagement_clients/view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android/engagement_clients/view.sql --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android/engagement_clients/view.sql 1970-01-01 00:00:00.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android/engagement_clients/view.sql 2024-06-03 16:01:23.000000000 +0000 @@ -0,0 +1,51 @@ +-- Query generated via `mobile_kpi_support_metrics` SQL generator. +CREATE OR REPLACE VIEW + `moz-fx-data-shared-prod.focus_android.engagement_clients` +AS +WITH active_users AS ( + SELECT + submission_date, + client_id, + sample_id, + first_seen_date, + app_name, + normalized_channel, + locale, + country, + isp, + app_display_version, + is_dau, + is_wau, + is_mau, + is_mobile, + FROM + `moz-fx-data-shared-prod.focus_android.active_users` +) +SELECT + submission_date, + client_id, + sample_id, + first_seen_date, + app_name, + normalized_channel, + app_display_version AS app_version, + locale, + country, + isp, + is_dau, + is_wau, + is_mau, + is_mobile, + CASE + WHEN active_users.submission_date = first_seen_date + THEN 'new_profile' + WHEN DATE_DIFF(active_users.submission_date, first_seen_date, DAY) + BETWEEN 1 + AND 27 + THEN 'repeat_user' + WHEN DATE_DIFF(active_users.submission_date, first_seen_date, DAY) >= 28 + THEN 'existing_user' + ELSE 'Unknown' + END AS lifecycle_stage, +FROM + active_users diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android/retention/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android/retention/metadata.yaml --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android/retention/metadata.yaml 1970-01-01 00:00:00.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android/retention/metadata.yaml 2024-06-03 16:11:53.000000000 +0000 @@ -0,0 +1,13 @@ +friendly_name: Retention +description: |- + Please provide a description for the query +owners: [] +labels: {} +bigquery: null +workgroup_access: +- role: roles/bigquery.dataViewer + members: + - workgroup:mozilla-confidential +references: + view.sql: + - moz-fx-data-shared-prod.focus_android_derived.retention_v1 diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android/retention/view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android/retention/view.sql --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android/retention/view.sql 1970-01-01 00:00:00.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android/retention/view.sql 2024-06-03 16:01:23.000000000 +0000 @@ -0,0 +1,19 @@ +-- Query generated via `mobile_kpi_support_metrics` SQL generator. +CREATE OR REPLACE VIEW + `moz-fx-data-shared-prod.focus_android.retention` +AS +SELECT + *, + CASE + WHEN first_seen_date = metric_date + THEN 'new_profile' + WHEN DATE_DIFF(metric_date, first_seen_date, DAY) + BETWEEN 1 + AND 27 + THEN 'repeat_user' + WHEN DATE_DIFF(metric_date, first_seen_date, DAY) >= 28 + THEN 'existing_user' + ELSE 'Unknown' + END AS lifecycle_stage, +FROM + `moz-fx-data-shared-prod.focus_android_derived.retention_v1` diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android/retention_clients/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android/retention_clients/metadata.yaml --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android/retention_clients/metadata.yaml 1970-01-01 00:00:00.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android/retention_clients/metadata.yaml 2024-06-03 16:11:53.000000000 +0000 @@ -0,0 +1,14 @@ +friendly_name: Retention Clients +description: |- + Please provide a description for the query +owners: [] +labels: {} +bigquery: null +workgroup_access: +- role: roles/bigquery.dataViewer + members: + - workgroup:mozilla-confidential +references: + view.sql: + - moz-fx-data-shared-prod.focus_android.active_users + - moz-fx-data-shared-prod.focus_android.baseline_clients_daily diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android/retention_clients/view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android/retention_clients/view.sql --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android/retention_clients/view.sql 1970-01-01 00:00:00.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android/retention_clients/view.sql 2024-06-03 16:01:22.000000000 +0000 @@ -0,0 +1,77 @@ +-- Query generated via `mobile_kpi_support_metrics` SQL generator. +CREATE OR REPLACE VIEW + `moz-fx-data-shared-prod.focus_android.retention_clients` +AS +WITH active_users AS ( + SELECT + submission_date, + client_id, + sample_id, + app_name, + normalized_channel, + mozfun.bits28.retention(days_seen_bits, submission_date) AS retention_seen, + mozfun.bits28.retention(days_active_bits & days_seen_bits, submission_date) AS retention_active, + days_seen_bits, + days_active_bits, + is_mobile, + FROM + `moz-fx-data-shared-prod.focus_android.active_users` +) +SELECT + active_users.submission_date AS submission_date, + clients_daily.submission_date AS metric_date, + clients_daily.first_seen_date, + clients_daily.client_id, + clients_daily.sample_id, + active_users.app_name, + clients_daily.normalized_channel, + clients_daily.country, + clients_daily.app_display_version AS app_version, + clients_daily.locale, + clients_daily.isp, + active_users.is_mobile, + -- ping sent retention + active_users.retention_seen.day_27.active_on_metric_date AS ping_sent_metric_date, + ( + active_users.retention_seen.day_27.active_on_metric_date + AND active_users.retention_seen.day_27.active_in_week_3 + ) AS ping_sent_week_4, + -- activity retention + active_users.retention_active.day_27.active_on_metric_date AS active_metric_date, + ( + active_users.retention_active.day_27.active_on_metric_date + AND active_users.retention_active.day_27.active_in_week_3 + ) AS retained_week_4, + -- new profile retention + clients_daily.is_new_profile AS new_profile_metric_date, + ( + clients_daily.is_new_profile + AND active_users.retention_active.day_27.active_in_week_3 + ) AS retained_week_4_new_profile, + ( + clients_daily.is_new_profile + -- Looking back at 27 days to support the official definition of repeat_profile (someone active between days 2 and 28): + AND BIT_COUNT(mozfun.bits28.range(active_users.days_active_bits, -26, 27)) > 0 + ) AS repeat_profile, + active_users.days_seen_bits, + active_users.days_active_bits, + CASE + WHEN clients_daily.submission_date = first_seen_date + THEN 'new_profile' + WHEN DATE_DIFF(clients_daily.submission_date, first_seen_date, DAY) + BETWEEN 1 + AND 27 + THEN 'repeat_user' + WHEN DATE_DIFF(clients_daily.submission_date, first_seen_date, DAY) >= 28 + THEN 'existing_user' + ELSE 'Unknown' + END AS lifecycle_stage, +FROM + `moz-fx-data-shared-prod.focus_android.baseline_clients_daily` AS clients_daily +INNER JOIN + active_users + ON clients_daily.submission_date = active_users.retention_seen.day_27.metric_date + AND clients_daily.client_id = active_users.client_id + AND clients_daily.normalized_channel = active_users.normalized_channel +WHERE + active_users.retention_seen.day_27.active_on_metric_date diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android_derived/engagement_v1/checks.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android_derived/engagement_v1/checks.sql --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android_derived/engagement_v1/checks.sql 1970-01-01 00:00:00.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android_derived/engagement_v1/checks.sql 2024-06-03 16:01:23.000000000 +0000 @@ -0,0 +1,3 @@ +-- Query generated via `mobile_kpi_support_metrics` SQL generator. +# warn +{{ min_row_count(1), "WHERE submission_date = @submission_date" }} diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android_derived/engagement_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android_derived/engagement_v1/metadata.yaml --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android_derived/engagement_v1/metadata.yaml 1970-01-01 00:00:00.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android_derived/engagement_v1/metadata.yaml 2024-06-03 16:11:47.000000000 +0000 @@ -0,0 +1,29 @@ +friendly_name: Profile / Client Engagement - Focus Android (Aggregated) +description: |- + Profile / Client Engagement (Focus Android) aggregated metrics +owners: +- mozilla/kpi_table_reviewers +- kik@mozilla.com +labels: + schedule: daily + incremental: true + owner1: kik +bigquery: + time_partitioning: + type: day + field: submission_date + require_partition_filter: false + expiration_days: null + range_partitioning: null + clustering: + fields: + - app_name + - country + - first_seen_date +workgroup_access: +- role: roles/bigquery.dataViewer + members: + - workgroup:mozilla-confidential +references: + query.sql: + - moz-fx-data-shared-prod.focus_android.engagement_clients diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android_derived/engagement_v1/query.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android_derived/engagement_v1/query.sql --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android_derived/engagement_v1/query.sql 1970-01-01 00:00:00.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android_derived/engagement_v1/query.sql 2024-06-03 16:01:23.000000000 +0000 @@ -0,0 +1,30 @@ +-- Query generated via `mobile_kpi_support_metrics` SQL generator. +SELECT + submission_date, + first_seen_date, + normalized_channel, + app_name, + app_version, + country, + locale, + is_mobile, + COUNTIF(is_dau) AS dau, + COUNTIF(is_wau) AS wau, + COUNTIF(is_mau) AS mau, +FROM + `moz-fx-data-shared-prod.focus_android.engagement_clients` +WHERE + {% if is_init() %} + submission_date < CURRENT_DATE + {% else %} + submission_date = @submission_date + {% endif %} +GROUP BY + submission_date, + first_seen_date, + normalized_channel, + app_name, + app_version, + country, + locale, + is_mobile diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android_derived/engagement_v1/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android_derived/engagement_v1/schema.yaml --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android_derived/engagement_v1/schema.yaml 1970-01-01 00:00:00.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android_derived/engagement_v1/schema.yaml 2024-06-03 16:01:23.000000000 +0000 @@ -0,0 +1,55 @@ +fields: +- mode: NULLABLE + name: submission_date + type: DATE + description: + +- mode: NULLABLE + name: first_seen_date + type: DATE + description: When the profile has been seen for the first time. + +- mode: NULLABLE + name: normalized_channel + type: STRING + description: Release channel of the app the profile is using. + +- name: app_name + type: STRING + mode: NULLABLE + description: App name the profile is using. + +- name: app_version + type: STRING + mode: NULLABLE + description: Client's app version on the metric date. + +- name: country + type: STRING + mode: NULLABLE + description: Client's country on the metric date. + +- name: locale + type: STRING + mode: NULLABLE + description: Client's locale on the metric date. + +- name: is_mobile + type: BOOLEAN + mode: NULLABLE + description: Indicates if this specific entry is used towards calculating mobile DAU. + +- name: dau + type: INTEGER + mode: NULLABLE + description: DAU - Daily Active Users + +- name: wau + type: INTEGER + mode: NULLABLE + description: WAU - Weekly Active Users + +- name: mau + type: INTEGER + mode: NULLABLE + description: MAU - Monthly Active Users diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android_derived/retention_v1/checks.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android_derived/retention_v1/checks.sql --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android_derived/retention_v1/checks.sql 1970-01-01 00:00:00.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android_derived/retention_v1/checks.sql 2024-06-03 16:01:23.000000000 +0000 @@ -0,0 +1,3 @@ +-- Query generated via `mobile_kpi_support_metrics` SQL generator. +# warn +{{ min_row_count(1), "WHERE metric_date = DATE_SUB(@submission_date, INTERVAL 27 DAY)" }} diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android_derived/retention_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android_derived/retention_v1/metadata.yaml --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android_derived/retention_v1/metadata.yaml 1970-01-01 00:00:00.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android_derived/retention_v1/metadata.yaml 2024-06-03 16:11:47.000000000 +0000 @@ -0,0 +1,39 @@ +friendly_name: Profile / Client Retention - Focus Android (Aggregated) +description: |- + Profile / Client Retention (Focus Android) aggregated metrics + + For all profiles that sent us a ping on the metric date. + Client's attribute values as observed on the metric date. + + see: DENG-3183 for more information. +owners: +- mozilla/kpi_table_reviewers +- kik@mozilla.com +labels: + schedule: daily + incremental: true + owner1: kik +scheduling: + date_partition_parameter: metric_date + date_partition_offset: -27 + parameters: + - submission_date:DATE:{{ds}} +bigquery: + time_partitioning: + type: day + field: metric_date + require_partition_filter: false + expiration_days: null + range_partitioning: null + clustering: + fields: + - app_name + - country + - first_seen_date +workgroup_access: +- role: roles/bigquery.dataViewer + members: + - workgroup:mozilla-confidential +references: + query.sql: + - moz-fx-data-shared-prod.focus_android.retention_clients diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android_derived/retention_v1/query.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android_derived/retention_v1/query.sql --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android_derived/retention_v1/query.sql 1970-01-01 00:00:00.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android_derived/retention_v1/query.sql 2024-06-03 16:01:23.000000000 +0000 @@ -0,0 +1,36 @@ +-- Query generated via `mobile_kpi_support_metrics` SQL generator. +SELECT + metric_date, + first_seen_date, + app_name, + normalized_channel, + country, + app_version, + locale, + is_mobile, + COUNTIF(ping_sent_metric_date) AS ping_sent_metric_date, + COUNTIF(ping_sent_week_4) AS ping_sent_week_4, + COUNTIF(active_metric_date) AS active_metric_date, + COUNTIF(retained_week_4) AS retained_week_4, + COUNTIF(retained_week_4_new_profile) AS retained_week_4_new_profiles, + COUNTIF(new_profile_metric_date) AS new_profiles_metric_date, + COUNTIF(repeat_profile) AS repeat_profiles, +FROM + `moz-fx-data-shared-prod.focus_android.retention_clients` +WHERE + {% if is_init() %} + metric_date < DATE_SUB(CURRENT_DATE, INTERVAL 27 DAY) + AND submission_date < CURRENT_DATE + {% else %} + metric_date = DATE_SUB(@submission_date, INTERVAL 27 DAY) + AND submission_date = @submission_date + {% endif %} +GROUP BY + metric_date, + first_seen_date, + app_name, + normalized_channel, + country, + app_version, + locale, + is_mobile diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android_derived/retention_v1/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android_derived/retention_v1/schema.yaml --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android_derived/retention_v1/schema.yaml 1970-01-01 00:00:00.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android_derived/retention_v1/schema.yaml 2024-06-03 16:01:23.000000000 +0000 @@ -0,0 +1,75 @@ +fields: +- mode: NULLABLE + name: metric_date + type: DATE + description: This is 28 days before processing. + +- mode: NULLABLE + name: first_seen_date + type: DATE + description: When the profile has been seen for the first time. + +- name: app_name + type: STRING + mode: NULLABLE + description: App name the profile is using. + +- mode: NULLABLE + name: normalized_channel + type: STRING + description: Release channel of the app the profile is using. + +- name: country + type: STRING + mode: NULLABLE + description: Client's country on the metric date. + +- name: app_version + type: STRING + mode: NULLABLE + description: Client's app version on the metric date. + +- name: locale + type: STRING + mode: NULLABLE + description: Client's locale on the metric date. + +- name: is_mobile + type: BOOLEAN + mode: NULLABLE + description: Indicates if this specific entry is used towards calculating mobile DAU. + +- name: ping_sent_metric_date + type: INTEGER + mode: NULLABLE + description: Number of profiles that sent a ping on the metric date. + +- name: ping_sent_week_4 + type: INTEGER + mode: NULLABLE + description: Number of profiles that sent a ping between days 22 and 28 after the metric date. + +- name: active_metric_date + type: INTEGER + mode: NULLABLE + description: Number of DAU profiles on the metric date. + +- name: retained_week_4 + type: INTEGER + mode: NULLABLE + description: Number of profiles that were DAU on the metric date and between days 22 and 28 after the metric date. + +- name: retained_week_4_new_profiles + type: INTEGER + mode: NULLABLE + description: Number of new profiles on the metric date that were DAU between days 22 and 28 after the metric date. + +- name: new_profiles_metric_date + type: INTEGER + mode: NULLABLE + description: Number of new profiles on the metric date. + +- name: repeat_profiles + type: INTEGER + mode: NULLABLE + description: Number of new profiles on the metric date that were DAU at least twice in the next 28 days. diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_ios/engagement/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_ios/engagement/metadata.yaml --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_ios/engagement/metadata.yaml 1970-01-01 00:00:00.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_ios/engagement/metadata.yaml 2024-06-03 16:11:53.000000000 +0000 @@ -0,0 +1,13 @@ +friendly_name: Engagement +description: |- + Please provide a description for the query +owners: [] +labels: {} +bigquery: null +workgroup_access: +- role: roles/bigquery.dataViewer + members: + - workgroup:mozilla-confidential +references: + view.sql: + - moz-fx-data-shared-prod.focus_ios_derived.engagement_v1 diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_ios/engagement/view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_ios/engagement/view.sql --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_ios/engagement/view.sql 1970-01-01 00:00:00.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_ios/engagement/view.sql 2024-06-03 16:01:23.000000000 +0000 @@ -0,0 +1,19 @@ +-- Query generated via `mobile_kpi_support_metrics` SQL generator. +CREATE OR REPLACE VIEW + `moz-fx-data-shared-prod.focus_ios.engagement` +AS +SELECT + *, + CASE + WHEN first_seen_date = submission_date + THEN 'new_profile' + WHEN DATE_DIFF(submission_date, first_seen_date, DAY) + BETWEEN 1 + AND 27 + THEN 'repeat_user' + WHEN DATE_DIFF(submission_date, first_seen_date, DAY) >= 28 + THEN 'existing_user' + ELSE 'Unknown' + END AS lifecycle_stage, +FROM + `moz-fx-data-shared-prod.focus_ios_derived.engagement_v1` diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_ios/engagement_clients/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_ios/engagement_clients/metadata.yaml --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_ios/engagement_clients/metadata.yaml 1970-01-01 00:00:00.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_ios/engagement_clients/metadata.yaml 2024-06-03 16:11:53.000000000 +0000 @@ -0,0 +1,13 @@ +friendly_name: Engagement Clients +description: |- + Please provide a description for the query +owners: [] +labels: {} +bigquery: null +workgroup_access: +- role: roles/bigquery.dataViewer + members: + - workgroup:mozilla-confidential +references: + view.sql: + - moz-fx-data-shared-prod.focus_ios.active_users diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_ios/engagement_clients/view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_ios/engagement_clients/view.sql --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_ios/engagement_clients/view.sql 1970-01-01 00:00:00.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_ios/engagement_clients/view.sql 2024-06-03 16:01:23.000000000 +0000 @@ -0,0 +1,51 @@ +-- Query generated via `mobile_kpi_support_metrics` SQL generator. +CREATE OR REPLACE VIEW + `moz-fx-data-shared-prod.focus_ios.engagement_clients` +AS +WITH active_users AS ( + SELECT + submission_date, + client_id, + sample_id, + first_seen_date, + app_name, + normalized_channel, + locale, + country, + isp, + app_display_version, + is_dau, + is_wau, + is_mau, + is_mobile, + FROM + `moz-fx-data-shared-prod.focus_ios.active_users` +) +SELECT + submission_date, + client_id, + sample_id, + first_seen_date, + app_name, + normalized_channel, + app_display_version AS app_version, + locale, + country, + isp, + is_dau, + is_wau, + is_mau, + is_mobile, + CASE + WHEN active_users.submission_date = first_seen_date + THEN 'new_profile' + WHEN DATE_DIFF(active_users.submission_date, first_seen_date, DAY) + BETWEEN 1 + AND 27 + THEN 'repeat_user' + WHEN DATE_DIFF(active_users.submission_date, first_seen_date, DAY) >= 28 + THEN 'existing_user' + ELSE 'Unknown' + END AS lifecycle_stage, +FROM + active_users diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_ios/retention/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_ios/retention/metadata.yaml --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_ios/retention/metadata.yaml 1970-01-01 00:00:00.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_ios/retention/metadata.yaml 2024-06-03 16:11:53.000000000 +0000 @@ -0,0 +1,13 @@ +friendly_name: Retention +description: |- + Please provide a description for the query +owners: [] +labels: {} +bigquery: null +workgroup_access: +- role: roles/bigquery.dataViewer + members: + - workgroup:mozilla-confidential +references: + view.sql: + - moz-fx-data-shared-prod.focus_ios_derived.retention_v1 diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_ios/retention/view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_ios/retention/view.sql --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_ios/retention/view.sql 1970-01-01 00:00:00.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_ios/retention/view.sql 2024-06-03 16:01:23.000000000 +0000 @@ -0,0 +1,19 @@ +-- Query generated via `mobile_kpi_support_metrics` SQL generator. +CREATE OR REPLACE VIEW + `moz-fx-data-shared-prod.focus_ios.retention` +AS +SELECT + *, + CASE + WHEN first_seen_date = metric_date + THEN 'new_profile' + WHEN DATE_DIFF(metric_date, first_seen_date, DAY) + BETWEEN 1 + AND 27 + THEN 'repeat_user' + WHEN DATE_DIFF(metric_date, first_seen_date, DAY) >= 28 + THEN 'existing_user' + ELSE 'Unknown' + END AS lifecycle_stage, +FROM + `moz-fx-data-shared-prod.focus_ios_derived.retention_v1` diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_ios/retention_clients/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_ios/retention_clients/metadata.yaml --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_ios/retention_clients/metadata.yaml 1970-01-01 00:00:00.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_ios/retention_clients/metadata.yaml 2024-06-03 16:11:53.000000000 +0000 @@ -0,0 +1,14 @@ +friendly_name: Retention Clients +description: |- + Please provide a description for the query +owners: [] +labels: {} +bigquery: null +workgroup_access: +- role: roles/bigquery.dataViewer + members: + - workgroup:mozilla-confidential +references: + view.sql: + - moz-fx-data-shared-prod.focus_ios.active_users + - moz-fx-data-shared-prod.focus_ios.baseline_clients_daily diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_ios/retention_clients/view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_ios/retention_clients/view.sql --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_ios/retention_clients/view.sql 1970-01-01 00:00:00.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_ios/retention_clients/view.sql 2024-06-03 16:01:22.000000000 +0000 @@ -0,0 +1,77 @@ +-- Query generated via `mobile_kpi_support_metrics` SQL generator. +CREATE OR REPLACE VIEW + `moz-fx-data-shared-prod.focus_ios.retention_clients` +AS +WITH active_users AS ( + SELECT + submission_date, + client_id, + sample_id, + app_name, + normalized_channel, + mozfun.bits28.retention(days_seen_bits, submission_date) AS retention_seen, + mozfun.bits28.retention(days_active_bits & days_seen_bits, submission_date) AS retention_active, + days_seen_bits, + days_active_bits, + is_mobile, + FROM + `moz-fx-data-shared-prod.focus_ios.active_users` +) +SELECT + active_users.submission_date AS submission_date, + clients_daily.submission_date AS metric_date, + clients_daily.first_seen_date, + clients_daily.client_id, + clients_daily.sample_id, + active_users.app_name, + clients_daily.normalized_channel, + clients_daily.country, + clients_daily.app_display_versi ```

⚠️ Only part of the diff is displayed.

Link to full diff

dataops-ci-bot commented 1 month ago

Integration report for "fix: trailing comma in queries"

sql.diff

Click to expand! ```diff Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android: engagement Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android: engagement_clients Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android: retention Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android: retention_clients Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android_derived: engagement_v1 Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android_derived: retention_v1 Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_ios: engagement Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_ios: engagement_clients Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_ios: retention Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_ios: retention_clients Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_ios_derived: engagement_v1 Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_ios_derived: retention_v1 Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/klar_android: engagement Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/klar_android: engagement_clients Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/klar_android: retention Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/klar_android: retention_clients Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/klar_android_derived: engagement_v1 Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/klar_android_derived: retention_v1 Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/klar_ios: engagement Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/klar_ios: engagement_clients Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/klar_ios: retention Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/klar_ios: retention_clients Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/klar_ios_derived: engagement_v1 Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/klar_ios_derived: retention_v1 diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/bedrock_derived/event_monitoring_live_v1/materialized_view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/bedrock_derived/event_monitoring_live_v1/materialized_view.sql --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/bedrock_derived/event_monitoring_live_v1/materialized_view.sql 2024-06-03 16:30:51.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/bedrock_derived/event_monitoring_live_v1/materialized_view.sql 2024-06-03 16:20:46.000000000 +0000 @@ -60,7 +60,7 @@ client_info.app_display_version AS version, ping_info FROM - `moz-fx-data-shared-prod.bedrock_live.events_v1` + `moz-fx-data-shared-prod.bedrock_live.interaction_v1` UNION ALL SELECT submission_timestamp, @@ -70,7 +70,7 @@ client_info.app_display_version AS version, ping_info FROM - `moz-fx-data-shared-prod.bedrock_live.interaction_v1` + `moz-fx-data-shared-prod.bedrock_live.events_v1` ) CROSS JOIN UNNEST(events) AS event, diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/engagement_clients/view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/engagement_clients/view.sql --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/engagement_clients/view.sql 2024-06-03 16:30:51.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/engagement_clients/view.sql 2024-06-03 16:20:45.000000000 +0000 @@ -26,19 +26,19 @@ client_id, sample_id, channel AS normalized_channel, + NULLIF(play_store_attribution_campaign, "") AS play_store_attribution_campaign, + NULLIF(play_store_attribution_medium, "") AS play_store_attribution_medium, + NULLIF(play_store_attribution_source, "") AS play_store_attribution_source, + NULLIF(meta_attribution_app, "") AS meta_attribution_app, + NULLIF(install_source, "") AS install_source, NULLIF(adjust_ad_group, "") AS adjust_ad_group, - NULLIF(adjust_creative, "") AS adjust_creative, - NULLIF(adjust_network, "") AS adjust_network, CASE WHEN adjust_network IN ('Google Organic Search', 'Organic') THEN 'Organic' ELSE NULLIF(adjust_campaign, "") END AS adjust_campaign, - NULLIF(play_store_attribution_campaign, "") AS play_store_attribution_campaign, - NULLIF(play_store_attribution_medium, "") AS play_store_attribution_medium, - NULLIF(play_store_attribution_source, "") AS play_store_attribution_source, - NULLIF(meta_attribution_app, "") AS meta_attribution_app, - NULLIF(install_source, "") AS install_source, + NULLIF(adjust_creative, "") AS adjust_creative, + NULLIF(adjust_network, "") AS adjust_network, FROM `moz-fx-data-shared-prod.fenix_derived.firefox_android_clients_v1` ) @@ -57,15 +57,15 @@ is_wau, is_mau, is_mobile, - attribution.adjust_ad_group, - attribution.adjust_campaign, - attribution.adjust_creative, - attribution.adjust_network, attribution.play_store_attribution_campaign, attribution.play_store_attribution_medium, attribution.play_store_attribution_source, attribution.meta_attribution_app, attribution.install_source, + attribution.adjust_ad_group, + attribution.adjust_campaign, + attribution.adjust_creative, + attribution.adjust_network, CASE WHEN active_users.submission_date = first_seen_date THEN 'new_profile' diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/retention_clients/view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/retention_clients/view.sql --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/retention_clients/view.sql 2024-06-03 16:30:51.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/retention_clients/view.sql 2024-06-03 16:20:45.000000000 +0000 @@ -22,19 +22,19 @@ client_id, sample_id, channel AS normalized_channel, + NULLIF(play_store_attribution_campaign, "") AS play_store_attribution_campaign, + NULLIF(play_store_attribution_medium, "") AS play_store_attribution_medium, + NULLIF(play_store_attribution_source, "") AS play_store_attribution_source, + NULLIF(meta_attribution_app, "") AS meta_attribution_app, + NULLIF(install_source, "") AS install_source, NULLIF(adjust_ad_group, "") AS adjust_ad_group, - NULLIF(adjust_creative, "") AS adjust_creative, - NULLIF(adjust_network, "") AS adjust_network, CASE WHEN adjust_network IN ('Google Organic Search', 'Organic') THEN 'Organic' ELSE NULLIF(adjust_campaign, "") END AS adjust_campaign, - NULLIF(play_store_attribution_campaign, "") AS play_store_attribution_campaign, - NULLIF(play_store_attribution_medium, "") AS play_store_attribution_medium, - NULLIF(play_store_attribution_source, "") AS play_store_attribution_source, - NULLIF(meta_attribution_app, "") AS meta_attribution_app, - NULLIF(install_source, "") AS install_source, + NULLIF(adjust_creative, "") AS adjust_creative, + NULLIF(adjust_network, "") AS adjust_network, FROM `moz-fx-data-shared-prod.fenix_derived.firefox_android_clients_v1` ) @@ -56,6 +56,10 @@ attribution.play_store_attribution_source, attribution.meta_attribution_app, attribution.install_source, + attribution.adjust_ad_group, + attribution.adjust_campaign, + attribution.adjust_creative, + attribution.adjust_network, -- ping sent retention active_users.retention_seen.day_27.active_on_metric_date AS ping_sent_metric_date, ( @@ -79,10 +83,6 @@ -- Looking back at 27 days to support the official definition of repeat_profile (someone active between days 2 and 28): AND BIT_COUNT(mozfun.bits28.range(active_users.days_active_bits, -26, 27)) > 0 ) AS repeat_profile, - attribution.adjust_ad_group, - attribution.adjust_campaign, - attribution.adjust_creative, - attribution.adjust_network, active_users.days_seen_bits, active_users.days_active_bits, CASE diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/engagement_v1/query.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/engagement_v1/query.sql --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/engagement_v1/query.sql 2024-06-03 16:30:51.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/engagement_v1/query.sql 2024-06-03 16:20:45.000000000 +0000 @@ -8,22 +8,26 @@ country, locale, is_mobile, - adjust_ad_group, - adjust_campaign, - adjust_creative, - adjust_network, play_store_attribution_campaign, play_store_attribution_medium, play_store_attribution_source, meta_attribution_app, install_source, + adjust_ad_group, + adjust_campaign, + adjust_creative, + adjust_network, COUNTIF(is_dau) AS dau, COUNTIF(is_wau) AS wau, - COUNTIF(is_mau) AS mau + COUNTIF(is_mau) AS mau, FROM `moz-fx-data-shared-prod.fenix.engagement_clients` WHERE + {% if is_init() %} + submission_date < CURRENT_DATE + {% else %} submission_date = @submission_date + {% endif %} GROUP BY submission_date, first_seen_date, @@ -33,12 +37,12 @@ country, locale, is_mobile, - adjust_ad_group, - adjust_campaign, - adjust_creative, - adjust_network, play_store_attribution_campaign, play_store_attribution_medium, play_store_attribution_source, meta_attribution_app, - install_source + install_source, + adjust_ad_group, + adjust_campaign, + adjust_creative, + adjust_network diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/engagement_v1/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/engagement_v1/schema.yaml --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/engagement_v1/schema.yaml 2024-06-03 16:30:51.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/engagement_v1/schema.yaml 2024-06-03 16:20:45.000000000 +0000 @@ -39,50 +39,50 @@ mode: NULLABLE description: Indicates if this specific entry is used towards calculating mobile DAU. -- name: adjust_ad_group +- name: play_store_attribution_campaign type: STRING mode: NULLABLE - description: Adjust Ad Group the profile is attributed to. + description: Play store campaign the profile is attributed to. -- name: adjust_campaign +- name: play_store_attribution_medium type: STRING mode: NULLABLE - description: Adjust Campaign the profile is attributed to. + description: Play store Medium the profile is attributed to. -- name: adjust_creative +- name: play_store_attribution_source type: STRING mode: NULLABLE - description: Adjust Creative the profile is attributed to. + description: Play store source the profile is attributed to. -- name: adjust_network +- name: meta_attribution_app type: STRING mode: NULLABLE - description: Adjust Network the profile is attributed to. + description: Facebook app linked to paid marketing. -- name: play_store_attribution_campaign +- name: install_source type: STRING mode: NULLABLE - description: Play store campaign the profile is attributed to. + description: The source of a profile installation. -- name: play_store_attribution_medium +- name: adjust_ad_group type: STRING mode: NULLABLE - description: Play store Medium the profile is attributed to. + description: Adjust Ad Group the profile is attributed to. -- name: play_store_attribution_source +- name: adjust_campaign type: STRING mode: NULLABLE - description: Play store source the profile is attributed to. + description: Adjust Campaign the profile is attributed to. -- name: meta_attribution_app +- name: adjust_creative type: STRING mode: NULLABLE - description: Facebook app linked to paid marketing. + description: Adjust Creative the profile is attributed to. -- name: install_source +- name: adjust_network type: STRING mode: NULLABLE - description: The source of a profile installation. + description: Adjust Network the profile is attributed to. - name: dau type: INTEGER diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/retention_v1/query.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/retention_v1/query.sql --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/retention_v1/query.sql 2024-06-03 16:30:51.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/retention_v1/query.sql 2024-06-03 16:20:45.000000000 +0000 @@ -8,15 +8,15 @@ app_version, locale, is_mobile, - adjust_ad_group, - adjust_campaign, - adjust_creative, - adjust_network, play_store_attribution_campaign, play_store_attribution_medium, play_store_attribution_source, meta_attribution_app, install_source, + adjust_ad_group, + adjust_campaign, + adjust_creative, + adjust_network, COUNTIF(ping_sent_metric_date) AS ping_sent_metric_date, COUNTIF(ping_sent_week_4) AS ping_sent_week_4, COUNTIF(active_metric_date) AS active_metric_date, @@ -27,8 +27,13 @@ FROM `moz-fx-data-shared-prod.fenix.retention_clients` WHERE + {% if is_init() %} + metric_date < DATE_SUB(CURRENT_DATE, INTERVAL 27 DAY) + AND submission_date < CURRENT_DATE + {% else %} metric_date = DATE_SUB(@submission_date, INTERVAL 27 DAY) AND submission_date = @submission_date + {% endif %} GROUP BY metric_date, first_seen_date, @@ -38,12 +43,12 @@ app_version, locale, is_mobile, - adjust_ad_group, - adjust_campaign, - adjust_creative, - adjust_network, play_store_attribution_campaign, play_store_attribution_medium, play_store_attribution_source, meta_attribution_app, - install_source + install_source, + adjust_ad_group, + adjust_campaign, + adjust_creative, + adjust_network diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/retention_v1/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/retention_v1/schema.yaml --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/retention_v1/schema.yaml 2024-06-03 16:30:51.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/retention_v1/schema.yaml 2024-06-03 16:20:45.000000000 +0000 @@ -1,5 +1,4 @@ fields: - - mode: NULLABLE name: metric_date type: DATE @@ -40,50 +39,50 @@ mode: NULLABLE description: Indicates if this specific entry is used towards calculating mobile DAU. -- name: adjust_ad_group +- name: play_store_attribution_campaign type: STRING mode: NULLABLE - description: Adjust Ad Group the profile is attributed to. + description: Play store campaign the profile is attributed to. -- name: adjust_campaign +- name: play_store_attribution_medium type: STRING mode: NULLABLE - description: Adjust Campaign the profile is attributed to. + description: Play store Medium the profile is attributed to. -- name: adjust_creative +- name: play_store_attribution_source type: STRING mode: NULLABLE - description: Adjust Creative the profile is attributed to. + description: Play store source the profile is attributed to. -- name: adjust_network +- name: meta_attribution_app type: STRING mode: NULLABLE - description: Adjust Network the profile is attributed to. + description: Facebook app linked to paid marketing. -- name: play_store_attribution_campaign +- name: install_source type: STRING mode: NULLABLE - description: Play store campaign the profile is attributed to. + description: The source of a profile installation. -- name: play_store_attribution_medium +- name: adjust_ad_group type: STRING mode: NULLABLE - description: Play store Medium the profile is attributed to. + description: Adjust Ad Group the profile is attributed to. -- name: play_store_attribution_source +- name: adjust_campaign type: STRING mode: NULLABLE - description: Play store source the profile is attributed to. + description: Adjust Campaign the profile is attributed to. -- name: meta_attribution_app +- name: adjust_creative type: STRING mode: NULLABLE - description: Facebook app linked to paid marketing. + description: Adjust Creative the profile is attributed to. -- name: install_source +- name: adjust_network type: STRING mode: NULLABLE - description: The source of a profile installation. + description: Adjust Network the profile is attributed to. - name: ping_sent_metric_date type: INTEGER diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_derived/event_monitoring_live_v1/materialized_view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_derived/event_monitoring_live_v1/materialized_view.sql --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_derived/event_monitoring_live_v1/materialized_view.sql 2024-06-03 16:30:51.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_derived/event_monitoring_live_v1/materialized_view.sql 2024-06-03 16:20:47.000000000 +0000 @@ -50,7 +50,7 @@ client_info.app_display_version AS version, ping_info FROM - `moz-fx-data-shared-prod.firefox_desktop_live.events_v1` + `moz-fx-data-shared-prod.firefox_desktop_live.urlbar_potential_exposure_v1` UNION ALL SELECT submission_timestamp, @@ -60,7 +60,7 @@ client_info.app_display_version AS version, ping_info FROM - `moz-fx-data-shared-prod.firefox_desktop_live.prototype_no_code_events_v1` + `moz-fx-data-shared-prod.firefox_desktop_live.events_v1` UNION ALL SELECT submission_timestamp, @@ -80,7 +80,7 @@ client_info.app_display_version AS version, ping_info FROM - `moz-fx-data-shared-prod.firefox_desktop_live.urlbar_potential_exposure_v1` + `moz-fx-data-shared-prod.firefox_desktop_live.prototype_no_code_events_v1` ) CROSS JOIN UNNEST(events) AS event, diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_ios/engagement_clients/view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios/engagement_clients/view.sql --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_ios/engagement_clients/view.sql 2024-06-03 16:30:51.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios/engagement_clients/view.sql 2024-06-03 16:20:45.000000000 +0000 @@ -26,11 +26,11 @@ client_id, sample_id, channel AS normalized_channel, + is_suspicious_device_client, NULLIF(adjust_ad_group, "") AS adjust_ad_group, + NULLIF(adjust_campaign, "") AS adjust_campaign, NULLIF(adjust_creative, "") AS adjust_creative, NULLIF(adjust_network, "") AS adjust_network, - NULLIF(adjust_campaign, "") AS adjust_campaign, - is_suspicious_device_client FROM `moz-fx-data-shared-prod.firefox_ios_derived.firefox_ios_clients_v1` ) @@ -49,11 +49,11 @@ is_wau, is_mau, is_mobile, + attribution.is_suspicious_device_client, attribution.adjust_ad_group, attribution.adjust_campaign, attribution.adjust_creative, attribution.adjust_network, - attribution.is_suspicious_device_client, CASE WHEN active_users.submission_date = first_seen_date THEN 'new_profile' diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_ios/retention_clients/view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios/retention_clients/view.sql --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_ios/retention_clients/view.sql 2024-06-03 16:30:51.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios/retention_clients/view.sql 2024-06-03 16:20:45.000000000 +0000 @@ -22,11 +22,11 @@ client_id, sample_id, channel AS normalized_channel, + is_suspicious_device_client, NULLIF(adjust_ad_group, "") AS adjust_ad_group, + NULLIF(adjust_campaign, "") AS adjust_campaign, NULLIF(adjust_creative, "") AS adjust_creative, NULLIF(adjust_network, "") AS adjust_network, - NULLIF(adjust_campaign, "") AS adjust_campaign, - is_suspicious_device_client FROM `moz-fx-data-shared-prod.firefox_ios_derived.firefox_ios_clients_v1` ) @@ -44,6 +44,10 @@ clients_daily.isp, active_users.is_mobile, attribution.is_suspicious_device_client, + attribution.adjust_ad_group, + attribution.adjust_campaign, + attribution.adjust_creative, + attribution.adjust_network, -- ping sent retention active_users.retention_seen.day_27.active_on_metric_date AS ping_sent_metric_date, ( @@ -67,10 +71,6 @@ -- Looking back at 27 days to support the official definition of repeat_profile (someone active between days 2 and 28): AND BIT_COUNT(mozfun.bits28.range(active_users.days_active_bits, -26, 27)) > 0 ) AS repeat_profile, - attribution.adjust_ad_group, - attribution.adjust_campaign, - attribution.adjust_creative, - attribution.adjust_network, active_users.days_seen_bits, active_users.days_active_bits, CASE diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/engagement_v1/query.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/engagement_v1/query.sql --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/engagement_v1/query.sql 2024-06-03 16:30:51.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/engagement_v1/query.sql 2024-06-03 16:20:45.000000000 +0000 @@ -8,18 +8,22 @@ country, locale, is_mobile, + is_suspicious_device_client, adjust_ad_group, adjust_campaign, adjust_creative, adjust_network, - is_suspicious_device_client, COUNTIF(is_dau) AS dau, COUNTIF(is_wau) AS wau, - COUNTIF(is_mau) AS mau + COUNTIF(is_mau) AS mau, FROM `moz-fx-data-shared-prod.firefox_ios.engagement_clients` WHERE + {% if is_init() %} + submission_date < CURRENT_DATE + {% else %} submission_date = @submission_date + {% endif %} GROUP BY submission_date, first_seen_date, @@ -29,8 +33,8 @@ country, locale, is_mobile, + is_suspicious_device_client, adjust_ad_group, adjust_campaign, adjust_creative, - adjust_network, - is_suspicious_device_client + adjust_network diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/engagement_v1/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/engagement_v1/schema.yaml --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/engagement_v1/schema.yaml 2024-06-03 16:30:51.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/engagement_v1/schema.yaml 2024-06-03 16:20:45.000000000 +0000 @@ -39,6 +39,11 @@ mode: NULLABLE description: Indicates if this specific entry is used towards calculating mobile DAU. +- name: is_suspicious_device_client + type: BOOLEAN + mode: NULLABLE + description: Flag to identify suspicious device users, see bug-1846554 for more info. + - name: adjust_ad_group type: STRING mode: NULLABLE @@ -59,11 +64,6 @@ mode: NULLABLE description: Adjust Network the profile is attributed to. -- name: is_suspicious_device_client - type: BOOLEAN - mode: NULLABLE - description: Flag to identify suspicious device users, see bug-1846554 for more info. - - name: dau type: INTEGER mode: NULLABLE diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/retention_v1/query.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/retention_v1/query.sql --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/retention_v1/query.sql 2024-06-03 16:30:51.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/retention_v1/query.sql 2024-06-03 16:20:45.000000000 +0000 @@ -8,11 +8,11 @@ app_version, locale, is_mobile, + is_suspicious_device_client, adjust_ad_group, adjust_campaign, adjust_creative, adjust_network, - is_suspicious_device_client, COUNTIF(ping_sent_metric_date) AS ping_sent_metric_date, COUNTIF(ping_sent_week_4) AS ping_sent_week_4, COUNTIF(active_metric_date) AS active_metric_date, @@ -23,8 +23,13 @@ FROM `moz-fx-data-shared-prod.firefox_ios.retention_clients` WHERE + {% if is_init() %} + metric_date < DATE_SUB(CURRENT_DATE, INTERVAL 27 DAY) + AND submission_date < CURRENT_DATE + {% else %} metric_date = DATE_SUB(@submission_date, INTERVAL 27 DAY) AND submission_date = @submission_date + {% endif %} GROUP BY metric_date, first_seen_date, @@ -34,8 +39,8 @@ app_version, locale, is_mobile, + is_suspicious_device_client, adjust_ad_group, adjust_campaign, adjust_creative, - adjust_network, - is_suspicious_device_client + adjust_network diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/retention_v1/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/retention_v1/schema.yaml --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/retention_v1/schema.yaml 2024-06-03 16:30:51.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/retention_v1/schema.yaml 2024-06-03 16:20:45.000000000 +0000 @@ -1,5 +1,4 @@ fields: - - mode: NULLABLE name: metric_date type: DATE @@ -40,6 +39,11 @@ mode: NULLABLE description: Indicates if this specific entry is used towards calculating mobile DAU. +- name: is_suspicious_device_client + type: BOOLEAN + mode: NULLABLE + description: Flag to identify suspicious device users, see bug-1846554 for more info. + - name: adjust_ad_group type: STRING mode: NULLABLE @@ -60,11 +64,6 @@ mode: NULLABLE description: Adjust Network the profile is attributed to. -- name: is_suspicious_device_client - type: BOOLEAN - mode: NULLABLE - description: Flag to identify suspicious device users, see bug-1846554 for more info. - - name: ping_sent_metric_date type: INTEGER mode: NULLABLE diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android/engagement/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android/engagement/metadata.yaml --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android/engagement/metadata.yaml 1970-01-01 00:00:00.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android/engagement/metadata.yaml 2024-06-03 16:31:19.000000000 +0000 @@ -0,0 +1,13 @@ +friendly_name: Engagement +description: |- + Please provide a description for the query +owners: [] +labels: {} +bigquery: null +workgroup_access: +- role: roles/bigquery.dataViewer + members: + - workgroup:mozilla-confidential +references: + view.sql: + - moz-fx-data-shared-prod.focus_android_derived.engagement_v1 diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android/engagement/view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android/engagement/view.sql --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android/engagement/view.sql 1970-01-01 00:00:00.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android/engagement/view.sql 2024-06-03 16:20:45.000000000 +0000 @@ -0,0 +1,19 @@ +-- Query generated via `mobile_kpi_support_metrics` SQL generator. +CREATE OR REPLACE VIEW + `moz-fx-data-shared-prod.focus_android.engagement` +AS +SELECT + *, + CASE + WHEN first_seen_date = submission_date + THEN 'new_profile' + WHEN DATE_DIFF(submission_date, first_seen_date, DAY) + BETWEEN 1 + AND 27 + THEN 'repeat_user' + WHEN DATE_DIFF(submission_date, first_seen_date, DAY) >= 28 + THEN 'existing_user' + ELSE 'Unknown' + END AS lifecycle_stage, +FROM + `moz-fx-data-shared-prod.focus_android_derived.engagement_v1` diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android/engagement_clients/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android/engagement_clients/metadata.yaml --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android/engagement_clients/metadata.yaml 1970-01-01 00:00:00.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android/engagement_clients/metadata.yaml 2024-06-03 16:31:19.000000000 +0000 @@ -0,0 +1,13 @@ +friendly_name: Engagement Clients +description: |- + Please provide a description for the query +owners: [] +labels: {} +bigquery: null +workgroup_access: +- role: roles/bigquery.dataViewer + members: + - workgroup:mozilla-confidential +references: + view.sql: + - moz-fx-data-shared-prod.focus_android.active_users diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android/engagement_clients/view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android/engagement_clients/view.sql --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android/engagement_clients/view.sql 1970-01-01 00:00:00.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android/engagement_clients/view.sql 2024-06-03 16:20:45.000000000 +0000 @@ -0,0 +1,51 @@ +-- Query generated via `mobile_kpi_support_metrics` SQL generator. +CREATE OR REPLACE VIEW + `moz-fx-data-shared-prod.focus_android.engagement_clients` +AS +WITH active_users AS ( + SELECT + submission_date, + client_id, + sample_id, + first_seen_date, + app_name, + normalized_channel, + locale, + country, + isp, + app_display_version, + is_dau, + is_wau, + is_mau, + is_mobile, + FROM + `moz-fx-data-shared-prod.focus_android.active_users` +) +SELECT + submission_date, + client_id, + sample_id, + first_seen_date, + app_name, + normalized_channel, + app_display_version AS app_version, + locale, + country, + isp, + is_dau, + is_wau, + is_mau, + is_mobile, + CASE + WHEN active_users.submission_date = first_seen_date + THEN 'new_profile' + WHEN DATE_DIFF(active_users.submission_date, first_seen_date, DAY) + BETWEEN 1 + AND 27 + THEN 'repeat_user' + WHEN DATE_DIFF(active_users.submission_date, first_seen_date, DAY) >= 28 + THEN 'existing_user' + ELSE 'Unknown' + END AS lifecycle_stage, +FROM + active_users diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android/retention/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android/retention/metadata.yaml --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android/retention/metadata.yaml 1970-01-01 00:00:00.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android/retention/metadata.yaml 2024-06-03 16:31:19.000000000 +0000 @@ -0,0 +1,13 @@ +friendly_name: Retention +description: |- + Please provide a description for the query +owners: [] +labels: {} +bigquery: null +workgroup_access: +- role: roles/bigquery.dataViewer + members: + - workgroup:mozilla-confidential +references: + view.sql: + - moz-fx-data-shared-prod.focus_android_derived.retention_v1 diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android/retention/view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android/retention/view.sql --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android/retention/view.sql 1970-01-01 00:00:00.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android/retention/view.sql 2024-06-03 16:20:45.000000000 +0000 @@ -0,0 +1,19 @@ +-- Query generated via `mobile_kpi_support_metrics` SQL generator. +CREATE OR REPLACE VIEW + `moz-fx-data-shared-prod.focus_android.retention` +AS +SELECT + *, + CASE + WHEN first_seen_date = metric_date + THEN 'new_profile' + WHEN DATE_DIFF(metric_date, first_seen_date, DAY) + BETWEEN 1 + AND 27 + THEN 'repeat_user' + WHEN DATE_DIFF(metric_date, first_seen_date, DAY) >= 28 + THEN 'existing_user' + ELSE 'Unknown' + END AS lifecycle_stage, +FROM + `moz-fx-data-shared-prod.focus_android_derived.retention_v1` diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android/retention_clients/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android/retention_clients/metadata.yaml --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android/retention_clients/metadata.yaml 1970-01-01 00:00:00.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android/retention_clients/metadata.yaml 2024-06-03 16:31:19.000000000 +0000 @@ -0,0 +1,14 @@ +friendly_name: Retention Clients +description: |- + Please provide a description for the query +owners: [] +labels: {} +bigquery: null +workgroup_access: +- role: roles/bigquery.dataViewer + members: + - workgroup:mozilla-confidential +references: + view.sql: + - moz-fx-data-shared-prod.focus_android.active_users + - moz-fx-data-shared-prod.focus_android.baseline_clients_daily diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android/retention_clients/view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android/retention_clients/view.sql --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android/retention_clients/view.sql 1970-01-01 00:00:00.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android/retention_clients/view.sql 2024-06-03 16:20:45.000000000 +0000 @@ -0,0 +1,77 @@ +-- Query generated via `mobile_kpi_support_metrics` SQL generator. +CREATE OR REPLACE VIEW + `moz-fx-data-shared-prod.focus_android.retention_clients` +AS +WITH active_users AS ( + SELECT + submission_date, + client_id, + sample_id, + app_name, + normalized_channel, + mozfun.bits28.retention(days_seen_bits, submission_date) AS retention_seen, + mozfun.bits28.retention(days_active_bits & days_seen_bits, submission_date) AS retention_active, + days_seen_bits, + days_active_bits, + is_mobile, + FROM + `moz-fx-data-shared-prod.focus_android.active_users` +) +SELECT + active_users.submission_date AS submission_date, + clients_daily.submission_date AS metric_date, + clients_daily.first_seen_date, + clients_daily.client_id, + clients_daily.sample_id, + active_users.app_name, + clients_daily.normalized_channel, + clients_daily.country, + clients_daily.app_display_version AS app_version, + clients_daily.locale, + clients_daily.isp, + active_users.is_mobile, + -- ping sent retention + active_users.retention_seen.day_27.active_on_metric_date AS ping_sent_metric_date, + ( + active_users.retention_seen.day_27.active_on_metric_date + AND active_users.retention_seen.day_27.active_in_week_3 + ) AS ping_sent_week_4, + -- activity retention + active_users.retention_active.day_27.active_on_metric_date AS active_metric_date, + ( + active_users.retention_active.day_27.active_on_metric_date + AND active_users.retention_active.day_27.active_in_week_3 + ) AS retained_week_4, + -- new profile retention + clients_daily.is_new_profile AS new_profile_metric_date, + ( + clients_daily.is_new_profile + AND active_users.retention_active.day_27.active_in_week_3 + ) AS retained_week_4_new_profile, + ( + clients_daily.is_new_profile + -- Looking back at 27 days to support the official definition of repeat_profile (someone active between days 2 and 28): + AND BIT_COUNT(mozfun.bits28.range(active_users.days_active_bits, -26, 27)) > 0 + ) AS repeat_profile, + active_users.days_seen_bits, + active_users.days_active_bits, + CASE + WHEN clients_daily.submission_date = first_seen_date + THEN 'new_profile' + WHEN DATE_DIFF(clients_daily.submission_date, first_seen_date, DAY) + BETWEEN 1 + AND 27 + THEN 'repeat_user' + WHEN DATE_DIFF(clients_daily.submission_date, first_seen_date, DAY) >= 28 + THEN 'existing_user' + ELSE 'Unknown' + END AS lifecycle_stage, +FROM + `moz-fx-data-shared-prod.focus_android.baseline_clients_daily` AS clients_daily +INNER JOIN + active_users + ON clients_daily.submission_date = active_users.retention_seen.day_27.metric_date + AND clients_daily.client_id = active_users.client_id + AND clients_daily.normalized_channel = active_users.normalized_channel +WHERE + active_users.retention_seen.day_27.active_on_metric_date diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android_derived/engagement_v1/checks.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android_derived/engagement_v1/checks.sql --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android_derived/engagement_v1/checks.sql 1970-01-01 00:00:00.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android_derived/engagement_v1/checks.sql 2024-06-03 16:20:45.000000000 +0000 @@ -0,0 +1,3 @@ +-- Query generated via `mobile_kpi_support_metrics` SQL generator. +# warn +{{ min_row_count(1), "WHERE submission_date = @submission_date" }} diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android_derived/engagement_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android_derived/engagement_v1/metadata.yaml --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android_derived/engagement_v1/metadata.yaml 1970-01-01 00:00:00.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android_derived/engagement_v1/metadata.yaml 2024-06-03 16:31:14.000000000 +0000 @@ -0,0 +1,29 @@ +friendly_name: Profile / Client Engagement - Focus Android (Aggregated) +description: |- + Profile / Client Engagement (Focus Android) aggregated metrics +owners: +- mozilla/kpi_table_reviewers +- kik@mozilla.com +labels: + schedule: daily + incremental: true + owner1: kik +bigquery: + time_partitioning: + type: day + field: submission_date + require_partition_filter: false + expiration_days: null + range_partitioning: null + clustering: + fields: + - app_name + - country + - first_seen_date +workgroup_access: +- role: roles/bigquery.dataViewer + members: + - workgroup:mozilla-confidential +references: + query.sql: + - moz-fx-data-shared-prod.focus_android.engagement_clients diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android_derived/engagement_v1/query.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android_derived/engagement_v1/query.sql --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android_derived/engagement_v1/query.sql 1970-01-01 00:00:00.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android_derived/engagement_v1/query.sql 2024-06-03 16:20:45.000000000 +0000 @@ -0,0 +1,30 @@ +-- Query generated via `mobile_kpi_support_metrics` SQL generator. +SELECT + submission_date, + first_seen_date, + normalized_channel, + app_name, + app_version, + country, + locale, + is_mobile, + COUNTIF(is_dau) AS dau, + COUNTIF(is_wau) AS wau, + COUNTIF(is_mau) AS mau, +FROM + `moz-fx-data-shared-prod.focus_android.engagement_clients` +WHERE + {% if is_init() %} + submission_date < CURRENT_DATE + {% else %} + submission_date = @submission_date + {% endif %} +GROUP BY + submission_date, + first_seen_date, + normalized_channel, + app_name, + app_version, + country, + locale, + is_mobile diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android_derived/engagement_v1/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android_derived/engagement_v1/schema.yaml --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android_derived/engagement_v1/schema.yaml 1970-01-01 00:00:00.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android_derived/engagement_v1/schema.yaml 2024-06-03 16:20:45.000000000 +0000 @@ -0,0 +1,55 @@ +fields: +- mode: NULLABLE + name: submission_date + type: DATE + description: + +- mode: NULLABLE + name: first_seen_date + type: DATE + description: When the profile has been seen for the first time. + +- mode: NULLABLE + name: normalized_channel + type: STRING + description: Release channel of the app the profile is using. + +- name: app_name + type: STRING + mode: NULLABLE + description: App name the profile is using. + +- name: app_version + type: STRING + mode: NULLABLE + description: Client's app version on the metric date. + +- name: country + type: STRING + mode: NULLABLE + description: Client's country on the metric date. + +- name: locale + type: STRING + mode: NULLABLE + description: Client's locale on the metric date. + +- name: is_mobile + type: BOOLEAN + mode: NULLABLE + description: Indicates if this specific entry is used towards calculating mobile DAU. + +- name: dau + type: INTEGER + mode: NULLABLE + description: DAU - Daily Active Users + +- name: wau + type: INTEGER + mode: NULLABLE + description: WAU - Weekly Active Users + +- name: mau + type: INTEGER + mode: NULLABLE + description: MAU - Monthly Active Users diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android_derived/retention_v1/checks.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android_derived/retention_v1/checks.sql --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android_derived/retention_v1/checks.sql 1970-01-01 00:00:00.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android_derived/retention_v1/checks.sql 2024-06-03 16:20:45.000000000 +0000 @@ -0,0 +1,3 @@ +-- Query generated via `mobile_kpi_support_metrics` SQL generator. +# warn +{{ min_row_count(1), "WHERE metric_date = DATE_SUB(@submission_date, INTERVAL 27 DAY)" }} diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android_derived/retention_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android_derived/retention_v1/metadata.yaml --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android_derived/retention_v1/metadata.yaml 1970-01-01 00:00:00.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android_derived/retention_v1/metadata.yaml 2024-06-03 16:31:14.000000000 +0000 @@ -0,0 +1,39 @@ +friendly_name: Profile / Client Retention - Focus Android (Aggregated) +description: |- + Profile / Client Retention (Focus Android) aggregated metrics + + For all profiles that sent us a ping on the metric date. + Client's attribute values as observed on the metric date. + + see: DENG-3183 for more information. +owners: +- mozilla/kpi_table_reviewers +- kik@mozilla.com +labels: + schedule: daily + incremental: true + owner1: kik +scheduling: + date_partition_parameter: metric_date + date_partition_offset: -27 + parameters: + - submission_date:DATE:{{ds}} +bigquery: + time_partitioning: + type: day + field: metric_date + require_partition_filter: false + expiration_days: null + range_partitioning: null + clustering: + fields: + - app_name + - country + - first_seen_date +workgroup_access: +- role: roles/bigquery.dataViewer + members: + - workgroup:mozilla-confidential +references: + query.sql: + - moz-fx-data-shared-prod.focus_android.retention_clients diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android_derived/retention_v1/query.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android_derived/retention_v1/query.sql --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android_derived/retention_v1/query.sql 1970-01-01 00:00:00.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android_derived/retention_v1/query.sql 2024-06-03 16:20:45.000000000 +0000 @@ -0,0 +1,36 @@ +-- Query generated via `mobile_kpi_support_metrics` SQL generator. +SELECT + metric_date, + first_seen_date, + app_name, + normalized_channel, + country, + app_version, + locale, + is_mobile, + COUNTIF(ping_sent_metric_date) AS ping_sent_metric_date, + COUNTIF(ping_sent_week_4) AS ping_sent_week_4, + COUNTIF(active_metric_date) AS active_metric_date, + COUNTIF(retained_week_4) AS retained_week_4, + COUNTIF(retained_week_4_new_profile) AS retained_week_4_new_profiles, + COUNTIF(new_profile_metric_date) AS new_profiles_metric_date, + COUNTIF(repeat_profile) AS repeat_profiles, +FROM + `moz-fx-data-shared-prod.focus_android.retention_clients` +WHERE + {% if is_init() %} + metric_date < DATE_SUB(CURRENT_DATE, INTERVAL 27 DAY) + AND submission_date < CURRENT_DATE + {% else %} + metric_date = DATE_SUB(@submission_date, INTERVAL 27 DAY) + AND submission_date = @submission_date + {% endif %} +GROUP BY + metric_date, + first_seen_date, + app_name, + normalized_channel, + country, + app_version, + locale, + is_mobile diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android_derived/retention_v1/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android_derived/retention_v1/schema.yaml --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android_derived/retention_v1/schema.yaml 1970-01-01 00:00:00.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android_derived/retention_v1/schema.yaml 2024-06-03 16:20:45.000000000 +0000 @@ -0,0 +1,75 @@ +fields: +- mode: NULLABLE + name: metric_date + type: DATE + description: This is 28 days before processing. + +- mode: NULLABLE + name: first_seen_date + type: DATE + description: When the profile has been seen for the first time. + +- name: app_name + type: STRING + mode: NULLABLE + description: App name the profile is using. + +- mode: NULLABLE + name: normalized_channel + type: STRING + description: Release channel of the app the profile is using. + +- name: country + type: STRING + mode: NULLABLE + description: Client's country on the metric date. + +- name: app_version + type: STRING + mode: NULLABLE + description: Client's app version on the metric date. + +- name: locale + type: STRING + mode: NULLABLE + description: Client's locale on the metric date. + +- name: is_mobile + type: BOOLEAN + mode: NULLABLE + description: Indicates if this specific entry is used towards calculating mobile DAU. + +- name: ping_sent_metric_date + type: INTEGER + mode: NULLABLE + description: Number of profiles that sent a ping on the metric date. + +- name: ping_sent_week_4 + type: INTEGER + mode: NULLABLE + description: Number of profiles that sent a ping between days 22 and 28 after the metric date. + +- name: active_metric_date + type: INTEGER + mode: NULLABLE + description: Number of DAU profiles on the metric date. + +- name: retained_week_4 + type: INTEGER + mode: NULLABLE + description: Number of profiles that were DAU on the metric date and between days 22 and 28 after the metric date. + +- name: retained_week_4_new_profiles + type: INTEGER + mode: NULLABLE + description: Number of new profiles on the metric date that were DAU between days 22 and 28 after the metric date. + +- name: new_profiles_metric_date + type: INTEGER + mode: NULLABLE + description: Number of new profiles on the metric date. + +- name: repeat_profiles + type: INTEGER + mode: NULLABLE + description: Number of new profiles on the metric date that were DAU at least twice in the next 28 days. diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_ios/engagement/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_ios/engagement/metadata.yaml --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_ios/engagement/metadata.yaml 1970-01-01 00:00:00.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_ios/engagement/metadata.yaml 2024-06-03 16:31:19.000000000 +0000 @@ -0,0 +1,13 @@ +friendly_name: Engagement +description: |- + Please provide a description for the query +owners: [] +labels: {} +bigquery: null +workgroup_access: +- role: roles/bigquery.dataViewer + members: + - workgroup:mozilla-confidential +references: + view.sql: + - moz-fx-data-shared-prod.focus_ios_derived.engagement_v1 diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_ios/engagement/view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_ios/engagement/view.sql --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_ios/engagement/view.sql 1970-01-01 00:00:00.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_ios/engagement/view.sql 2024-06-03 16:20:45.000000000 +0000 @@ -0,0 +1,19 @@ +-- Query generated via `mobile_kpi_support_metrics` SQL generator. +CREATE OR REPLACE VIEW + `moz-fx-data-shared-prod.focus_ios.engagement` +AS +SELECT + *, + CASE + WHEN first_seen_date = submission_date + THEN 'new_profile' + WHEN DATE_DIFF(submission_date, first_seen_date, DAY) + BETWEEN 1 + AND 27 + THEN 'repeat_user' + WHEN DATE_DIFF(submission_date, first_seen_date, DAY) >= 28 + THEN 'existing_user' + ELSE 'Unknown' + END AS lifecycle_stage, +FROM + `moz-fx-data-shared-prod.focus_ios_derived.engagement_v1` diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_ios/engagement_clients/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_ios/engagement_clients/metadata.yaml --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_ios/engagement_clients/metadata.yaml 1970-01-01 00:00:00.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_ios/engagement_clients/metadata.yaml 2024-06-03 16:31:19.000000000 +0000 @@ -0,0 +1,13 @@ +friendly_name: Engagement Clients +description: |- + Please provide a description for the query +owners: [] +labels: {} +bigquery: null +workgroup_access: +- role: roles/bigquery.dataViewer + members: + - workgroup:mozilla-confidential +references: + view.sql: + - moz-fx-data-shared-prod.focus_ios.active_users diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_ios/engagement_clients/view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_ios/engagement_clients/view.sql --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_ios/engagement_clients/view.sql 1970-01-01 00:00:00.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_ios/engagement_clients/view.sql 2024-06-03 16:20:45.000000000 +0000 @@ -0,0 +1,51 @@ +-- Query generated via `mobile_kpi_support_metrics` SQL generator. +CREATE OR REPLACE VIEW + `moz-fx-data-shared-prod.focus_ios.engagement_clients` +AS +WITH active_users AS ( + SELECT + submission_date, + client_id, + sample_id, + first_seen_date, + app_name, + normalized_channel, + locale, + country, + isp, + app_display_version, + is_dau, + is_wau, + is_mau, + is_mobile, + FROM + `moz-fx-data-shared-prod.focus_ios.active_users` +) +SELECT + submission_date, + client_id, + sample_id, + first_seen_date, + app_name, + normalized_channel, + app_display_version AS app_version, + locale, + country, + isp, + is_dau, + is_wau, + is_mau, + is_mobile, + CASE + WHEN active_users.submission_date = first_seen_date + THEN 'new_profile' + WHEN DATE_DIFF(active_users.submission_date, first_seen_date, DAY) + BETWEEN 1 + AND 27 + THEN 'repeat_user' + WHEN DATE_DIFF(active_users.submission_date, first_seen_date, DAY) >= 28 + THEN 'existing_user' + ELSE 'Unknown' + END AS lifecycle_stage, +FROM + active_users diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_ios/retention/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_ios/retention/metadata.yaml --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_ios/retention/metadata.yaml 1970-01-01 00:00:00.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_ios/retention/metadata.yaml 2024-06-03 16:31:19.000000000 +0000 @@ -0,0 +1,13 @@ +friendly_name: Retention +description: |- + Please provide a description for the query +owners: [] +labels: {} +bigquery: null +workgroup_access: +- role: roles/bigquery.dataViewer + members: + - workgroup:mozilla-confidential +references: + view.sql: + - moz-fx-data-shared-prod.focus_ios_derived.retention_v1 diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_ios/retention/view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_ios/retention/view.sql --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_ios/retention/view.sql 1970-01-01 00:00:00.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_ios/retention/view.sql 2024-06-03 16:20:45.000000000 +0000 @@ -0,0 +1,19 @@ +-- Query generated via `mobile_kpi_support_metrics` SQL generator. +CREATE OR REPLACE VIEW + `moz-fx-data-shared-prod.focus_ios.retention` +AS +SELECT + *, + CASE + WHEN first_seen_date = metric_date + THEN 'new_profile' + WHEN DATE_DIFF(metric_date, first_seen_date, DAY) + BETWEEN 1 + AND 27 + THEN 'repeat_user' + WHEN DATE_DIFF(metric_date, first_seen_date, DAY) >= 28 + THEN 'existing_user' + ELSE 'Unknown' + END AS lifecycle_stage, +FROM + `moz-fx-data-shared-prod.focus_ios_derived.retention_v1` diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_ios/retention_clients/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_ios/retention_clients/metadata.yaml --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_ios/retention_clients/metadata.yaml 1970-01-01 00:00:00.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_ios/retention_clients/metadata.yaml 2024-06-03 16:31:19.000000000 +0000 @@ -0,0 +1,14 @@ +friendly_name: Retention Clients +description: |- + Please provide a description for the query +owners: [] +labels: {} +bigquery: null +workgroup_access: +- role: roles/bigquery.dataViewer + members: + - workgroup:mozilla-confidential +references: + view.sql: + - moz-fx-data-shared-prod.focus_ios.active_users + - moz-fx-data-shared-prod.focus_ios.baseline_clients_daily diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_ios/retention_clients/view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_ios/retention_clients/view.sql --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_ios/retention_clients/view.sql 1970-01-01 00:00:00.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_ios/retention_clients/view.sql 2024-06-03 16:20:45.000000000 +0000 @@ -0,0 +1,77 @@ +-- Query generated via `mobile_kpi_support_metrics` SQL generator. +CREATE OR REPLACE VIEW + `moz-fx-data-shared-prod.focus_ios.retention_clients` +AS +WITH active_users AS ( + SELECT + submission_date, + client_id, + sample_id, + app_name, + normalized_channel, + mozfun.bits28.retention(days_seen_bits, submission_date) AS retention_seen, + mozfun.bits28.retention(days_active_bits & days_seen_bits, submission_date) AS retention_active, + days_seen_bits, + days_active_bits, + is_mobile, + FROM + `moz-fx-data-shared-prod.focus_ios.active_users` +) +SELECT + active_users.submission_date AS submission_date, + clients_daily.submission_date AS metric_date, + clients_daily.first_seen_date, + clients_daily.client_id, + clients_daily.sample_id, + active_users.app_name, + clients_daily.normalized_channel, + clients_daily.country, + clients_daily.app_display_version AS app_version, + clients_daily.locale, + clients_daily.isp, + active_users.is_mobile, + -- ping sent retention + active_users.retention_seen.day_27.active_on_metric_date AS ping_sent_metric_date, + ( + active_users.retention_seen.day_27.active_on_metric_date + AND active_users.retention_seen.day_27.active_in_week_3 + ) AS ping_sent_week_4, + -- activity retention + active_users.retention_active.day_27.active_on_metric_date AS active_metric_date, + ( + active_users.retention_active.day_27.active_on_metric_date + AND active_users.retention_active.day_27.active_in_week_3 + ) AS retained_week_4, + -- new profile retention + clients_daily.is_new_profile AS new_profile_metric_date, + ( + clients_daily.is_new_profile + AND active_users.retention_active.day_27.active_in_week_3 + ) AS retained_week_4_new_profile, + ( + clients_daily.is_new_profile + -- Looking back at 27 days to support the official definition of repeat_profile (someone active between days 2 and 28): + AND BIT_COUNT(mozfun.bits28.range(active_users.days_active_bits, -26, 27)) > 0 + ) AS repeat_profile, + active_users.days_seen_bits, + active_users.days_active_bits, + CASE + WHEN clients_daily.submission_date = first_seen_date + THEN 'new_profile' + WHEN DATE_DIFF(clients_daily.submission_date, first_seen_date, DAY) + BETWEEN 1 + AND 27 + THEN 'repeat_user' + WHEN DATE_DIFF(clients_daily.submission_date, first_seen_date, DAY) >= 28 + THEN 'existing_user' + ELSE 'Unknown' + END AS lifecycle_stage, +FROM + `moz-fx-data-shared-prod.focus_ios.baseline_clients_daily` AS clients_daily +INNER JOIN + active_users + ON clients_daily.submission_date = active_users.retention_seen.day_27.metric_date + AND clients_daily.client_id = active_users.client_id + AND clients_daily.normalized_channel = active_users.normalized_channel +WHERE + active_users.retention_seen.day_27.active_on_metric_date diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_ios_derived/engagement_v1/checks.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_ios_derived/engagement_v1/checks.sql --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_ios_derived/engagement_v1/checks.sql 1970-01-01 00:00:00.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_ios_derived/engagement_v1/checks.sql 2024-06-03 16:20:45.000000000 +0000 @@ -0,0 +1,3 @@ +-- Query generated via `mobile_kpi_support_metrics` SQL generator. +# warn +{{ min_row_count(1), "WHERE submission_date = @submission_date" }} diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_ios_derived/engagement_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_ios_derived/engagement_v1/metadata.yaml --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_ios_derived/engagement_v1/metadata.yaml 1970-01-01 00:00:00.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_ios_derived/engagement_v1/metadata.yaml 2024-06-03 16:31:13.000000000 +0000 @@ -0,0 +1,29 @@ +friendly_name: Profile / Client Engagement - Focus iOS (Aggregated) +description: |- + Profile / Client Engagement (Focus iOS) aggregated metrics +owners: +- mozilla/kpi_table_reviewers +- kik@mozilla.com +labels: + schedule: daily + incremental: true + owner1: kik +bigquery: + time_partitioning: + type: day + field: submission_date + require_partition_filter: false + expiration_days: null + range_partitioning: null + clustering: + fields: + - app_name + - country + - first_seen_date +workgroup_access: +- role: roles/bigquery.dataViewer + members: + - workgroup:mozilla-confidential +references: + query.sql: + - moz-fx-data-shared-prod.focus_ios.engagement_clients diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_ios_derived/engagement_v1/query.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_ios_derived/engagement_v1/query.sql --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_ios_derived/engagement_v1/query.sql 1970-01-01 00:00:00.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_ios_derived/engagement_v1/query.sql 2024-06-03 16:20:45.000000000 +0000 @@ -0,0 +1,30 @@ +-- Query generated via `mobile_kpi_support_metrics` SQL generator. +SELECT + submission_date, + first_seen_date, + normalized_channel, + app_name, + app_version, + country, + locale, + is_mobile, + COUNTIF(is_dau) AS dau, + COUNTIF(is_wau) AS wau, + COUNTIF(is_mau) AS mau, +FROM + `moz-fx-data-shared-prod.focus_ios.engagement_clients` +WHERE + {% if is_init() %} + submission_date < CURRENT_DATE + {% else %} + submission_date = @submission_date + {% endif %} +GROUP BY + submission_date, + first_seen_date, + normalized_channel, + app_name, + app_version, + country, + locale, + is_mobile diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_ios_derived/engagement_v1/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_ios_derived/engagement_v1/schema.yaml --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_ios_derived/engagement_v1/schema.yaml 1970-01-01 00:00:00.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_ios_derived/engagement_v1/schema.yaml 2024-06-03 16:20:45.000000000 +0000 @@ -0,0 +1,55 @@ +fields: +- mode: NULLABLE + name: submission_date + type: DATE + description: + +- mode: NULLABLE + name: first_seen_date + type: DATE + description: When the profile has been seen for the first time. + +- mode: NULLABLE + name: normalized_channel + type: STRING + description: Release channel of the app the profile is using. + +- name: app_name + type: STRING + mode: NULLABLE + description: App name the profile is using. + +- name: app_version + type: STRING + mode: NULLABLE + description: Client's app version on the metric date. + +- name: country + type: STRING + mode: NULLABLE + description: Client's country on the metric date. + +- name: locale + type: STRING + mode: NULLABLE + description: Client's locale on the metric date. + +- name: is_mobile + type: BOOLEAN + mode: NULLABLE + description: Indicates if this specific entry is used towards calculating mobile DAU. + +- name: dau + type: INTEGER + mode: NULLABLE + description: DAU - Daily Active Users + +- name: wau + type: INTEGER + mode: NULLABLE + description: WAU - Weekly Active Users + +- name: mau + type: INTEGER + mode: NULLABLE + description: MAU - Monthly Active Users diff -bur --no-dereference --ne ```

⚠️ Only part of the diff is displayed.

Link to full diff