DENG-2407 Use flow_id string metric for accounts event flow monitoring

[akkomar]

Follow up to https://github.com/mozilla/bigquery-etl/pull/5801 - this is needed because accounts send flow_id in a string metric.

Checklist for reviewer:

• [ ] Commits should reference a bug or github issue, if relevant (if a bug is referenced, the pull request should include the bug number in the title).
• [ ] If the PR comes from a fork, trigger integration CI tests by running the Push to upstream workflow and provide the <username>:<branch> of the fork as parameter. The parameter will also show up in the logs of the manual-trigger-required-for-fork CI task together with more detailed instructions.
• [ ] If adding a new field to a query, ensure that the schema and dependent downstream schemas have been updated.
• [ ] When adding a new derived dataset, ensure that data is not available already (fully or partially) and recommend extending an existing dataset in favor of creating new ones. Data can be available in the bigquery-etl repository, looker-hub or in looker-spoke-default.

For modifications to schemas in restricted namespaces (see CODEOWNERS):

• [ ] Follow the change control procedure

┆Issue is synchronized with this Jira Task

[dataops-ci-bot]

Integration report for "DENG-2407 Use flow_id string metric for accounts event flow monitoring"

sql.diff

Click to expand!

```diff Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop: fx_accounts diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates/schema.yaml --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates/schema.yaml 2024-06-20 10:11:00.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates/schema.yaml 2024-06-20 10:21:44.000000000 +0000 @@ -1,49 +1,49 @@ fields: -- mode: NULLABLE - name: submission_date +- name: submission_date type: DATE -- mode: NULLABLE - name: source + mode: NULLABLE +- name: source type: STRING -- mode: NULLABLE - name: event_type + mode: NULLABLE +- name: event_type type: STRING -- mode: NULLABLE - name: form_factor + mode: NULLABLE +- name: form_factor type: STRING -- mode: NULLABLE - name: country + mode: NULLABLE +- name: country type: STRING -- mode: NULLABLE - name: subdivision1 + mode: NULLABLE +- name: subdivision1 type: STRING -- mode: NULLABLE - name: advertiser + mode: NULLABLE +- name: advertiser type: STRING -- mode: NULLABLE - name: release_channel + mode: NULLABLE +- name: release_channel type: STRING -- mode: NULLABLE - name: position + mode: NULLABLE +- name: position type: INTEGER -- mode: NULLABLE - name: provider + mode: NULLABLE +- name: provider type: STRING -- mode: NULLABLE - name: match_type + mode: NULLABLE +- name: match_type type: STRING -- mode: NULLABLE - name: normalized_os + mode: NULLABLE +- name: normalized_os type: STRING -- mode: NULLABLE - name: suggest_data_sharing_enabled + mode: NULLABLE +- name: suggest_data_sharing_enabled type: BOOLEAN -- mode: NULLABLE - name: event_count + mode: NULLABLE +- name: event_count type: INTEGER -- mode: NULLABLE - name: user_count + mode: NULLABLE +- name: user_count type: INTEGER -- mode: NULLABLE - name: query_type + mode: NULLABLE +- name: query_type type: STRING + mode: NULLABLE diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates_suggest/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates_suggest/schema.yaml --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates_suggest/schema.yaml 2024-06-20 10:11:00.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates_suggest/schema.yaml 2024-06-20 10:21:44.000000000 +0000 @@ -1,40 +1,40 @@ fields: -- mode: NULLABLE - name: submission_date +- name: submission_date type: DATE -- mode: NULLABLE - name: form_factor + mode: NULLABLE +- name: form_factor type: STRING -- mode: NULLABLE - name: country + mode: NULLABLE +- name: country type: STRING -- mode: NULLABLE - name: advertiser + mode: NULLABLE +- name: advertiser type: STRING -- mode: NULLABLE - name: normalized_os + mode: NULLABLE +- name: normalized_os type: STRING -- mode: NULLABLE - name: release_channel + mode: NULLABLE +- name: release_channel type: STRING -- mode: NULLABLE - name: position + mode: NULLABLE +- name: position type: INTEGER -- mode: NULLABLE - name: provider + mode: NULLABLE +- name: provider type: STRING -- mode: NULLABLE - name: match_type + mode: NULLABLE +- name: match_type type: STRING -- mode: NULLABLE - name: suggest_data_sharing_enabled + mode: NULLABLE +- name: suggest_data_sharing_enabled type: BOOLEAN -- mode: NULLABLE - name: impression_count + mode: NULLABLE +- name: impression_count type: INTEGER -- mode: NULLABLE - name: click_count + mode: NULLABLE +- name: click_count type: INTEGER -- mode: NULLABLE - name: query_type + mode: NULLABLE +- name: query_type type: STRING + mode: NULLABLE diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_clients/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_clients/schema.yaml --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_clients/schema.yaml 2024-06-20 10:11:00.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_clients/schema.yaml 2024-06-20 10:19:30.000000000 +0000 @@ -26,6 +26,9 @@ - name: adjust_network type: STRING mode: NULLABLE +- name: install_source + type: STRING + mode: NULLABLE - name: retained_week_2 type: BOOLEAN mode: NULLABLE diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_week_4/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_week_4/schema.yaml --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_week_4/schema.yaml 2024-06-20 10:11:00.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_week_4/schema.yaml 2024-06-20 10:19:31.000000000 +0000 @@ -48,6 +48,10 @@ description: 'The type of source of a client installation. ' +- name: install_source + type: STRING + mode: NULLABLE + description: null - name: new_profiles type: INTEGER mode: NULLABLE diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop/fx_accounts/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop/fx_accounts/metadata.yaml --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop/fx_accounts/metadata.yaml 1970-01-01 00:00:00.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop/fx_accounts/metadata.yaml 2024-06-20 10:27:44.000000000 +0000 @@ -0,0 +1,21 @@ +friendly_name: Historical Pings for `firefox-desktop/fx-accounts` +description: |- + A historical view of pings sent for the + `firefox-desktop/fx-accounts` + document type. + + This view is guaranteed to contain only complete days + (per `submission_timestamp`) + and to contain only one row per distinct `document_id` within a given date. + + Clustering fields: `normalized_channel`, `sample_id` +owners: [] +labels: {} +bigquery: null +workgroup_access: +- role: roles/bigquery.dataViewer + members: + - workgroup:mozilla-confidential +references: + view.sql: + - moz-fx-data-shared-prod.firefox_desktop_stable.fx_accounts_v1 diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop/fx_accounts/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop/fx_accounts/schema.yaml --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop/fx_accounts/schema.yaml 1970-01-01 00:00:00.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop/fx_accounts/schema.yaml 2024-06-20 10:12:43.000000000 +0000 @@ -0,0 +1,407 @@ +fields: +- name: additional_properties + type: STRING + mode: NULLABLE + description: A JSON string containing any payload properties not present in the + schema +- name: client_info + type: RECORD + mode: NULLABLE + fields: + - name: android_sdk_version + type: STRING + mode: NULLABLE + description: The optional Android specific SDK version of the software running + on this hardware device. + - name: app_build + type: STRING + mode: NULLABLE + description: The build identifier generated by the CI system (e.g. "1234/A"). + For language bindings that provide automatic detection for this value, (e.g. + Android/Kotlin), in the unlikely event that the build identifier can not be + retrieved from the OS, it is set to "inaccessible". For other language bindings, + if the value was not provided through configuration, this metric gets set to + `Unknown`. + - name: app_channel + type: STRING + mode: NULLABLE + description: The channel the application is being distributed on. + - name: app_display_version + type: STRING + mode: NULLABLE + description: The user visible version string (e.g. "1.0.3"). In the unlikely + event that the display version can not be retrieved, it is set to "inaccessible". + - name: architecture + type: STRING + mode: NULLABLE + description: The architecture of the device, (e.g. "arm", "x86"). + - name: build_date + type: STRING + mode: NULLABLE + description: The date & time the application was built + - name: client_id + type: STRING + mode: NULLABLE + description: A UUID uniquely identifying the client. + - name: device_manufacturer + type: STRING + mode: NULLABLE + description: The manufacturer of the device the application is running on. Not + set if the device manufacturer can't be determined (e.g. on Desktop). + - name: device_model + type: STRING + mode: NULLABLE + description: The model of the device the application is running on. On Android, + this is Build.MODEL, the user-visible marketing name, like "Pixel 2 XL". Not + set if the device model can't be determined (e.g. on Desktop). + - name: first_run_date + type: STRING + mode: NULLABLE + description: The date of the first run of the application. + - name: locale + type: STRING + mode: NULLABLE + description: The locale of the application during initialization (e.g. "es-ES"). + If the locale can't be determined on the system, the value is ["und"](https://unicode.org/reports/tr35/#Unknown_or_Invalid_Identifiers), + to indicate "undetermined". + - name: os + type: STRING + mode: NULLABLE + description: 'The name of the operating system. Possible values: Android, iOS, + Linux, Darwin, Windows, FreeBSD, NetBSD, OpenBSD, Solaris, unknown' + - name: os_version + type: STRING + mode: NULLABLE + description: The user-visible version of the operating system (e.g. "1.2.3"). + If the version detection fails, this metric gets set to `Unknown`. + - name: session_count + type: INTEGER + mode: NULLABLE + description: An optional running counter of the number of sessions for a client. + - name: session_id + type: STRING + mode: NULLABLE + description: An optional UUID uniquely identifying the client's current session. + - name: telemetry_sdk_build + type: STRING + mode: NULLABLE + description: The version of the Glean SDK + - name: windows_build_number + type: INTEGER + mode: NULLABLE + description: The optional Windows build number, reported by Windows (e.g. 22000) + and not set for other platforms +- name: document_id + type: STRING + mode: NULLABLE + description: The document ID specified in the URI when the client sent this message +- name: events + type: RECORD + mode: REPEATED + fields: + - name: category + type: STRING + mode: NULLABLE + - name: extra + type: RECORD + mode: REPEATED + fields: + - name: key + type: STRING + mode: NULLABLE + - name: value + type: STRING + mode: NULLABLE + - name: name + type: STRING + mode: NULLABLE + - name: timestamp + type: INTEGER + mode: NULLABLE +- name: metadata + type: RECORD + mode: NULLABLE + fields: + - name: geo + type: RECORD + mode: NULLABLE + fields: + - name: city + type: STRING + mode: NULLABLE + - name: country + type: STRING + mode: NULLABLE + description: An ISO 3166-1 alpha-2 country code + - name: db_version + type: STRING + mode: NULLABLE + description: The specific geo database version used for this lookup + - name: subdivision1 + type: STRING + mode: NULLABLE + description: First major country subdivision, typically a state, province, or + county + - name: subdivision2 + type: STRING + mode: NULLABLE + description: Second major country subdivision; not applicable for most countries + description: Results of a geographic lookup based on the client's IP address + - name: header + type: RECORD + mode: NULLABLE + fields: + - name: date + type: STRING + mode: NULLABLE + description: Date HTTP header + - name: dnt + type: STRING + mode: NULLABLE + description: DNT (Do Not Track) HTTP header + - name: x_debug_id + type: STRING + mode: NULLABLE + description: X-Debug-Id HTTP header + - name: x_foxsec_ip_reputation + type: STRING + mode: NULLABLE + description: X-Foxsec-IP-Reputation header + - name: x_lb_tags + type: STRING + mode: NULLABLE + description: X-LB-Tags HTTP header + - name: x_pingsender_version + type: STRING + mode: NULLABLE + description: X-PingSender-Version HTTP header + - name: x_source_tags + type: STRING + mode: NULLABLE + description: X-Source-Tags HTTP header + - name: x_telemetry_agent + type: STRING + mode: NULLABLE + description: X-Telemetry-Agent HTTP header + - name: parsed_date + type: TIMESTAMP + mode: NULLABLE + - name: parsed_x_source_tags + type: STRING + mode: REPEATED + - name: parsed_x_lb_tags + type: RECORD + mode: NULLABLE + fields: + - name: tls_version + type: STRING + mode: NULLABLE + - name: tls_cipher_hex + type: STRING + mode: NULLABLE + description: Headers included in the client's HTTP request + - name: isp + type: RECORD + mode: NULLABLE + fields: + - name: db_version + type: STRING + mode: NULLABLE + description: The specific geo ISP database version used for this lookup + - name: name + type: STRING + mode: NULLABLE + description: The name of the ISP associated with the client's IP address + - name: organization + type: STRING + mode: NULLABLE + description: The name of a specific business entity associated with the client's + IP address when available; otherwise the ISP name + description: Results of ISP lookup based on the client's IP address + - name: user_agent + type: RECORD + mode: NULLABLE + fields: + - name: browser + type: STRING + mode: NULLABLE + - name: os + type: STRING + mode: NULLABLE + - name: version + type: STRING + mode: NULLABLE + description: Parsed components of the client's user agent string +- name: metrics + type: RECORD + mode: NULLABLE + fields: + - name: labeled_counter + type: RECORD + mode: NULLABLE + fields: + - name: glean_error_invalid_label + type: RECORD + mode: REPEATED + fields: + - name: key + type: STRING + mode: NULLABLE + - name: value + type: INTEGER + mode: NULLABLE + description: 'Counts the number of times a metric was set with an invalid label. + + The labels are the `category.name` identifier of the metric. + + ' + - name: glean_error_invalid_overflow + type: RECORD + mode: REPEATED + fields: + - name: key + type: STRING + mode: NULLABLE + - name: value + type: INTEGER + mode: NULLABLE + description: 'Counts the number of times a metric was set a value that overflowed. + + The labels are the `category.name` identifier of the metric. + + ' + - name: glean_error_invalid_state + type: RECORD + mode: REPEATED + fields: + - name: key + type: STRING + mode: NULLABLE + - name: value + type: INTEGER + mode: NULLABLE + description: 'Counts the number of times a timing metric was used incorrectly. + + The labels are the `category.name` identifier of the metric. + + ' + - name: glean_error_invalid_value + type: RECORD + mode: REPEATED + fields: + - name: key + type: STRING + mode: NULLABLE + - name: value + type: INTEGER + mode: NULLABLE + description: 'Counts the number of times a metric was set to an invalid value. + + The labels are the `category.name` identifier of the metric. + + ' + - name: string + type: RECORD + mode: NULLABLE + fields: + - name: client_association_uid + type: STRING + mode: NULLABLE + description: 'The Mozilla Account UID associated with the user. + + ' + - name: glean_client_annotation_experimentation_id + type: STRING + mode: NULLABLE + description: 'An experimentation identifier derived and provided by the application + + for the purpose of experimentation enrollment. + + ' + - name: uuid + type: RECORD + mode: NULLABLE + fields: + - name: client_association_legacy_client_id + type: STRING + mode: NULLABLE + description: 'The legacy Telemetry client ID associated with this user profile. + + ' +- name: normalized_app_name + type: STRING + mode: NULLABLE + description: Set to "Other" if this message contained an unrecognized app name +- name: normalized_channel + type: STRING + mode: NULLABLE + description: Set to "Other" if this message contained an unrecognized channel name +- name: normalized_country_code + type: STRING + mode: NULLABLE + description: An ISO 3166-1 alpha-2 country code +- name: normalized_os + type: STRING + mode: NULLABLE + description: Set to "Other" if this message contained an unrecognized OS name +- name: normalized_os_version + type: STRING + mode: NULLABLE +- name: ping_info + type: RECORD + mode: NULLABLE + fields: + - name: end_time + type: STRING + mode: NULLABLE + - name: experiments + type: RECORD + mode: REPEATED + fields: + - name: key + type: STRING + mode: NULLABLE + - name: value + type: RECORD + mode: NULLABLE + fields: + - name: branch + type: STRING + mode: NULLABLE + - name: extra + type: RECORD + mode: NULLABLE + fields: + - name: enrollment_id + type: STRING + mode: NULLABLE + - name: type + type: STRING + mode: NULLABLE + - name: ping_type + type: STRING + mode: NULLABLE + - name: reason + type: STRING + mode: NULLABLE + - name: seq + type: INTEGER + mode: NULLABLE + - name: start_time + type: STRING + mode: NULLABLE + - name: parsed_start_time + type: TIMESTAMP + mode: NULLABLE + - name: parsed_end_time + type: TIMESTAMP + mode: NULLABLE +- name: sample_id + type: INTEGER + mode: NULLABLE + description: Hashed version of client_id (if present) useful for partitioning; ranges + from 0 to 99 +- name: submission_timestamp + type: TIMESTAMP + mode: NULLABLE + description: Time when the ingestion edge server accepted this message diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop/fx_accounts/view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop/fx_accounts/view.sql --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop/fx_accounts/view.sql 1970-01-01 00:00:00.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop/fx_accounts/view.sql 2024-06-20 10:12:42.000000000 +0000 @@ -0,0 +1,12 @@ +-- Generated via ./bqetl generate stable_views +CREATE OR REPLACE VIEW + `moz-fx-data-shared-prod.firefox_desktop.fx_accounts` +AS +SELECT + * REPLACE ( + mozfun.norm.metadata(metadata) AS metadata, + mozfun.norm.glean_ping_info(ping_info) AS ping_info, + 'Firefox' AS normalized_app_name + ) +FROM + `moz-fx-data-shared-prod.firefox_desktop_stable.fx_accounts_v1` diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/monitoring/airflow_trigger/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/monitoring/airflow_trigger/schema.yaml --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/monitoring/airflow_trigger/schema.yaml 2024-06-20 10:11:38.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/monitoring/airflow_trigger/schema.yaml 2024-06-20 10:19:12.000000000 +0000 @@ -17,11 +17,3 @@ - name: triggerer_id type: INTEGER mode: NULLABLE -- name: trigger_id - type: INTEGER - mode: NULLABLE - description: null -- name: created_at - type: TIMESTAMP - mode: NULLABLE - description: null diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/monitoring_derived/event_flow_monitoring_aggregates_v1/script.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/monitoring_derived/event_flow_monitoring_aggregates_v1/script.sql --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/monitoring_derived/event_flow_monitoring_aggregates_v1/script.sql 2024-06-20 10:11:38.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/monitoring_derived/event_flow_monitoring_aggregates_v1/script.sql 2024-06-20 10:15:44.000000000 +0000 @@ -623,7 +623,8 @@ UNNEST(event_extra) AS ext WHERE DATE(submission_timestamp) = @submission_date - AND ext.key = "flow_id" + AND metrics.string.session_flow_id IS NOT NULL + AND metrics.string.session_flow_id != "" UNION ALL SELECT DISTINCT @submission_date AS submission_date, @@ -642,7 +643,8 @@ UNNEST(event_extra) AS ext WHERE DATE(submission_timestamp) = @submission_date - AND ext.key = "flow_id" + AND metrics.string.session_flow_id IS NOT NULL + AND metrics.string.session_flow_id != "" UNION ALL SELECT DISTINCT @submission_date AS submission_date, diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/mozilla_vpn/subscriptions/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/mozilla_vpn/subscriptions/schema.yaml --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/mozilla_vpn/subscriptions/schema.yaml 2024-06-20 10:11:38.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/mozilla_vpn/subscriptions/schema.yaml 2024-06-20 10:21:52.000000000 +0000 @@ -20,45 +20,3 @@ - name: type type: STRING mode: NULLABLE -- name: provider - type: STRING - mode: NULLABLE -- name: apple_receipt - type: RECORD - mode: NULLABLE - fields: - - name: environment - type: STRING - mode: NULLABLE - - name: active_period - type: RECORD - mode: NULLABLE - fields: - - name: start_date - type: DATE - mode: NULLABLE - - name: end_date - type: DATE - mode: NULLABLE - - name: start_time - type: TIMESTAMP - mode: NULLABLE - - name: end_time - type: TIMESTAMP - mode: NULLABLE - - name: interval - type: STRING - mode: NULLABLE - - name: interval_count - type: INTEGER - mode: NULLABLE - - name: trial_period - type: RECORD - mode: NULLABLE - fields: - - name: start_time - type: TIMESTAMP - mode: NULLABLE - - name: end_time - type: TIMESTAMP - mode: NULLABLE diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/org_mozilla_fenix/geckoview_version/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/org_mozilla_fenix/geckoview_version/schema.yaml --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/org_mozilla_fenix/geckoview_version/schema.yaml 2024-06-20 10:11:00.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/org_mozilla_fenix/geckoview_version/schema.yaml 2024-06-20 10:21:54.000000000 +0000 @@ -1,7 +1,13 @@ fields: -- type: DATETIME - name: build_hour -- type: INTEGER - name: geckoview_major_version -- type: INTEGER - name: n_pings +- name: build_hour + type: DATETIME + mode: NULLABLE + description: null +- name: geckoview_major_version + type: INTEGER + mode: NULLABLE + description: null +- name: n_pings + type: INTEGER + mode: NULLABLE + description: null diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/org_mozilla_fenix/metrics/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/org_mozilla_fenix/metrics/schema.yaml --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/org_mozilla_fenix/metrics/schema.yaml 2024-06-20 10:11:38.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/org_mozilla_fenix/metrics/schema.yaml 2024-06-20 10:13:12.000000000 +0000 @@ -1880,7 +1880,8 @@ - name: value type: INTEGER mode: NULLABLE - description: 'Failure occurs when initializing the audio stream. + description: 'Failure occurs when initializing the audio stream. (Migrated from + the geckoview metric of the same name). ' - name: glean_validation_pings_submitted diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/org_mozilla_fenix_nightly/metrics/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/org_mozilla_fenix_nightly/metrics/schema.yaml --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/org_mozilla_fenix_nightly/metrics/schema.yaml 2024-06-20 10:11:38.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/org_mozilla_fenix_nightly/metrics/schema.yaml 2024-06-20 10:13:04.000000000 +0000 @@ -1880,7 +1880,8 @@ - name: value type: INTEGER mode: NULLABLE - description: 'Failure occurs when initializing the audio stream. + description: 'Failure occurs when initializing the audio stream. (Migrated from + the geckoview metric of the same name). ' - name: glean_validation_pings_submitted diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/org_mozilla_fennec_aurora/metrics/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/org_mozilla_fennec_aurora/metrics/schema.yaml --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/org_mozilla_fennec_aurora/metrics/schema.yaml 2024-06-20 10:11:37.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/org_mozilla_fennec_aurora/metrics/schema.yaml 2024-06-20 10:13:02.000000000 +0000 @@ -3265,7 +3265,8 @@ - name: value type: INTEGER mode: NULLABLE - description: 'Failure occurs when initializing the audio stream. + description: 'Failure occurs when initializing the audio stream. (Migrated from + the geckoview metric of the same name). ' - name: glean_validation_pings_submitted diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/org_mozilla_firefox/metrics/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/org_mozilla_firefox/metrics/schema.yaml --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/org_mozilla_firefox/metrics/schema.yaml 2024-06-20 10:11:38.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/org_mozilla_firefox/metrics/schema.yaml 2024-06-20 10:13:13.000000000 +0000 @@ -3265,8 +3265,7 @@ - name: value type: INTEGER mode: NULLABLE - description: 'Failure occurs when initializing the audio stream. (Migrated from - the geckoview metric of the same name). + description: 'Failure occurs when initializing the audio stream. ' - name: glean_validation_pings_submitted diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/org_mozilla_firefox_beta/metrics/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/org_mozilla_firefox_beta/metrics/schema.yaml --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/org_mozilla_firefox_beta/metrics/schema.yaml 2024-06-20 10:11:38.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/org_mozilla_firefox_beta/metrics/schema.yaml 2024-06-20 10:13:08.000000000 +0000 @@ -3265,7 +3265,8 @@ - name: value type: INTEGER mode: NULLABLE - description: 'Failure occurs when initializing the audio stream. + description: 'Failure occurs when initializing the audio stream. (Migrated from + the geckoview metric of the same name). ' - name: glean_validation_pings_submitted diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/telemetry/eng_workflow_build_parquet/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/telemetry/eng_workflow_build_parquet/schema.yaml --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/telemetry/eng_workflow_build_parquet/schema.yaml 2024-06-20 10:11:38.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/telemetry/eng_workflow_build_parquet/schema.yaml 2024-06-20 10:22:08.000000000 +0000 @@ -28,48 +28,36 @@ - name: artifact type: BOOLEAN mode: NULLABLE - description: true if --enable-artifact-builds - name: ccache type: BOOLEAN mode: NULLABLE - description: true if ccache is in use (--with-ccache) - name: compiler type: STRING mode: NULLABLE - description: The compiler type in use (CC_TYPE) - name: debug type: BOOLEAN mode: NULLABLE - description: true if build is debug (--enable-debug) - name: icecream type: BOOLEAN mode: NULLABLE - description: true if icecream in use - name: opt type: BOOLEAN mode: NULLABLE - description: true if build is optimized (--enable-optimize) - name: sccache type: BOOLEAN mode: NULLABLE - description: true if ccache in use is sccache - description: Selected build options - name: client_id type: STRING mode: NULLABLE - description: A UUID to uniquely identify a client - name: command type: STRING mode: NULLABLE - description: The mach command that was invoked - name: duration_ms type: FLOAT mode: NULLABLE - description: Command duration in milliseconds - name: success type: BOOLEAN mode: NULLABLE - description: true if the command succeeded - name: system type: RECORD mode: NULLABLE @@ -77,40 +65,30 @@ - name: os type: STRING mode: NULLABLE - description: Operating system - name: cpu_brand type: STRING mode: NULLABLE - description: CPU brand string from CPUID - name: drive_is_ssd type: BOOLEAN mode: NULLABLE - description: true if the source directory is on a solid-state disk - name: logical_cores type: INTEGER mode: NULLABLE - description: Number of logical CPU cores present - name: memory_gb type: INTEGER mode: NULLABLE - description: System memory in GB - name: physical_cores type: INTEGER mode: NULLABLE - description: Number of physical CPU cores present - name: virtual_machine type: BOOLEAN mode: NULLABLE - description: true if the OS appears to be running in a virtual machine - name: time type: STRING mode: NULLABLE - description: Time at which this event happened - name: exception type: STRING mode: NULLABLE - description: If a Python exception was encountered during the execution of the command, - this value contains the result of calling `repr` on the exception object. - name: file_types_changed type: RECORD mode: REPEATED @@ -118,11 +96,9 @@ - name: count type: INTEGER mode: NULLABLE - description: Count of changed files with this extension - name: ext type: STRING mode: NULLABLE - description: File extension - name: build_attrs type: RECORD mode: NULLABLE @@ -130,9 +106,6 @@ - name: clobber type: BOOLEAN mode: NULLABLE - description: true if the build was a clobber/full build - name: cpu_percent type: FLOAT mode: NULLABLE - description: cpu utilization observed during a build - description: Attributes characterizing a build diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/telemetry/releases_latest/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/telemetry/releases_latest/schema.yaml --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/telemetry/releases_latest/schema.yaml 2024-06-20 10:11:38.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/telemetry/releases_latest/schema.yaml 2024-06-20 10:22:06.000000000 +0000 @@ -2,29 +2,24 @@ - name: date type: DATE mode: NULLABLE - description: null - name: product type: STRING mode: NULLABLE - description: null - name: category type: STRING mode: NULLABLE - description: null - name: channel type: STRING mode: NULLABLE - name: build_number type: INTEGER mode: NULLABLE - description: null - name: release_date type: DATE mode: NULLABLE - name: version type: STRING mode: NULLABLE - description: null - name: major_version type: NUMERIC mode: NULLABLE ```

Link to full diff