Add telemetry_derived.latest_version to data-observability-dev

scholtzan commented 2 months ago

Add a non-incremental table to the observability sandbox. The updates are scheduled via telemetry-airflow.

Scheduled via https://github.com/mozilla/telemetry-airflow/pull/1999

Checklist for reviewer:

[ ] Commits should reference a bug or github issue, if relevant (if a bug is referenced, the pull request should include the bug number in the title).
[ ] If the PR comes from a fork, trigger integration CI tests by running the Push to upstream workflow and provide the <username>:<branch> of the fork as parameter. The parameter will also show up in the logs of the manual-trigger-required-for-fork CI task together with more detailed instructions.
[ ] If adding a new field to a query, ensure that the schema and dependent downstream schemas have been updated.
[ ] When adding a new derived dataset, ensure that data is not available already (fully or partially) and recommend extending an existing dataset in favor of creating new ones. Data can be available in the bigquery-etl repository, looker-hub or in looker-spoke-default.

For modifications to schemas in restricted namespaces (see CODEOWNERS):

[ ] Follow the change control procedure

┆Issue is synchronized with this Jira Task

dataops-ci-bot commented 2 months ago

Integration report for "Add telemetry_derived.latest_version to data-observability-dev"

`sql.diff`

Click to expand!

```diff Only in /tmp/workspace/generated-sql/sql/data-observability-dev/telemetry_derived: latest_versions_v1 diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/data-observability-dev/fenix_derived/metrics_clients_last_seen_v1/query.sql /tmp/workspace/generated-sql/sql/data-observability-dev/fenix_derived/metrics_clients_last_seen_v1/query.sql --- /tmp/workspace/main-generated-sql/sql/data-observability-dev/fenix_derived/metrics_clients_last_seen_v1/query.sql 2024-05-27 17:49:34.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/data-observability-dev/fenix_derived/metrics_clients_last_seen_v1/query.sql 2024-05-27 17:49:48.000000000 +0000 @@ -15,12 +15,13 @@ AND submission_date = "2024-05-23" AND sample_id < 50 UNION ALL --- create entries where normalized channel is null +-- create an entry where normalized channel is null SELECT * REPLACE (NULL AS normalized_channel), FROM `moz-fx-data-shared-prod.fenix_derived.metrics_clients_last_seen_v1` WHERE submission_date = @submission_date - AND submission_date = "2024-05-27" - AND sample_id = 0 + AND submission_date = "2024-05-24" +LIMIT + 1 diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/data-observability-dev/telemetry_derived/latest_versions_v1/metadata.yaml /tmp/workspace/generated-sql/sql/data-observability-dev/telemetry_derived/latest_versions_v1/metadata.yaml --- /tmp/workspace/main-generated-sql/sql/data-observability-dev/telemetry_derived/latest_versions_v1/metadata.yaml 1970-01-01 00:00:00.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/data-observability-dev/telemetry_derived/latest_versions_v1/metadata.yaml 2024-05-27 17:52:10.000000000 +0000 @@ -0,0 +1,17 @@ +friendly_name: Latest Versions +description: |- + [Missing] +owners: +- ascholtz@mozilla.com +labels: + application: firefox + schedule: daily + owner1: ascholtz +bigquery: null +workgroup_access: +- role: roles/bigquery.dataViewer + members: + - workgroup:mozilla-confidential +references: + query.sql: + - moz-fx-data-shared-prod.telemetry.latest_versions diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/data-observability-dev/telemetry_derived/latest_versions_v1/query.sql /tmp/workspace/generated-sql/sql/data-observability-dev/telemetry_derived/latest_versions_v1/query.sql --- /tmp/workspace/main-generated-sql/sql/data-observability-dev/telemetry_derived/latest_versions_v1/query.sql 1970-01-01 00:00:00.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/data-observability-dev/telemetry_derived/latest_versions_v1/query.sql 2024-05-27 17:49:48.000000000 +0000 @@ -0,0 +1,4 @@ +SELECT + * +FROM + `moz-fx-data-shared-prod.telemetry.latest_versions` diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/data-observability-dev/telemetry_derived/latest_versions_v1/schema.yaml /tmp/workspace/generated-sql/sql/data-observability-dev/telemetry_derived/latest_versions_v1/schema.yaml --- /tmp/workspace/main-generated-sql/sql/data-observability-dev/telemetry_derived/latest_versions_v1/schema.yaml 1970-01-01 00:00:00.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/data-observability-dev/telemetry_derived/latest_versions_v1/schema.yaml 2024-05-27 17:49:48.000000000 +0000 @@ -0,0 +1,7 @@ +fields: +- name: channel + type: STRING + mode: NULLABLE +- name: latest_version + type: NUMERIC + mode: NULLABLE ```

Link to full diff

dataops-ci-bot commented 2 months ago

Integration report for "Merge branch 'main' into more-poc-data"

`sql.diff`

Click to expand!

```diff Only in /tmp/workspace/generated-sql/sql/data-observability-dev/telemetry_derived: latest_versions_v1 diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/data-observability-dev/telemetry_derived/latest_versions_v1/metadata.yaml /tmp/workspace/generated-sql/sql/data-observability-dev/telemetry_derived/latest_versions_v1/metadata.yaml --- /tmp/workspace/main-generated-sql/sql/data-observability-dev/telemetry_derived/latest_versions_v1/metadata.yaml 1970-01-01 00:00:00.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/data-observability-dev/telemetry_derived/latest_versions_v1/metadata.yaml 2024-05-28 16:36:31.000000000 +0000 @@ -0,0 +1,17 @@ +friendly_name: Latest Versions +description: |- + [Missing] +owners: +- ascholtz@mozilla.com +labels: + application: firefox + schedule: daily + owner1: ascholtz +bigquery: null +workgroup_access: +- role: roles/bigquery.dataViewer + members: + - workgroup:mozilla-confidential +references: + query.sql: + - moz-fx-data-shared-prod.telemetry.latest_versions diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/data-observability-dev/telemetry_derived/latest_versions_v1/query.sql /tmp/workspace/generated-sql/sql/data-observability-dev/telemetry_derived/latest_versions_v1/query.sql --- /tmp/workspace/main-generated-sql/sql/data-observability-dev/telemetry_derived/latest_versions_v1/query.sql 1970-01-01 00:00:00.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/data-observability-dev/telemetry_derived/latest_versions_v1/query.sql 2024-05-28 16:34:40.000000000 +0000 @@ -0,0 +1,4 @@ +SELECT + * +FROM + `moz-fx-data-shared-prod.telemetry.latest_versions` diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/data-observability-dev/telemetry_derived/latest_versions_v1/schema.yaml /tmp/workspace/generated-sql/sql/data-observability-dev/telemetry_derived/latest_versions_v1/schema.yaml --- /tmp/workspace/main-generated-sql/sql/data-observability-dev/telemetry_derived/latest_versions_v1/schema.yaml 1970-01-01 00:00:00.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/data-observability-dev/telemetry_derived/latest_versions_v1/schema.yaml 2024-05-28 16:34:40.000000000 +0000 @@ -0,0 +1,7 @@ +fields: +- name: channel + type: STRING + mode: NULLABLE +- name: latest_version + type: NUMERIC + mode: NULLABLE ```

Link to full diff

mozilla / bigquery-etl