mozilla / bigquery-etl

Bigquery ETL
https://mozilla.github.io/bigquery-etl
Mozilla Public License 2.0
242 stars 98 forks source link

Remove hubs service access to hubs datasets #5741

Closed whd closed 3 weeks ago

whd commented 1 month ago

This service has been decommissioned. Dataset decomm TBD.

Checklist for reviewer:

For modifications to schemas in restricted namespaces (see CODEOWNERS):

┆Issue is synchronized with this Jira Task

dataops-ci-bot commented 1 month ago

Integration report for "Remove hubs service access to hubs datasets"

sql.diff

Click to expand! ```diff diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/hubs/dataset_metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/hubs/dataset_metadata.yaml --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/hubs/dataset_metadata.yaml 2024-06-05 02:41:57.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/hubs/dataset_metadata.yaml 2024-06-05 02:41:52.000000000 +0000 @@ -8,4 +8,4 @@ - role: roles/bigquery.dataViewer members: - workgroup:mozilla-confidential - - workgroup:hubs/service + diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/hubs_derived/active_subscription_ids_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/hubs_derived/active_subscription_ids_v1/metadata.yaml --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/hubs_derived/active_subscription_ids_v1/metadata.yaml 2024-06-05 02:43:49.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/hubs_derived/active_subscription_ids_v1/metadata.yaml 2024-06-05 02:43:32.000000000 +0000 @@ -25,7 +25,6 @@ - role: roles/bigquery.dataViewer members: - workgroup:mozilla-confidential - - workgroup:hubs/service references: query.sql: - moz-fx-data-shared-prod.hubs_derived.active_subscription_ids_live diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/hubs_derived/active_subscriptions_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/hubs_derived/active_subscriptions_v1/metadata.yaml --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/hubs_derived/active_subscriptions_v1/metadata.yaml 2024-06-05 02:43:49.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/hubs_derived/active_subscriptions_v1/metadata.yaml 2024-06-05 02:43:32.000000000 +0000 @@ -24,7 +24,6 @@ - role: roles/bigquery.dataViewer members: - workgroup:mozilla-confidential - - workgroup:hubs/service references: query.sql: - moz-fx-data-shared-prod.hubs_derived.active_subscriptions_live diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/hubs_derived/dataset_metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/hubs_derived/dataset_metadata.yaml --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/hubs_derived/dataset_metadata.yaml 2024-06-05 02:41:57.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/hubs_derived/dataset_metadata.yaml 2024-06-05 02:41:52.000000000 +0000 @@ -8,4 +8,4 @@ - role: roles/bigquery.dataViewer members: - workgroup:mozilla-confidential - - workgroup:hubs/service + diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/hubs_derived/subscription_events_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/hubs_derived/subscription_events_v1/metadata.yaml --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/hubs_derived/subscription_events_v1/metadata.yaml 2024-06-05 02:43:49.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/hubs_derived/subscription_events_v1/metadata.yaml 2024-06-05 02:43:32.000000000 +0000 @@ -24,7 +24,6 @@ - role: roles/bigquery.dataViewer members: - workgroup:mozilla-confidential - - workgroup:hubs/service references: query.sql: - moz-fx-data-shared-prod.hubs_derived.subscription_events_live diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/hubs_derived/subscriptions_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/hubs_derived/subscriptions_v1/metadata.yaml --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/hubs_derived/subscriptions_v1/metadata.yaml 2024-06-05 02:43:49.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/hubs_derived/subscriptions_v1/metadata.yaml 2024-06-05 02:43:32.000000000 +0000 @@ -16,7 +16,6 @@ - role: roles/bigquery.dataViewer members: - workgroup:mozilla-confidential - - workgroup:hubs/service references: query.sql: - moz-fx-data-shared-prod.static.third_party_standardized_country_names ```

Link to full diff

dataops-ci-bot commented 1 month ago

Integration report for "Remove hubs service access to hubs datasets"

sql.diff

Click to expand! ```diff diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/hubs/dataset_metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/hubs/dataset_metadata.yaml --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/hubs/dataset_metadata.yaml 2024-06-05 13:56:11.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/hubs/dataset_metadata.yaml 2024-06-05 13:56:10.000000000 +0000 @@ -8,4 +8,3 @@ - role: roles/bigquery.dataViewer members: - workgroup:mozilla-confidential - - workgroup:hubs/service diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/hubs_derived/active_subscription_ids_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/hubs_derived/active_subscription_ids_v1/metadata.yaml --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/hubs_derived/active_subscription_ids_v1/metadata.yaml 2024-06-05 13:57:44.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/hubs_derived/active_subscription_ids_v1/metadata.yaml 2024-06-05 13:58:08.000000000 +0000 @@ -25,7 +25,6 @@ - role: roles/bigquery.dataViewer members: - workgroup:mozilla-confidential - - workgroup:hubs/service references: query.sql: - moz-fx-data-shared-prod.hubs_derived.active_subscription_ids_live diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/hubs_derived/active_subscriptions_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/hubs_derived/active_subscriptions_v1/metadata.yaml --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/hubs_derived/active_subscriptions_v1/metadata.yaml 2024-06-05 13:57:44.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/hubs_derived/active_subscriptions_v1/metadata.yaml 2024-06-05 13:58:08.000000000 +0000 @@ -24,7 +24,6 @@ - role: roles/bigquery.dataViewer members: - workgroup:mozilla-confidential - - workgroup:hubs/service references: query.sql: - moz-fx-data-shared-prod.hubs_derived.active_subscriptions_live diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/hubs_derived/dataset_metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/hubs_derived/dataset_metadata.yaml --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/hubs_derived/dataset_metadata.yaml 2024-06-05 13:56:11.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/hubs_derived/dataset_metadata.yaml 2024-06-05 13:56:10.000000000 +0000 @@ -8,4 +8,3 @@ - role: roles/bigquery.dataViewer members: - workgroup:mozilla-confidential - - workgroup:hubs/service diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/hubs_derived/subscription_events_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/hubs_derived/subscription_events_v1/metadata.yaml --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/hubs_derived/subscription_events_v1/metadata.yaml 2024-06-05 13:57:44.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/hubs_derived/subscription_events_v1/metadata.yaml 2024-06-05 13:58:08.000000000 +0000 @@ -24,7 +24,6 @@ - role: roles/bigquery.dataViewer members: - workgroup:mozilla-confidential - - workgroup:hubs/service references: query.sql: - moz-fx-data-shared-prod.hubs_derived.subscription_events_live diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/hubs_derived/subscriptions_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/hubs_derived/subscriptions_v1/metadata.yaml --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/hubs_derived/subscriptions_v1/metadata.yaml 2024-06-05 13:57:44.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/hubs_derived/subscriptions_v1/metadata.yaml 2024-06-05 13:58:08.000000000 +0000 @@ -16,7 +16,6 @@ - role: roles/bigquery.dataViewer members: - workgroup:mozilla-confidential - - workgroup:hubs/service references: query.sql: - moz-fx-data-shared-prod.static.third_party_standardized_country_names ```

Link to full diff

dataops-ci-bot commented 3 weeks ago

Integration report for "Merge branch 'main' into hubs"

sql.diff

Click to expand! ```diff diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/hubs/dataset_metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/hubs/dataset_metadata.yaml --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/hubs/dataset_metadata.yaml 2024-06-11 16:15:49.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/hubs/dataset_metadata.yaml 2024-06-11 16:15:36.000000000 +0000 @@ -8,4 +8,3 @@ - role: roles/bigquery.dataViewer members: - workgroup:mozilla-confidential - - workgroup:hubs/service diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/hubs_derived/active_subscription_ids_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/hubs_derived/active_subscription_ids_v1/metadata.yaml --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/hubs_derived/active_subscription_ids_v1/metadata.yaml 2024-06-11 16:17:22.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/hubs_derived/active_subscription_ids_v1/metadata.yaml 2024-06-11 16:17:29.000000000 +0000 @@ -25,7 +25,6 @@ - role: roles/bigquery.dataViewer members: - workgroup:mozilla-confidential - - workgroup:hubs/service references: query.sql: - moz-fx-data-shared-prod.hubs_derived.active_subscription_ids_live diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/hubs_derived/active_subscriptions_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/hubs_derived/active_subscriptions_v1/metadata.yaml --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/hubs_derived/active_subscriptions_v1/metadata.yaml 2024-06-11 16:17:22.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/hubs_derived/active_subscriptions_v1/metadata.yaml 2024-06-11 16:17:29.000000000 +0000 @@ -24,7 +24,6 @@ - role: roles/bigquery.dataViewer members: - workgroup:mozilla-confidential - - workgroup:hubs/service references: query.sql: - moz-fx-data-shared-prod.hubs_derived.active_subscriptions_live diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/hubs_derived/dataset_metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/hubs_derived/dataset_metadata.yaml --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/hubs_derived/dataset_metadata.yaml 2024-06-11 16:15:49.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/hubs_derived/dataset_metadata.yaml 2024-06-11 16:15:36.000000000 +0000 @@ -8,4 +8,3 @@ - role: roles/bigquery.dataViewer members: - workgroup:mozilla-confidential - - workgroup:hubs/service diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/hubs_derived/subscription_events_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/hubs_derived/subscription_events_v1/metadata.yaml --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/hubs_derived/subscription_events_v1/metadata.yaml 2024-06-11 16:17:22.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/hubs_derived/subscription_events_v1/metadata.yaml 2024-06-11 16:17:29.000000000 +0000 @@ -24,7 +24,6 @@ - role: roles/bigquery.dataViewer members: - workgroup:mozilla-confidential - - workgroup:hubs/service references: query.sql: - moz-fx-data-shared-prod.hubs_derived.subscription_events_live diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/hubs_derived/subscriptions_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/hubs_derived/subscriptions_v1/metadata.yaml --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/hubs_derived/subscriptions_v1/metadata.yaml 2024-06-11 16:17:22.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/hubs_derived/subscriptions_v1/metadata.yaml 2024-06-11 16:17:29.000000000 +0000 @@ -16,7 +16,6 @@ - role: roles/bigquery.dataViewer members: - workgroup:mozilla-confidential - - workgroup:hubs/service references: query.sql: - moz-fx-data-shared-prod.static.third_party_standardized_country_names ```

Link to full diff