Handle unauthenticated exception while joining collection

[pritishpai]

Changes

After successful workspace UCX installation, the installer fails with an Unauthenticated error if account admin credentials are not used. Handling the error with a warning to use the right account admin credentials and run cli command to join collection.

Linked issues

Resolves #1881

[pritishpai]

[github-actions[bot]]

❌ 196/197 passed, 1 failed, 24 skipped, 3h22m22s total

❌ test_running_real_validate_groups_permissions_job_fails: databricks.labs.blueprint.parallel.ManyError: Detected 1 failures: ResourceDoesNotExist: Scope sdk-sP0s does not exist! (3m19.461s)

``` databricks.labs.blueprint.parallel.ManyError: Detected 1 failures: ResourceDoesNotExist: Scope sdk-sP0s does not exist! [gw9] linux -- Python 3.10.14 /home/runner/work/ucx/ucx/.venv/bin/python 15:25 DEBUG [databricks.labs.ucx.mixins.fixtures] added workspace user fixture: User(active=True, display_name='sdk-zijvsfpo@example.com', emails=[ComplexValue(display=None, primary=True, ref=None, type='work', value='sdk-zijvsfpo@example.com')], entitlements=[], external_id=None, groups=[], id='513135327215568', name=Name(family_name=None, given_name='sdk-zijvsfpo@example.com'), roles=[], schemas=[, ], user_name='sdk-zijvsfpo@example.com') 15:25 INFO [databricks.labs.ucx.mixins.fixtures] Workspace group ucx_GqaTY: https://DATABRICKS_HOST#setting/accounts/groups/791933909540995 15:25 DEBUG [databricks.labs.ucx.mixins.fixtures] added workspace group fixture: Group(display_name='ucx_GqaTY', entitlements=[ComplexValue(display=None, primary=None, ref=None, type=None, value='allow-cluster-create')], external_id=None, groups=[], id='791933909540995', members=[ComplexValue(display='sdk-zijvsfpo@example.com', primary=None, ref='Users/513135327215568', type=None, value='513135327215568')], meta=ResourceMeta(resource_type='WorkspaceGroup'), roles=[], schemas=[]) 15:25 INFO [databricks.labs.ucx.mixins.fixtures] Account group ucx_GqaTY: https://accounts.CLOUD_ENVdatabricks.net/users/groups/198526173027308/members 15:25 DEBUG [databricks.labs.ucx.mixins.fixtures] added account group fixture: Group(display_name='ucx_GqaTY', entitlements=[], external_id=None, groups=[], id='198526173027308', members=[ComplexValue(display='sdk-zijvsfpo@example.com', primary=None, ref='Users/513135327215568', type=None, value='513135327215568')], meta=None, roles=[], schemas=[]) 15:25 INFO [databricks.labs.ucx.mixins.fixtures] Cluster policy: https://DATABRICKS_HOST#setting/clusters/cluster-policies/view/0007300A06F8D36F 15:25 DEBUG [databricks.labs.ucx.mixins.fixtures] added cluster policy fixture: CreatePolicyResponse(policy_id='0007300A06F8D36F') 15:25 DEBUG [databricks.labs.ucx.mixins.fixtures] added cluster_policy permissions fixture: 0007300A06F8D36F [group_name admins CAN_USE] -> [group_name ucx_GqaTY CAN_USE] 15:25 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_sm5j5: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_sm5j5 15:25 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_sm5j5', metastore_id=None, name='ucx_sm5j5', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 15:25 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.bTLZ/config.yml) doesn't exist. 15:25 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration 15:25 INFO [databricks.labs.ucx.installer.hms_lineage] HMS Lineage feature creates one system table named system.hms_to_uc_migration.table_access and helps in your migration process from HMS to UC by allowing you to programmatically query HMS lineage data. 15:25 INFO [databricks.labs.ucx.install] Fetching installations... 15:26 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy. 15:26 DEBUG [tests.integration.conftest] Waiting for clusters to start... 15:26 DEBUG [tests.integration.conftest] Waiting for clusters to start... 15:26 INFO [databricks.labs.ucx.install] Installing UCX v0.27.2+1520240701152604 15:26 INFO [databricks.labs.ucx.install] Creating ucx schemas... 15:26 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-tables-ctas 15:26 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=scan-tables-in-mounts-experimental 15:26 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions 15:26 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=failing 15:26 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-experimental 15:26 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-hiveserde-tables-in-place-experimental 15:26 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups 15:26 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment 15:26 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental 15:26 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables 15:26 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups 15:26 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-data-reconciliation 15:26 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=experimental-workflow-linter 15:26 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.bTLZ/README for the next steps. 15:26 DEBUG [databricks.labs.ucx.workspace_access.manager] Crawling permissions 15:26 INFO [databricks.labs.ucx.workspace_access.generic] Listed clusters in 0:00:00.079650 15:26 INFO [databricks.labs.ucx.workspace_access.generic] Listed cluster-policies in 0:00:00.056119 15:26 INFO [databricks.labs.ucx.workspace_access.generic] Listed instance-pools in 0:00:00.074174 15:26 INFO [databricks.labs.ucx.workspace_access.generic] Listed sql/warehouses in 0:00:00.050817 15:26 INFO [databricks.labs.ucx.workspace_access.generic] Listed jobs in 0:00:00.472200 15:26 INFO [databricks.labs.ucx.workspace_access.generic] Listed pipelines in 0:00:00.108126 15:26 INFO [databricks.labs.ucx.workspace_access.generic] Listed serving-endpoints in 0:00:01.112384 15:26 INFO [databricks.labs.ucx.workspace_access.generic] Listed experiments in 0:00:00.850954 15:26 INFO [databricks.labs.ucx.workspace_access.generic] Listed registered-models in 0:00:00.304880 15:26 INFO [databricks.labs.ucx.workspace_access.generic] Listed registered-models in 0:00:00.000005 15:26 INFO [databricks.labs.ucx.workspace_access.generic] Listed authorization in 0:00:00.000006 15:26 INFO [databricks.labs.ucx.workspace_access.generic] Listed feature-tables in 0:00:00.052539 15:26 INFO [databricks.labs.ucx.workspace_access.generic] Listed feature-tables in 0:00:00.000007 15:26 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.ucx_sm5j5.workspace_objects] fetching workspace_objects inventory 15:26 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.ucx_sm5j5.workspace_objects] crawling new batch for workspace_objects 15:26 INFO [databricks.labs.ucx.workspace_access.listing] Recursive WorkspaceFS listing started at 2024-07-01 15:26:55.505259 15:26 DEBUG [databricks.labs.ucx.workspace_access.listing] Listed /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.bTLZ, found 1 sub-directories and 6 other objects 15:26 DEBUG [databricks.labs.ucx.workspace_access.listing] Listed /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.bTLZ/wheels, found 0 sub-directories and 2 other objects 15:26 INFO [databricks.labs.ucx.workspace_access.listing] Recursive WorkspaceFS listing finished at 2024-07-01 15:26:55.756570. Total time taken for workspace listing: 0:00:00.251322 15:26 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.ucx_sm5j5.workspace_objects] found 10 new records for workspace_objects 15:26 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.ucx_sm5j5.grants] fetching grants inventory 15:26 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.ucx_sm5j5.grants] crawling new batch for grants 15:26 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.ucx_sm5j5.tables] fetching tables inventory 15:27 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.ucx_sm5j5.tables] crawling new batch for tables 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.TEST_SCHEMA] listing tables 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_s0xw9] listing tables 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_s7hty] listing tables 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sat56] listing tables 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sczcn] listing tables 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_si3gx] listing tables 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sjjk7] listing tables 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_slp37] listing tables 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sltvu] listing tables 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sm5j5] listing tables 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sm5kr] listing tables 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sm8ar] listing tables 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sp52d] listing tables 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sqvye] listing tables 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_strbu] listing tables 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_svdux] listing tables 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_svvq8] listing tables 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sx8m0] listing tables 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sxy2l] listing tables 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sxzcf] listing tables 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.TEST_SCHEMA.ajs_grant_testing] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.TEST_SCHEMA.ajs_view_grant_testing] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.TEST_SCHEMA.ajs_view_testing] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.TEST_SCHEMA.foo] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.TEST_SCHEMA.some] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.TEST_SCHEMA.ucx_t08ek] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.TEST_SCHEMA.ucx_t7mgw] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.TEST_SCHEMA.ucx_tiixw] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.TEST_SCHEMA.ucx_tkbmh] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.TEST_SCHEMA.ucx_tkgro] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.TEST_SCHEMA.ucx_tlodc] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.TEST_SCHEMA.ucx_tloia] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.TEST_SCHEMA.ucx_tr11d] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.TEST_SCHEMA.ucx_tspzh] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.TEST_SCHEMA.ucx_twblz] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.TEST_SCHEMA.ucx_twmnr] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.TEST_SCHEMA.ucx_ty0j0] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_s0xw9.ucx_tapqi] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_s7hty.CLOUD_ENV_service_principals] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_s7hty.clusters] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_s7hty.code_patterns] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_s7hty.external_locations] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_s7hty.global_init_scripts] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_s7hty.grant_detail] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_s7hty.grants] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_s7hty.groups] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_s7hty.jobs] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_s7hty.logs] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_s7hty.migration_status] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_s7hty.misc_patterns] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_s7hty.mounts] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_s7hty.objects] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_s7hty.permissions] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_s7hty.pipelines] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_s7hty.policies] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_s7hty.recon_results] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_s7hty.reconciliation_results] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_s7hty.submit_runs] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_s7hty.table_estimates] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_s7hty.table_failures] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_s7hty.table_size] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_s7hty.tables] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_s7hty.udfs] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_s7hty.workflow_problems] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_s7hty.workspace_objects] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_si3gx.CLOUD_ENV_service_principals] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_si3gx.clusters] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_si3gx.code_patterns] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_si3gx.external_locations] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_si3gx.global_init_scripts] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_si3gx.grant_detail] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_si3gx.grants] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_si3gx.groups] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_si3gx.jobs] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_si3gx.logs] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_si3gx.migration_status] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_si3gx.misc_patterns] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_si3gx.mounts] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_si3gx.objects] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_si3gx.permissions] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_si3gx.pipelines] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_si3gx.policies] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_si3gx.recon_results] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_si3gx.reconciliation_results] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_si3gx.submit_runs] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_si3gx.table_estimates] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_si3gx.table_failures] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_si3gx.table_size] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_si3gx.tables] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_si3gx.udfs] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_si3gx.workflow_problems] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_si3gx.workspace_objects] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sjjk7.ucx_t8iua] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_slp37.ucx_tdzwv] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sltvu.tables] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sltvu.udfs] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sm5j5.CLOUD_ENV_service_principals] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sm5j5.clusters] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sm5j5.code_patterns] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sm5j5.external_locations] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sm5j5.global_init_scripts] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sm5j5.grant_detail] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sm5j5.grants] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sm5j5.groups] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sm5j5.jobs] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sm5j5.logs] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sm5j5.migration_status] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sm5j5.misc_patterns] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sm5j5.mounts] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sm5j5.objects] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sm5j5.permissions] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sm5j5.pipelines] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sm5j5.policies] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sm5j5.recon_results] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sm5j5.reconciliation_results] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sm5j5.submit_runs] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sm5j5.table_estimates] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sm5j5.table_failures] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sm5j5.table_size] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sm5j5.tables] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sm5j5.udfs] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sm5j5.workflow_problems] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sm5j5.workspace_objects] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sp52d.groups] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sqvye.CLOUD_ENV_service_principals] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sqvye.clusters] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sqvye.code_patterns] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sqvye.external_locations] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sqvye.global_init_scripts] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sqvye.grant_detail] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sqvye.grants] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sqvye.groups] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sqvye.jobs] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sqvye.logs] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sqvye.migration_status] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sqvye.misc_patterns] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sqvye.mounts] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sqvye.objects] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sqvye.permissions] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sqvye.pipelines] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sqvye.policies] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sqvye.recon_results] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sqvye.reconciliation_results] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sqvye.submit_runs] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sqvye.table_estimates] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sqvye.table_failures] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sqvye.table_size] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sqvye.tables] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sqvye.udfs] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sqvye.workflow_problems] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sqvye.workspace_objects] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_strbu.ucx_tujaz] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_svvq8.groups] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sxy2l.grants] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sxy2l.tables] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sxy2l.udfs] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sxzcf.ucx_tomvu] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sxzcf.ucx_tst2z] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.ucx_sm5j5.tables] found 138 new records for tables 15:27 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.ucx_sm5j5.udfs] fetching udfs inventory 15:27 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.ucx_sm5j5.udfs] crawling new batch for udfs 15:27 DEBUG [databricks.labs.ucx.hive_metastore.udfs] [hive_metastore.TEST_SCHEMA] listing udfs 15:27 DEBUG [databricks.labs.ucx.hive_metastore.udfs] [hive_metastore.ucx_s0xw9] listing udfs 15:27 DEBUG [databricks.labs.ucx.hive_metastore.udfs] [hive_metastore.ucx_s7hty] listing udfs 15:27 DEBUG [databricks.labs.ucx.hive_metastore.udfs] [hive_metastore.ucx_sat56] listing udfs 15:27 DEBUG [databricks.labs.ucx.hive_metastore.udfs] [hive_metastore.ucx_sczcn] listing udfs 15:27 DEBUG [databricks.labs.ucx.hive_metastore.udfs] [hive_metastore.ucx_si3gx] listing udfs 15:27 DEBUG [databricks.labs.ucx.hive_metastore.udfs] [hive_metastore.ucx_sjjk7] listing udfs 15:27 DEBUG [databricks.labs.ucx.hive_metastore.udfs] [hive_metastore.ucx_slp37] listing udfs 15:27 DEBUG [databricks.labs.ucx.hive_metastore.udfs] [hive_metastore.ucx_sltvu] listing udfs 15:27 DEBUG [databricks.labs.ucx.hive_metastore.udfs] [hive_metastore.ucx_sm5j5] listing udfs 15:27 DEBUG [databricks.labs.ucx.hive_metastore.udfs] [hive_metastore.ucx_sm5kr] listing udfs 15:27 DEBUG [databricks.labs.ucx.hive_metastore.udfs] [hive_metastore.ucx_sm8ar] listing udfs 15:27 DEBUG [databricks.labs.ucx.hive_metastore.udfs] [hive_metastore.ucx_sp52d] listing udfs 15:27 DEBUG [databricks.labs.ucx.hive_metastore.udfs] [hive_metastore.ucx_sqvye] listing udfs 15:27 DEBUG [databricks.labs.ucx.hive_metastore.udfs] [hive_metastore.ucx_strbu] listing udfs 15:27 DEBUG [databricks.labs.ucx.hive_metastore.udfs] [hive_metastore.ucx_svdux] listing udfs 15:27 DEBUG [databricks.labs.ucx.hive_metastore.udfs] [hive_metastore.ucx_svvq8] listing udfs 15:27 DEBUG [databricks.labs.ucx.hive_metastore.udfs] [hive_metastore.ucx_sx8m0] listing udfs 15:27 DEBUG [databricks.labs.ucx.hive_metastore.udfs] [hive_metastore.ucx_sxy2l] listing udfs 15:27 DEBUG [databricks.labs.ucx.hive_metastore.udfs] [hive_metastore.ucx_sxzcf] listing udfs 15:27 DEBUG [databricks.labs.ucx.hive_metastore.udfs] [hive_metastore.TEST_SCHEMA.convert_f_to_c] fetching udf metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.udfs] [hive_metastore.TEST_SCHEMA.hiveudaf] fetching udf metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.udfs] [hive_metastore.TEST_SCHEMA.hiveudtf] fetching udf metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.udfs] [hive_metastore.TEST_SCHEMA.testudf] fetching udf metadata 15:27 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.ucx_sm5j5.udfs] found 4 new records for udfs 15:27 WARNING [databricks.labs.ucx.hive_metastore.grants] Schema hive_metastore.ucx_sx8m0 no longer existed 15:27 WARNING [databricks.labs.ucx.hive_metastore.grants] Schema hive_metastore.ucx_sp52d no longer existed 15:28 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.ucx_sm5j5.grants] found 164 new records for grants 15:28 INFO [databricks.labs.ucx.workspace_access.manager] Starting to crawl permissions. Total tasks: 2773 15:28 WARNING [databricks.labs.ucx.workspace_access.generic] jobs or cluster removed on backend: serving-endpoints None 15:28 WARNING [databricks.labs.ucx.workspace_access.generic] Object serving-endpoints None doesn't have any permissions 15:28 WARNING [databricks.labs.ucx.workspace_access.generic] jobs or cluster removed on backend: serving-endpoints None 15:28 WARNING [databricks.labs.ucx.workspace_access.generic] Object serving-endpoints None doesn't have any permissions 15:28 WARNING [databricks.labs.ucx.workspace_access.generic] jobs or cluster removed on backend: serving-endpoints None 15:28 WARNING [databricks.labs.ucx.workspace_access.generic] Object serving-endpoints None doesn't have any permissions 15:28 WARNING [databricks.labs.ucx.workspace_access.generic] jobs or cluster removed on backend: serving-endpoints None 15:28 WARNING [databricks.labs.ucx.workspace_access.generic] Object serving-endpoints None doesn't have any permissions 15:28 WARNING [databricks.labs.ucx.workspace_access.generic] jobs or cluster removed on backend: serving-endpoints None 15:28 WARNING [databricks.labs.ucx.workspace_access.generic] Object serving-endpoints None doesn't have any permissions 15:28 WARNING [databricks.labs.ucx.workspace_access.generic] jobs or cluster removed on backend: serving-endpoints None 15:28 WARNING [databricks.labs.ucx.workspace_access.generic] Object serving-endpoints None doesn't have any permissions 15:28 WARNING [databricks.labs.ucx.workspace_access.generic] jobs or cluster removed on backend: serving-endpoints None 15:28 WARNING [databricks.labs.ucx.workspace_access.generic] Object serving-endpoints None doesn't have any permissions 15:28 WARNING [databricks.labs.ucx.workspace_access.generic] removed on backend: authorization passwords 15:28 WARNING [databricks.labs.ucx.workspace_access.generic] Object authorization passwords doesn't have any permissions 15:28 WARNING [databricks.labs.ucx.workspace_access.generic] jobs or cluster removed on backend: serving-endpoints None 15:28 WARNING [databricks.labs.ucx.workspace_access.generic] Object serving-endpoints None doesn't have any permissions 15:28 ERROR [databricks.labs.blueprint.parallel] crawl permissions(SecretScope(backend_type=, keyvault_metadata=None, name='sdk-sP0s')) task failed: Scope sdk-sP0s does not exist! Traceback (most recent call last): File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/labs/blueprint/parallel.py", line 158, in inner return func(*args, **kwargs), None File "/home/runner/work/ucx/ucx/src/databricks/labs/ucx/workspace_access/secrets.py", line 37, in _crawler_task acl_items = self._ws.secrets.list_acls(scope.name) File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/service/workspace.py", line 1995, in list_acls json = self._api.do('GET', '/api/2.0/secrets/acls/list', query=query, headers=headers) File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/core.py", line 131, in do response = retryable(self._perform)(method, File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/retries.py", line 54, in wrapper raise err File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/retries.py", line 33, in wrapper return func(*args, **kwargs) File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/core.py", line 242, in _perform raise self._make_nicer_error(response=response, **payload) from None databricks.sdk.errors.platform.ResourceDoesNotExist: Scope sdk-sP0s does not exist! 15:28 WARNING [databricks.labs.blueprint.parallel] Some 'crawl permissions' tasks failed: 100% results available (2763/2773). Took 0:00:34.112482 15:28 ERROR [databricks.labs.ucx.workspace_access.manager] Error while crawling permissions: Scope sdk-sP0s does not exist! 15:25 DEBUG [databricks.labs.ucx.mixins.fixtures] added workspace user fixture: User(active=True, display_name='sdk-zijvsfpo@example.com', emails=[ComplexValue(display=None, primary=True, ref=None, type='work', value='sdk-zijvsfpo@example.com')], entitlements=[], external_id=None, groups=[], id='513135327215568', name=Name(family_name=None, given_name='sdk-zijvsfpo@example.com'), roles=[], schemas=[, ], user_name='sdk-zijvsfpo@example.com') 15:25 INFO [databricks.labs.ucx.mixins.fixtures] Workspace group ucx_GqaTY: https://DATABRICKS_HOST#setting/accounts/groups/791933909540995 15:25 DEBUG [databricks.labs.ucx.mixins.fixtures] added workspace group fixture: Group(display_name='ucx_GqaTY', entitlements=[ComplexValue(display=None, primary=None, ref=None, type=None, value='allow-cluster-create')], external_id=None, groups=[], id='791933909540995', members=[ComplexValue(display='sdk-zijvsfpo@example.com', primary=None, ref='Users/513135327215568', type=None, value='513135327215568')], meta=ResourceMeta(resource_type='WorkspaceGroup'), roles=[], schemas=[]) 15:25 INFO [databricks.labs.ucx.mixins.fixtures] Account group ucx_GqaTY: https://accounts.CLOUD_ENVdatabricks.net/users/groups/198526173027308/members 15:25 DEBUG [databricks.labs.ucx.mixins.fixtures] added account group fixture: Group(display_name='ucx_GqaTY', entitlements=[], external_id=None, groups=[], id='198526173027308', members=[ComplexValue(display='sdk-zijvsfpo@example.com', primary=None, ref='Users/513135327215568', type=None, value='513135327215568')], meta=None, roles=[], schemas=[]) 15:25 INFO [databricks.labs.ucx.mixins.fixtures] Cluster policy: https://DATABRICKS_HOST#setting/clusters/cluster-policies/view/0007300A06F8D36F 15:25 DEBUG [databricks.labs.ucx.mixins.fixtures] added cluster policy fixture: CreatePolicyResponse(policy_id='0007300A06F8D36F') 15:25 DEBUG [databricks.labs.ucx.mixins.fixtures] added cluster_policy permissions fixture: 0007300A06F8D36F [group_name admins CAN_USE] -> [group_name ucx_GqaTY CAN_USE] 15:25 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_sm5j5: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_sm5j5 15:25 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_sm5j5', metastore_id=None, name='ucx_sm5j5', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 15:25 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.bTLZ/config.yml) doesn't exist. 15:25 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration 15:25 INFO [databricks.labs.ucx.installer.hms_lineage] HMS Lineage feature creates one system table named system.hms_to_uc_migration.table_access and helps in your migration process from HMS to UC by allowing you to programmatically query HMS lineage data. 15:25 INFO [databricks.labs.ucx.install] Fetching installations... 15:26 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy. 15:26 DEBUG [tests.integration.conftest] Waiting for clusters to start... 15:26 DEBUG [tests.integration.conftest] Waiting for clusters to start... 15:26 INFO [databricks.labs.ucx.install] Installing UCX v0.27.2+1520240701152604 15:26 INFO [databricks.labs.ucx.install] Creating ucx schemas... 15:26 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-tables-ctas 15:26 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=scan-tables-in-mounts-experimental 15:26 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions 15:26 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=failing 15:26 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-experimental 15:26 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-hiveserde-tables-in-place-experimental 15:26 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups 15:26 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment 15:26 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental 15:26 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables 15:26 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups 15:26 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-data-reconciliation 15:26 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=experimental-workflow-linter 15:26 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.bTLZ/README for the next steps. 15:26 DEBUG [databricks.labs.ucx.workspace_access.manager] Crawling permissions 15:26 INFO [databricks.labs.ucx.workspace_access.generic] Listed clusters in 0:00:00.079650 15:26 INFO [databricks.labs.ucx.workspace_access.generic] Listed cluster-policies in 0:00:00.056119 15:26 INFO [databricks.labs.ucx.workspace_access.generic] Listed instance-pools in 0:00:00.074174 15:26 INFO [databricks.labs.ucx.workspace_access.generic] Listed sql/warehouses in 0:00:00.050817 15:26 INFO [databricks.labs.ucx.workspace_access.generic] Listed jobs in 0:00:00.472200 15:26 INFO [databricks.labs.ucx.workspace_access.generic] Listed pipelines in 0:00:00.108126 15:26 INFO [databricks.labs.ucx.workspace_access.generic] Listed serving-endpoints in 0:00:01.112384 15:26 INFO [databricks.labs.ucx.workspace_access.generic] Listed experiments in 0:00:00.850954 15:26 INFO [databricks.labs.ucx.workspace_access.generic] Listed registered-models in 0:00:00.304880 15:26 INFO [databricks.labs.ucx.workspace_access.generic] Listed registered-models in 0:00:00.000005 15:26 INFO [databricks.labs.ucx.workspace_access.generic] Listed authorization in 0:00:00.000006 15:26 INFO [databricks.labs.ucx.workspace_access.generic] Listed feature-tables in 0:00:00.052539 15:26 INFO [databricks.labs.ucx.workspace_access.generic] Listed feature-tables in 0:00:00.000007 15:26 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.ucx_sm5j5.workspace_objects] fetching workspace_objects inventory 15:26 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.ucx_sm5j5.workspace_objects] crawling new batch for workspace_objects 15:26 INFO [databricks.labs.ucx.workspace_access.listing] Recursive WorkspaceFS listing started at 2024-07-01 15:26:55.505259 15:26 DEBUG [databricks.labs.ucx.workspace_access.listing] Listed /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.bTLZ, found 1 sub-directories and 6 other objects 15:26 DEBUG [databricks.labs.ucx.workspace_access.listing] Listed /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.bTLZ/wheels, found 0 sub-directories and 2 other objects 15:26 INFO [databricks.labs.ucx.workspace_access.listing] Recursive WorkspaceFS listing finished at 2024-07-01 15:26:55.756570. Total time taken for workspace listing: 0:00:00.251322 15:26 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.ucx_sm5j5.workspace_objects] found 10 new records for workspace_objects 15:26 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.ucx_sm5j5.grants] fetching grants inventory 15:26 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.ucx_sm5j5.grants] crawling new batch for grants 15:26 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.ucx_sm5j5.tables] fetching tables inventory 15:27 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.ucx_sm5j5.tables] crawling new batch for tables 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.TEST_SCHEMA] listing tables 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_s0xw9] listing tables 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_s7hty] listing tables 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sat56] listing tables 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sczcn] listing tables 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_si3gx] listing tables 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sjjk7] listing tables 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_slp37] listing tables 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sltvu] listing tables 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sm5j5] listing tables 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sm5kr] listing tables 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sm8ar] listing tables 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sp52d] listing tables 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sqvye] listing tables 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_strbu] listing tables 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_svdux] listing tables 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_svvq8] listing tables 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sx8m0] listing tables 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sxy2l] listing tables 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sxzcf] listing tables 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.TEST_SCHEMA.ajs_grant_testing] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.TEST_SCHEMA.ajs_view_grant_testing] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.TEST_SCHEMA.ajs_view_testing] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.TEST_SCHEMA.foo] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.TEST_SCHEMA.some] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.TEST_SCHEMA.ucx_t08ek] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.TEST_SCHEMA.ucx_t7mgw] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.TEST_SCHEMA.ucx_tiixw] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.TEST_SCHEMA.ucx_tkbmh] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.TEST_SCHEMA.ucx_tkgro] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.TEST_SCHEMA.ucx_tlodc] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.TEST_SCHEMA.ucx_tloia] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.TEST_SCHEMA.ucx_tr11d] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.TEST_SCHEMA.ucx_tspzh] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.TEST_SCHEMA.ucx_twblz] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.TEST_SCHEMA.ucx_twmnr] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.TEST_SCHEMA.ucx_ty0j0] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_s0xw9.ucx_tapqi] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_s7hty.CLOUD_ENV_service_principals] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_s7hty.clusters] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_s7hty.code_patterns] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_s7hty.external_locations] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_s7hty.global_init_scripts] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_s7hty.grant_detail] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_s7hty.grants] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_s7hty.groups] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_s7hty.jobs] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_s7hty.logs] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_s7hty.migration_status] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_s7hty.misc_patterns] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_s7hty.mounts] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_s7hty.objects] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_s7hty.permissions] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_s7hty.pipelines] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_s7hty.policies] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_s7hty.recon_results] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_s7hty.reconciliation_results] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_s7hty.submit_runs] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_s7hty.table_estimates] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_s7hty.table_failures] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_s7hty.table_size] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_s7hty.tables] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_s7hty.udfs] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_s7hty.workflow_problems] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_s7hty.workspace_objects] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_si3gx.CLOUD_ENV_service_principals] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_si3gx.clusters] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_si3gx.code_patterns] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_si3gx.external_locations] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_si3gx.global_init_scripts] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_si3gx.grant_detail] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_si3gx.grants] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_si3gx.groups] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_si3gx.jobs] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_si3gx.logs] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_si3gx.migration_status] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_si3gx.misc_patterns] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_si3gx.mounts] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_si3gx.objects] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_si3gx.permissions] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_si3gx.pipelines] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_si3gx.policies] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_si3gx.recon_results] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_si3gx.reconciliation_results] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_si3gx.submit_runs] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_si3gx.table_estimates] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_si3gx.table_failures] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_si3gx.table_size] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_si3gx.tables] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_si3gx.udfs] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_si3gx.workflow_problems] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_si3gx.workspace_objects] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sjjk7.ucx_t8iua] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_slp37.ucx_tdzwv] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sltvu.tables] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sltvu.udfs] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sm5j5.CLOUD_ENV_service_principals] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sm5j5.clusters] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sm5j5.code_patterns] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sm5j5.external_locations] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sm5j5.global_init_scripts] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sm5j5.grant_detail] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sm5j5.grants] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sm5j5.groups] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sm5j5.jobs] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sm5j5.logs] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sm5j5.migration_status] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sm5j5.misc_patterns] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sm5j5.mounts] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sm5j5.objects] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sm5j5.permissions] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sm5j5.pipelines] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sm5j5.policies] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sm5j5.recon_results] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sm5j5.reconciliation_results] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sm5j5.submit_runs] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sm5j5.table_estimates] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sm5j5.table_failures] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sm5j5.table_size] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sm5j5.tables] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sm5j5.udfs] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sm5j5.workflow_problems] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sm5j5.workspace_objects] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sp52d.groups] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sqvye.CLOUD_ENV_service_principals] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sqvye.clusters] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sqvye.code_patterns] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sqvye.external_locations] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sqvye.global_init_scripts] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sqvye.grant_detail] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sqvye.grants] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sqvye.groups] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sqvye.jobs] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sqvye.logs] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sqvye.migration_status] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sqvye.misc_patterns] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sqvye.mounts] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sqvye.objects] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sqvye.permissions] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sqvye.pipelines] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sqvye.policies] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sqvye.recon_results] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sqvye.reconciliation_results] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sqvye.submit_runs] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sqvye.table_estimates] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sqvye.table_failures] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sqvye.table_size] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sqvye.tables] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sqvye.udfs] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sqvye.workflow_problems] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sqvye.workspace_objects] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_strbu.ucx_tujaz] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_svvq8.groups] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sxy2l.grants] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sxy2l.tables] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sxy2l.udfs] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sxzcf.ucx_tomvu] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sxzcf.ucx_tst2z] fetching table metadata 15:27 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.ucx_sm5j5.tables] found 138 new records for tables 15:27 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.ucx_sm5j5.udfs] fetching udfs inventory 15:27 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.ucx_sm5j5.udfs] crawling new batch for udfs 15:27 DEBUG [databricks.labs.ucx.hive_metastore.udfs] [hive_metastore.TEST_SCHEMA] listing udfs 15:27 DEBUG [databricks.labs.ucx.hive_metastore.udfs] [hive_metastore.ucx_s0xw9] listing udfs 15:27 DEBUG [databricks.labs.ucx.hive_metastore.udfs] [hive_metastore.ucx_s7hty] listing udfs 15:27 DEBUG [databricks.labs.ucx.hive_metastore.udfs] [hive_metastore.ucx_sat56] listing udfs 15:27 DEBUG [databricks.labs.ucx.hive_metastore.udfs] [hive_metastore.ucx_sczcn] listing udfs 15:27 DEBUG [databricks.labs.ucx.hive_metastore.udfs] [hive_metastore.ucx_si3gx] listing udfs 15:27 DEBUG [databricks.labs.ucx.hive_metastore.udfs] [hive_metastore.ucx_sjjk7] listing udfs 15:27 DEBUG [databricks.labs.ucx.hive_metastore.udfs] [hive_metastore.ucx_slp37] listing udfs 15:27 DEBUG [databricks.labs.ucx.hive_metastore.udfs] [hive_metastore.ucx_sltvu] listing udfs 15:27 DEBUG [databricks.labs.ucx.hive_metastore.udfs] [hive_metastore.ucx_sm5j5] listing udfs 15:27 DEBUG [databricks.labs.ucx.hive_metastore.udfs] [hive_metastore.ucx_sm5kr] listing udfs 15:27 DEBUG [databricks.labs.ucx.hive_metastore.udfs] [hive_metastore.ucx_sm8ar] listing udfs 15:27 DEBUG [databricks.labs.ucx.hive_metastore.udfs] [hive_metastore.ucx_sp52d] listing udfs 15:27 DEBUG [databricks.labs.ucx.hive_metastore.udfs] [hive_metastore.ucx_sqvye] listing udfs 15:27 DEBUG [databricks.labs.ucx.hive_metastore.udfs] [hive_metastore.ucx_strbu] listing udfs 15:27 DEBUG [databricks.labs.ucx.hive_metastore.udfs] [hive_metastore.ucx_svdux] listing udfs 15:27 DEBUG [databricks.labs.ucx.hive_metastore.udfs] [hive_metastore.ucx_svvq8] listing udfs 15:27 DEBUG [databricks.labs.ucx.hive_metastore.udfs] [hive_metastore.ucx_sx8m0] listing udfs 15:27 DEBUG [databricks.labs.ucx.hive_metastore.udfs] [hive_metastore.ucx_sxy2l] listing udfs 15:27 DEBUG [databricks.labs.ucx.hive_metastore.udfs] [hive_metastore.ucx_sxzcf] listing udfs 15:27 DEBUG [databricks.labs.ucx.hive_metastore.udfs] [hive_metastore.TEST_SCHEMA.convert_f_to_c] fetching udf metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.udfs] [hive_metastore.TEST_SCHEMA.hiveudaf] fetching udf metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.udfs] [hive_metastore.TEST_SCHEMA.hiveudtf] fetching udf metadata 15:27 DEBUG [databricks.labs.ucx.hive_metastore.udfs] [hive_metastore.TEST_SCHEMA.testudf] fetching udf metadata 15:27 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.ucx_sm5j5.udfs] found 4 new records for udfs 15:27 WARNING [databricks.labs.ucx.hive_metastore.grants] Schema hive_metastore.ucx_sx8m0 no longer existed 15:27 WARNING [databricks.labs.ucx.hive_metastore.grants] Schema hive_metastore.ucx_sp52d no longer existed 15:28 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.ucx_sm5j5.grants] found 164 new records for grants 15:28 INFO [databricks.labs.ucx.workspace_access.manager] Starting to crawl permissions. Total tasks: 2773 15:28 WARNING [databricks.labs.ucx.workspace_access.generic] jobs or cluster removed on backend: serving-endpoints None 15:28 WARNING [databricks.labs.ucx.workspace_access.generic] Object serving-endpoints None doesn't have any permissions 15:28 WARNING [databricks.labs.ucx.workspace_access.generic] jobs or cluster removed on backend: serving-endpoints None 15:28 WARNING [databricks.labs.ucx.workspace_access.generic] Object serving-endpoints None doesn't have any permissions 15:28 WARNING [databricks.labs.ucx.workspace_access.generic] jobs or cluster removed on backend: serving-endpoints None 15:28 WARNING [databricks.labs.ucx.workspace_access.generic] Object serving-endpoints None doesn't have any permissions 15:28 WARNING [databricks.labs.ucx.workspace_access.generic] jobs or cluster removed on backend: serving-endpoints None 15:28 WARNING [databricks.labs.ucx.workspace_access.generic] Object serving-endpoints None doesn't have any permissions 15:28 WARNING [databricks.labs.ucx.workspace_access.generic] jobs or cluster removed on backend: serving-endpoints None 15:28 WARNING [databricks.labs.ucx.workspace_access.generic] Object serving-endpoints None doesn't have any permissions 15:28 WARNING [databricks.labs.ucx.workspace_access.generic] jobs or cluster removed on backend: serving-endpoints None 15:28 WARNING [databricks.labs.ucx.workspace_access.generic] Object serving-endpoints None doesn't have any permissions 15:28 WARNING [databricks.labs.ucx.workspace_access.generic] jobs or cluster removed on backend: serving-endpoints None 15:28 WARNING [databricks.labs.ucx.workspace_access.generic] Object serving-endpoints None doesn't have any permissions 15:28 WARNING [databricks.labs.ucx.workspace_access.generic] removed on backend: authorization passwords 15:28 WARNING [databricks.labs.ucx.workspace_access.generic] Object authorization passwords doesn't have any permissions 15:28 WARNING [databricks.labs.ucx.workspace_access.generic] jobs or cluster removed on backend: serving-endpoints None 15:28 WARNING [databricks.labs.ucx.workspace_access.generic] Object serving-endpoints None doesn't have any permissions 15:28 ERROR [databricks.labs.blueprint.parallel] crawl permissions(SecretScope(backend_type=, keyvault_metadata=None, name='sdk-sP0s')) task failed: Scope sdk-sP0s does not exist! Traceback (most recent call last): File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/labs/blueprint/parallel.py", line 158, in inner return func(*args, **kwargs), None File "/home/runner/work/ucx/ucx/src/databricks/labs/ucx/workspace_access/secrets.py", line 37, in _crawler_task acl_items = self._ws.secrets.list_acls(scope.name) File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/service/workspace.py", line 1995, in list_acls json = self._api.do('GET', '/api/2.0/secrets/acls/list', query=query, headers=headers) File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/core.py", line 131, in do response = retryable(self._perform)(method, File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/retries.py", line 54, in wrapper raise err File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/retries.py", line 33, in wrapper return func(*args, **kwargs) File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/core.py", line 242, in _perform raise self._make_nicer_error(response=response, **payload) from None databricks.sdk.errors.platform.ResourceDoesNotExist: Scope sdk-sP0s does not exist! 15:28 WARNING [databricks.labs.blueprint.parallel] Some 'crawl permissions' tasks failed: 100% results available (2763/2773). Took 0:00:34.112482 15:28 ERROR [databricks.labs.ucx.workspace_access.manager] Error while crawling permissions: Scope sdk-sP0s does not exist! 15:28 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 cluster_policy permissions fixtures 15:28 DEBUG [databricks.labs.ucx.mixins.fixtures] removing cluster_policy permissions fixture: 0007300A06F8D36F [group_name admins CAN_USE] -> [group_name ucx_GqaTY CAN_USE] 15:28 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 cluster policy fixtures 15:28 DEBUG [databricks.labs.ucx.mixins.fixtures] removing cluster policy fixture: CreatePolicyResponse(policy_id='0007300A06F8D36F') 15:28 INFO [databricks.labs.ucx.install] Deleting UCX v0.27.2+1520240701152604 from https://DATABRICKS_HOST 15:28 INFO [databricks.labs.ucx.install] Deleting inventory database ucx_sm5j5 15:29 INFO [databricks.labs.ucx.install] Deleting jobs 15:29 INFO [databricks.labs.ucx.install] Deleting migrate-external-tables-ctas job_id=79376344929232. 15:29 INFO [databricks.labs.ucx.install] Deleting scan-tables-in-mounts-experimental job_id=634505184454115. 15:29 INFO [databricks.labs.ucx.install] Deleting validate-groups-permissions job_id=516482021134625. 15:29 INFO [databricks.labs.ucx.install] Deleting failing job_id=229676872797190. 15:29 INFO [databricks.labs.ucx.install] Deleting migrate-groups-experimental job_id=699931774276369. 15:29 INFO [databricks.labs.ucx.install] Deleting migrate-external-hiveserde-tables-in-place-experimental job_id=84043891194063. 15:29 INFO [databricks.labs.ucx.install] Deleting migrate-groups job_id=72148321298887. 15:29 INFO [databricks.labs.ucx.install] Deleting assessment job_id=977075766123315. 15:29 INFO [databricks.labs.ucx.install] Deleting migrate-tables-in-mounts-experimental job_id=874494288624312. 15:29 INFO [databricks.labs.ucx.install] Deleting migrate-tables job_id=1120666647271900. 15:29 INFO [databricks.labs.ucx.install] Deleting remove-workspace-local-backup-groups job_id=621293040300888. 15:29 INFO [databricks.labs.ucx.install] Deleting migrate-data-reconciliation job_id=629145507139314. 15:29 INFO [databricks.labs.ucx.install] Deleting experimental-workflow-linter job_id=1085690196022694. 15:29 INFO [databricks.labs.ucx.install] Deleting cluster policy 15:29 INFO [databricks.labs.ucx.install] Deleting secret scope 15:29 INFO [databricks.labs.ucx.install] UnInstalling UCX complete 15:29 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 workspace user fixtures 15:29 DEBUG [databricks.labs.ucx.mixins.fixtures] removing workspace user fixture: User(active=True, display_name='sdk-zijvsfpo@example.com', emails=[ComplexValue(display=None, primary=True, ref=None, type='work', value='sdk-zijvsfpo@example.com')], entitlements=[], external_id=None, groups=[], id='513135327215568', name=Name(family_name=None, given_name='sdk-zijvsfpo@example.com'), roles=[], schemas=[, ], user_name='sdk-zijvsfpo@example.com') 15:29 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 account group fixtures 15:29 DEBUG [databricks.labs.ucx.mixins.fixtures] removing account group fixture: Group(display_name='ucx_GqaTY', entitlements=[], external_id=None, groups=[], id='198526173027308', members=[ComplexValue(display='sdk-zijvsfpo@example.com', primary=None, ref='Users/513135327215568', type=None, value='513135327215568')], meta=None, roles=[], schemas=[]) 15:29 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 workspace group fixtures 15:29 DEBUG [databricks.labs.ucx.mixins.fixtures] removing workspace group fixture: Group(display_name='ucx_GqaTY', entitlements=[ComplexValue(display=None, primary=None, ref=None, type=None, value='allow-cluster-create')], external_id=None, groups=[], id='791933909540995', members=[ComplexValue(display='sdk-zijvsfpo@example.com', primary=None, ref='Users/513135327215568', type=None, value='513135327215568')], meta=ResourceMeta(resource_type='WorkspaceGroup'), roles=[], schemas=[]) 15:29 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 table fixtures 15:29 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 table fixtures 15:29 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 schema fixtures 15:29 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_sm5j5', metastore_id=None, name='ucx_sm5j5', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) [gw9] linux -- Python 3.10.14 /home/runner/work/ucx/ucx/.venv/bin/python ```

_{Running from acceptance #4084}