Adjust type hints in credentials handling for Azure API client

[asnare]

Changes

This PR is stacked on #1943; it is stacked to verify the type hinting changes via the CI environment.

The situation is:

• The Databricks SDK has partially implemented type hints for the credentials handlers.

• The CredentialsProvider is the fundamental type for a callable factory that can be used to obtain credentials.

• The CredentialsStrategy is the type that can be used for a function that can be used to obtain a provider, given the Config.

• The @credentials_strategy decorator is used to fuse the two:

  1. It decorates a function that, given a Config, returns a CredentialsProvider.
  2. The decorated function that it returns is a CredentialsStrategy.

• The type upstream type hinting for the @credentials_strategy decorator is incomplete: the return type is omitted. This confuses IntelliJ/PyCharm a little but not Mypy.

[github-actions[bot]]

❌ 168/172 passed, 4 failed, 24 skipped, 5h40m32s total

❌ test_running_real_workflow_linter_job: TimeoutError: timed out after 0:20:00: (21m49.008s)

``` TimeoutError: timed out after 0:20:00: [gw0] linux -- Python 3.10.14 /home/runner/work/ucx/ucx/.venv/bin/python 08:26 DEBUG [databricks.labs.ucx.mixins.fixtures] added directory fixture: /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/sdk-gITt 08:26 DEBUG [databricks.labs.ucx.mixins.fixtures] added notebook fixture: /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/sdk-gITt/notebook.ipynb 08:26 INFO [databricks.labs.ucx.mixins.fixtures] Job: https://DATABRICKS_HOST#job/211480981591453 08:26 DEBUG [databricks.labs.ucx.mixins.fixtures] added job fixture: CreateResponse(job_id=211480981591453) 08:26 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_se0cb: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_se0cb 08:26 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_se0cb', metastore_id=None, name='ucx_se0cb', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 08:26 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.WEM3/config.yml) doesn't exist. 08:26 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration 08:26 INFO [databricks.labs.ucx.installer.hms_lineage] HMS Lineage feature creates one system table named system.hms_to_uc_migration.table_access and helps in your migration process from HMS to UC by allowing you to programmatically query HMS lineage data. 08:26 INFO [databricks.labs.ucx.install] Fetching installations... 08:26 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy. 08:26 DEBUG [tests.integration.conftest] Waiting for clusters to start... 08:26 DEBUG [tests.integration.conftest] Waiting for clusters to start... 08:27 INFO [databricks.labs.ucx.install] Installing UCX v0.27.2+1920240702082700 08:27 INFO [databricks.labs.ucx.install] Creating ucx schemas... 08:27 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups 08:27 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=experimental-workflow-linter 08:27 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=failing 08:27 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-hiveserde-tables-in-place-experimental 08:27 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=scan-tables-in-mounts-experimental 08:27 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-tables-ctas 08:27 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-data-reconciliation 08:27 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables 08:27 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions 08:27 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental 08:27 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment 08:28 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups 08:28 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-experimental 08:28 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.WEM3/README for the next steps. 08:28 DEBUG [databricks.labs.ucx.installer.workflows] starting experimental-workflow-linter job: https://DATABRICKS_HOST#job/1108562563424488 08:26 DEBUG [databricks.labs.ucx.mixins.fixtures] added directory fixture: /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/sdk-gITt 08:26 DEBUG [databricks.labs.ucx.mixins.fixtures] added notebook fixture: /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/sdk-gITt/notebook.ipynb 08:26 INFO [databricks.labs.ucx.mixins.fixtures] Job: https://DATABRICKS_HOST#job/211480981591453 08:26 DEBUG [databricks.labs.ucx.mixins.fixtures] added job fixture: CreateResponse(job_id=211480981591453) 08:26 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_se0cb: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_se0cb 08:26 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_se0cb', metastore_id=None, name='ucx_se0cb', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 08:26 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.WEM3/config.yml) doesn't exist. 08:26 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration 08:26 INFO [databricks.labs.ucx.installer.hms_lineage] HMS Lineage feature creates one system table named system.hms_to_uc_migration.table_access and helps in your migration process from HMS to UC by allowing you to programmatically query HMS lineage data. 08:26 INFO [databricks.labs.ucx.install] Fetching installations... 08:26 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy. 08:26 DEBUG [tests.integration.conftest] Waiting for clusters to start... 08:26 DEBUG [tests.integration.conftest] Waiting for clusters to start... 08:27 INFO [databricks.labs.ucx.install] Installing UCX v0.27.2+1920240702082700 08:27 INFO [databricks.labs.ucx.install] Creating ucx schemas... 08:27 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups 08:27 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=experimental-workflow-linter 08:27 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=failing 08:27 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-hiveserde-tables-in-place-experimental 08:27 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=scan-tables-in-mounts-experimental 08:27 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-tables-ctas 08:27 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-data-reconciliation 08:27 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables 08:27 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions 08:27 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental 08:27 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment 08:28 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups 08:28 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-experimental 08:28 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.WEM3/README for the next steps. 08:28 DEBUG [databricks.labs.ucx.installer.workflows] starting experimental-workflow-linter job: https://DATABRICKS_HOST#job/1108562563424488 08:48 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 job fixtures 08:48 DEBUG [databricks.labs.ucx.mixins.fixtures] removing job fixture: CreateResponse(job_id=211480981591453) 08:48 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 directory fixtures 08:48 DEBUG [databricks.labs.ucx.mixins.fixtures] removing directory fixture: /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/sdk-gITt 08:48 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 notebook fixtures 08:48 DEBUG [databricks.labs.ucx.mixins.fixtures] removing notebook fixture: /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/sdk-gITt/notebook.ipynb 08:48 DEBUG [databricks.labs.ucx.mixins.fixtures] ignoring error while notebook /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/sdk-gITt/notebook.ipynb teardown: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/sdk-gITt/notebook.ipynb) doesn't exist. 08:48 INFO [databricks.labs.ucx.install] Deleting UCX v0.27.2+1920240702082700 from https://DATABRICKS_HOST 08:48 INFO [databricks.labs.ucx.install] Deleting inventory database ucx_se0cb 08:48 INFO [databricks.labs.ucx.install] Deleting jobs 08:48 INFO [databricks.labs.ucx.install] Deleting migrate-groups job_id=333390159726378. 08:48 INFO [databricks.labs.ucx.install] Deleting experimental-workflow-linter job_id=1108562563424488. 08:48 INFO [databricks.labs.ucx.install] Deleting failing job_id=586867081988849. 08:48 INFO [databricks.labs.ucx.install] Deleting migrate-external-hiveserde-tables-in-place-experimental job_id=286142826937246. 08:48 INFO [databricks.labs.ucx.install] Deleting scan-tables-in-mounts-experimental job_id=359306207860365. 08:48 INFO [databricks.labs.ucx.install] Deleting migrate-external-tables-ctas job_id=293368177752281. 08:48 INFO [databricks.labs.ucx.install] Deleting migrate-data-reconciliation job_id=722000416273587. 08:48 INFO [databricks.labs.ucx.install] Deleting migrate-tables job_id=643512032851768. 08:48 INFO [databricks.labs.ucx.install] Deleting validate-groups-permissions job_id=320282409385093. 08:48 INFO [databricks.labs.ucx.install] Deleting migrate-tables-in-mounts-experimental job_id=308622283731664. 08:48 INFO [databricks.labs.ucx.install] Deleting assessment job_id=122774938980681. 08:48 INFO [databricks.labs.ucx.install] Deleting remove-workspace-local-backup-groups job_id=111270810186782. 08:48 INFO [databricks.labs.ucx.install] Deleting migrate-groups-experimental job_id=486066255317487. 08:48 INFO [databricks.labs.ucx.install] Deleting cluster policy 08:48 INFO [databricks.labs.ucx.install] Deleting secret scope 08:48 INFO [databricks.labs.ucx.install] UnInstalling UCX complete 08:48 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 workspace user fixtures 08:48 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 account group fixtures 08:48 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 workspace group fixtures 08:48 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 table fixtures 08:48 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 table fixtures 08:48 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 schema fixtures 08:48 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_se0cb', metastore_id=None, name='ucx_se0cb', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) [gw0] linux -- Python 3.10.14 /home/runner/work/ucx/ucx/.venv/bin/python ```

❌ test_table_migration_job_cluster_override[regular]: AssertionError: assert False (12m15.845s)

``` AssertionError: assert False + where False = >('migrate-tables') + where > = .validate_step + where = .deployed_workflows 08:37 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/3R3u 08:37 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.migrate_ch2yw: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_ch2yw 08:37 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.migrate_ch2yw', metastore_id=None, name='migrate_ch2yw', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 08:37 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_ch2yw.ucx_t8hno: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_ch2yw/ucx_t8hno 08:37 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_ch2yw.ucx_t8hno', metastore_id=None, name='ucx_t8hno', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='migrate_ch2yw', sql_path=None, storage_credential_name=None, storage_location='dbfs:/user/hive/warehouse/migrate_ch2yw/ucx_t8hno', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 08:37 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_ch2yw.ucx_tsvtu: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_ch2yw/ucx_tsvtu 08:37 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_ch2yw.ucx_tsvtu', metastore_id=None, name='ucx_tsvtu', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='migrate_ch2yw', sql_path=None, storage_credential_name=None, storage_location='dbfs:/tmp/ucx_test_hbz4', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 08:37 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_ch2yw.ucx_ttrpa: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_ch2yw/ucx_ttrpa 08:37 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_ch2yw.ucx_ttrpa', metastore_id=None, name='ucx_ttrpa', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='migrate_ch2yw', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/b/3R3u', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 08:37 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_ch2yw.ucx_tl5sh: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_ch2yw/ucx_tl5sh 08:37 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_ch2yw.ucx_tl5sh', metastore_id=None, name='ucx_tl5sh', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='migrate_ch2yw', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_ch2yw.ucx_t8hno', view_dependencies=None) 08:37 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_ch2yw.ucx_tybtq: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_ch2yw/ucx_tybtq 08:37 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_ch2yw.ucx_tybtq', metastore_id=None, name='ucx_tybtq', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='migrate_ch2yw', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_ch2yw.ucx_tl5sh', view_dependencies=None) 08:37 DEBUG [databricks.labs.ucx.mixins.fixtures] added catalog fixture: CatalogInfo(browse_only=False, catalog_type=, comment='', connection_name=None, created_at=1719909467678, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cny0o', isolation_mode=, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_cny0o', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1719909467678, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d') 08:37 INFO [databricks.labs.ucx.mixins.fixtures] Schema ucx_cny0o.migrate_ch2yw: https://DATABRICKS_HOST/explore/data/ucx_cny0o/migrate_ch2yw 08:37 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_cny0o', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cny0o.migrate_ch2yw', metastore_id=None, name='migrate_ch2yw', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 08:37 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_sswar: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_sswar 08:37 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_sswar', metastore_id=None, name='ucx_sswar', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) [gw9] linux -- Python 3.10.14 /home/runner/work/ucx/ucx/.venv/bin/python 08:37 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/3R3u 08:37 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.migrate_ch2yw: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_ch2yw 08:37 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.migrate_ch2yw', metastore_id=None, name='migrate_ch2yw', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 08:37 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_ch2yw.ucx_t8hno: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_ch2yw/ucx_t8hno 08:37 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_ch2yw.ucx_t8hno', metastore_id=None, name='ucx_t8hno', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='migrate_ch2yw', sql_path=None, storage_credential_name=None, storage_location='dbfs:/user/hive/warehouse/migrate_ch2yw/ucx_t8hno', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 08:37 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_ch2yw.ucx_tsvtu: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_ch2yw/ucx_tsvtu 08:37 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_ch2yw.ucx_tsvtu', metastore_id=None, name='ucx_tsvtu', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='migrate_ch2yw', sql_path=None, storage_credential_name=None, storage_location='dbfs:/tmp/ucx_test_hbz4', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 08:37 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_ch2yw.ucx_ttrpa: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_ch2yw/ucx_ttrpa 08:37 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_ch2yw.ucx_ttrpa', metastore_id=None, name='ucx_ttrpa', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='migrate_ch2yw', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/b/3R3u', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 08:37 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_ch2yw.ucx_tl5sh: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_ch2yw/ucx_tl5sh 08:37 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_ch2yw.ucx_tl5sh', metastore_id=None, name='ucx_tl5sh', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='migrate_ch2yw', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_ch2yw.ucx_t8hno', view_dependencies=None) 08:37 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_ch2yw.ucx_tybtq: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_ch2yw/ucx_tybtq 08:37 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_ch2yw.ucx_tybtq', metastore_id=None, name='ucx_tybtq', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='migrate_ch2yw', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_ch2yw.ucx_tl5sh', view_dependencies=None) 08:37 DEBUG [databricks.labs.ucx.mixins.fixtures] added catalog fixture: CatalogInfo(browse_only=False, catalog_type=, comment='', connection_name=None, created_at=1719909467678, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cny0o', isolation_mode=, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_cny0o', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1719909467678, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d') 08:37 INFO [databricks.labs.ucx.mixins.fixtures] Schema ucx_cny0o.migrate_ch2yw: https://DATABRICKS_HOST/explore/data/ucx_cny0o/migrate_ch2yw 08:37 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_cny0o', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cny0o.migrate_ch2yw', metastore_id=None, name='migrate_ch2yw', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 08:37 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_sswar: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_sswar 08:37 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_sswar', metastore_id=None, name='ucx_sswar', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 08:38 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.MMbu/config.yml) doesn't exist. 08:38 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration 08:38 INFO [databricks.labs.ucx.installer.hms_lineage] HMS Lineage feature creates one system table named system.hms_to_uc_migration.table_access and helps in your migration process from HMS to UC by allowing you to programmatically query HMS lineage data. 08:38 INFO [databricks.labs.ucx.install] Fetching installations... 08:38 WARNING [databricks.labs.ucx.install] Existing installation at /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.MMbu is corrupted. Skipping... 08:38 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy. 08:38 DEBUG [tests.integration.conftest] Waiting for clusters to start... 08:38 DEBUG [tests.integration.conftest] Waiting for clusters to start... 08:38 INFO [databricks.labs.ucx.install] Installing UCX v0.27.2+1920240702083859 08:38 INFO [databricks.labs.ucx.install] Creating ucx schemas... 08:39 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-data-reconciliation 08:39 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-tables-ctas 08:39 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-hiveserde-tables-in-place-experimental 08:39 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental 08:39 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables 08:39 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment 08:39 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-experimental 08:39 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups 08:39 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions 08:39 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=scan-tables-in-mounts-experimental 08:39 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=failing 08:39 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups 08:39 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=experimental-workflow-linter 08:39 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.MMbu/README for the next steps. 08:39 DEBUG [databricks.labs.ucx.installer.workflows] starting migrate-tables job: https://DATABRICKS_HOST#job/767010569747575 08:48 DEBUG [databricks.labs.ucx.installer.workflows] Validating migrate-tables workflow: https://DATABRICKS_HOST#job/767010569747575 08:37 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/3R3u 08:37 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.migrate_ch2yw: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_ch2yw 08:37 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.migrate_ch2yw', metastore_id=None, name='migrate_ch2yw', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 08:37 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_ch2yw.ucx_t8hno: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_ch2yw/ucx_t8hno 08:37 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_ch2yw.ucx_t8hno', metastore_id=None, name='ucx_t8hno', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='migrate_ch2yw', sql_path=None, storage_credential_name=None, storage_location='dbfs:/user/hive/warehouse/migrate_ch2yw/ucx_t8hno', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 08:37 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_ch2yw.ucx_tsvtu: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_ch2yw/ucx_tsvtu 08:37 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_ch2yw.ucx_tsvtu', metastore_id=None, name='ucx_tsvtu', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='migrate_ch2yw', sql_path=None, storage_credential_name=None, storage_location='dbfs:/tmp/ucx_test_hbz4', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 08:37 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_ch2yw.ucx_ttrpa: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_ch2yw/ucx_ttrpa 08:37 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_ch2yw.ucx_ttrpa', metastore_id=None, name='ucx_ttrpa', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='migrate_ch2yw', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/b/3R3u', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 08:37 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_ch2yw.ucx_tl5sh: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_ch2yw/ucx_tl5sh 08:37 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_ch2yw.ucx_tl5sh', metastore_id=None, name='ucx_tl5sh', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='migrate_ch2yw', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_ch2yw.ucx_t8hno', view_dependencies=None) 08:37 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_ch2yw.ucx_tybtq: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_ch2yw/ucx_tybtq 08:37 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_ch2yw.ucx_tybtq', metastore_id=None, name='ucx_tybtq', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='migrate_ch2yw', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_ch2yw.ucx_tl5sh', view_dependencies=None) 08:37 DEBUG [databricks.labs.ucx.mixins.fixtures] added catalog fixture: CatalogInfo(browse_only=False, catalog_type=, comment='', connection_name=None, created_at=1719909467678, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cny0o', isolation_mode=, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_cny0o', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1719909467678, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d') 08:37 INFO [databricks.labs.ucx.mixins.fixtures] Schema ucx_cny0o.migrate_ch2yw: https://DATABRICKS_HOST/explore/data/ucx_cny0o/migrate_ch2yw 08:37 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_cny0o', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cny0o.migrate_ch2yw', metastore_id=None, name='migrate_ch2yw', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 08:37 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_sswar: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_sswar 08:37 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_sswar', metastore_id=None, name='ucx_sswar', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 08:38 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.MMbu/config.yml) doesn't exist. 08:38 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration 08:38 INFO [databricks.labs.ucx.installer.hms_lineage] HMS Lineage feature creates one system table named system.hms_to_uc_migration.table_access and helps in your migration process from HMS to UC by allowing you to programmatically query HMS lineage data. 08:38 INFO [databricks.labs.ucx.install] Fetching installations... 08:38 WARNING [databricks.labs.ucx.install] Existing installation at /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.MMbu is corrupted. Skipping... 08:38 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy. 08:38 DEBUG [tests.integration.conftest] Waiting for clusters to start... 08:38 DEBUG [tests.integration.conftest] Waiting for clusters to start... 08:38 INFO [databricks.labs.ucx.install] Installing UCX v0.27.2+1920240702083859 08:38 INFO [databricks.labs.ucx.install] Creating ucx schemas... 08:39 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-data-reconciliation 08:39 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-tables-ctas 08:39 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-hiveserde-tables-in-place-experimental 08:39 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental 08:39 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables 08:39 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment 08:39 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-experimental 08:39 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups 08:39 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions 08:39 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=scan-tables-in-mounts-experimental 08:39 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=failing 08:39 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups 08:39 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=experimental-workflow-linter 08:39 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.MMbu/README for the next steps. 08:39 DEBUG [databricks.labs.ucx.installer.workflows] starting migrate-tables job: https://DATABRICKS_HOST#job/767010569747575 08:48 DEBUG [databricks.labs.ucx.installer.workflows] Validating migrate-tables workflow: https://DATABRICKS_HOST#job/767010569747575 08:48 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 make_storage_dir fixtures 08:48 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 make_dbfs_data_copy fixtures 08:48 DEBUG [databricks.labs.ucx.mixins.fixtures] removing make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/3R3u 08:48 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 cluster fixtures 08:48 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 catalog fixtures 08:48 DEBUG [databricks.labs.ucx.mixins.fixtures] removing catalog fixture: CatalogInfo(browse_only=False, catalog_type=, comment='', connection_name=None, created_at=1719909467678, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cny0o', isolation_mode=, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_cny0o', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1719909467678, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d') 08:48 INFO [databricks.labs.ucx.install] Deleting UCX v0.27.2+1920240702083859 from https://DATABRICKS_HOST 08:48 INFO [databricks.labs.ucx.install] Deleting inventory database ucx_sswar 08:48 INFO [databricks.labs.ucx.install] Deleting jobs 08:48 INFO [databricks.labs.ucx.install] Deleting migrate-data-reconciliation job_id=648590615116669. 08:48 INFO [databricks.labs.ucx.install] Deleting migrate-external-tables-ctas job_id=695545576074879. 08:48 INFO [databricks.labs.ucx.install] Deleting migrate-external-hiveserde-tables-in-place-experimental job_id=1118156651358186. 08:48 INFO [databricks.labs.ucx.install] Deleting migrate-tables-in-mounts-experimental job_id=258219036433500. 08:48 INFO [databricks.labs.ucx.install] Deleting migrate-tables job_id=767010569747575. 08:48 INFO [databricks.labs.ucx.install] Deleting assessment job_id=314030895358644. 08:48 INFO [databricks.labs.ucx.install] Deleting migrate-groups-experimental job_id=571417885396629. 08:48 INFO [databricks.labs.ucx.install] Deleting migrate-groups job_id=52443650767869. 08:48 INFO [databricks.labs.ucx.install] Deleting validate-groups-permissions job_id=372933925240268. 08:48 INFO [databricks.labs.ucx.install] Deleting scan-tables-in-mounts-experimental job_id=332607072596468. 08:48 INFO [databricks.labs.ucx.install] Deleting failing job_id=148472965339827. 08:48 INFO [databricks.labs.ucx.install] Deleting remove-workspace-local-backup-groups job_id=118845183898113. 08:48 INFO [databricks.labs.ucx.install] Deleting experimental-workflow-linter job_id=768054714376730. 08:48 INFO [databricks.labs.ucx.install] Deleting cluster policy 08:48 INFO [databricks.labs.ucx.install] Deleting secret scope 08:48 INFO [databricks.labs.ucx.install] UnInstalling UCX complete 08:48 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 workspace user fixtures 08:48 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 account group fixtures 08:48 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 workspace group fixtures 08:48 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 table fixtures 08:48 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 5 table fixtures 08:48 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_ch2yw.ucx_t8hno', metastore_id=None, name='ucx_t8hno', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='migrate_ch2yw', sql_path=None, storage_credential_name=None, storage_location='dbfs:/user/hive/warehouse/migrate_ch2yw/ucx_t8hno', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 08:48 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_ch2yw.ucx_tsvtu', metastore_id=None, name='ucx_tsvtu', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='migrate_ch2yw', sql_path=None, storage_credential_name=None, storage_location='dbfs:/tmp/ucx_test_hbz4', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 08:48 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_ch2yw.ucx_ttrpa', metastore_id=None, name='ucx_ttrpa', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='migrate_ch2yw', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/b/3R3u', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 08:48 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_ch2yw.ucx_tl5sh', metastore_id=None, name='ucx_tl5sh', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='migrate_ch2yw', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_ch2yw.ucx_t8hno', view_dependencies=None) 08:48 DEBUG [databricks.labs.ucx.mixins.fixtures] ignoring error while table TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_ch2yw.ucx_tl5sh', metastore_id=None, name='ucx_tl5sh', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='migrate_ch2yw', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_ch2yw.ucx_t8hno', view_dependencies=None) teardown: [WRONG_COMMAND_FOR_OBJECT_TYPE] The operation DROP TABLE requires a EXTERNAL or MANAGED. But hive_metastore.migrate_ch2yw.ucx_tl5sh is a VIEW. Use DROP VIEW instead. SQLSTATE: 42809 08:48 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_ch2yw.ucx_tybtq', metastore_id=None, name='ucx_tybtq', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='migrate_ch2yw', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_ch2yw.ucx_tl5sh', view_dependencies=None) 08:49 DEBUG [databricks.labs.ucx.mixins.fixtures] ignoring error while table TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_ch2yw.ucx_tybtq', metastore_id=None, name='ucx_tybtq', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='migrate_ch2yw', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_ch2yw.ucx_tl5sh', view_dependencies=None) teardown: [WRONG_COMMAND_FOR_OBJECT_TYPE] The operation DROP TABLE requires a EXTERNAL or MANAGED. But hive_metastore.migrate_ch2yw.ucx_tybtq is a VIEW. Use DROP VIEW instead. SQLSTATE: 42809 08:49 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 3 schema fixtures 08:49 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.migrate_ch2yw', metastore_id=None, name='migrate_ch2yw', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 08:49 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_cny0o', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cny0o.migrate_ch2yw', metastore_id=None, name='migrate_ch2yw', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 08:49 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_sswar', metastore_id=None, name='ucx_sswar', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) [gw9] linux -- Python 3.10.14 /home/runner/work/ucx/ucx/.venv/bin/python ```

❌ test_table_migration_job_refreshes_migration_status[regular-migrate-tables]: AssertionError: No destination schema found for TableType.VIEW hive_metastore.migrate_e1qqa.ucx_tfnce given migration statuses Row(src_schema='migrate_e1qqa', src_table='ucx_tksdz', dst_catalog='ucx_c1zcx', dst_schema='migrate_e1qqa', dst_table='ucx_tksdz', update_ts='1719909569.860524') (24m2.578s)

``` AssertionError: No destination schema found for TableType.VIEW hive_metastore.migrate_e1qqa.ucx_tfnce given migration statuses Row(src_schema='migrate_e1qqa', src_table='ucx_tksdz', dst_catalog='ucx_c1zcx', dst_schema='migrate_e1qqa', dst_table='ucx_tksdz', update_ts='1719909569.860524') Row(src_schema='migrate_e1qqa', src_table='ucx_toulq', dst_catalog='ucx_c1zcx', dst_schema='migrate_e1qqa', dst_table='ucx_toulq', update_ts='1719909569.860524') Row(src_schema='migrate_e1qqa', src_table='ucx_tv3gb', dst_catalog='ucx_c1zcx', dst_schema='migrate_e1qqa', dst_table='ucx_tv3gb', update_ts='1719909569.860524') Row(src_schema='migrate_e1qqa', src_table='ucx_ttjfx', dst_catalog='ucx_c1zcx', dst_schema='migrate_e1qqa', dst_table='ucx_ttjfx', update_ts='1719909569.860524') Row(src_schema='migrate_e1qqa', src_table='ucx_tfnce', dst_catalog=None, dst_schema=None, dst_table=None, update_ts='1719909569.860524') assert 1 == 0 + where 1 = len(['No destination schema found for TableType.VIEW hive_metastore.migrate_e1qqa.ucx_tfnce']) 08:26 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/VAKn 08:26 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.migrate_e1qqa: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_e1qqa 08:26 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.migrate_e1qqa', metastore_id=None, name='migrate_e1qqa', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 08:26 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_e1qqa.ucx_tksdz: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_e1qqa/ucx_tksdz 08:26 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_e1qqa.ucx_tksdz', metastore_id=None, name='ucx_tksdz', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='migrate_e1qqa', sql_path=None, storage_credential_name=None, storage_location='dbfs:/user/hive/warehouse/migrate_e1qqa/ucx_tksdz', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 08:26 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_e1qqa.ucx_toulq: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_e1qqa/ucx_toulq 08:26 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_e1qqa.ucx_toulq', metastore_id=None, name='ucx_toulq', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='migrate_e1qqa', sql_path=None, storage_credential_name=None, storage_location='dbfs:/tmp/ucx_test_hqF7', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 08:26 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_e1qqa.ucx_tv3gb: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_e1qqa/ucx_tv3gb 08:26 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_e1qqa.ucx_tv3gb', metastore_id=None, name='ucx_tv3gb', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='migrate_e1qqa', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/b/VAKn', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 08:26 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_e1qqa.ucx_ttjfx: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_e1qqa/ucx_ttjfx 08:26 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_e1qqa.ucx_ttjfx', metastore_id=None, name='ucx_ttjfx', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='migrate_e1qqa', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_e1qqa.ucx_tksdz', view_dependencies=None) 08:26 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_e1qqa.ucx_tfnce: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_e1qqa/ucx_tfnce 08:26 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_e1qqa.ucx_tfnce', metastore_id=None, name='ucx_tfnce', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='migrate_e1qqa', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_e1qqa.ucx_ttjfx', view_dependencies=None) 08:26 DEBUG [databricks.labs.ucx.mixins.fixtures] added catalog fixture: CatalogInfo(browse_only=False, catalog_type=, comment='', connection_name=None, created_at=1719908806757, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_c1zcx', isolation_mode=, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_c1zcx', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1719908806757, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d') 08:26 INFO [databricks.labs.ucx.mixins.fixtures] Schema ucx_c1zcx.migrate_e1qqa: https://DATABRICKS_HOST/explore/data/ucx_c1zcx/migrate_e1qqa 08:26 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_c1zcx', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_c1zcx.migrate_e1qqa', metastore_id=None, name='migrate_e1qqa', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 08:26 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_shpgq: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_shpgq 08:26 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_shpgq', metastore_id=None, name='ucx_shpgq', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) [gw5] linux -- Python 3.10.14 /home/runner/work/ucx/ucx/.venv/bin/python 08:26 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/VAKn 08:26 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.migrate_e1qqa: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_e1qqa 08:26 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.migrate_e1qqa', metastore_id=None, name='migrate_e1qqa', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 08:26 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_e1qqa.ucx_tksdz: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_e1qqa/ucx_tksdz 08:26 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_e1qqa.ucx_tksdz', metastore_id=None, name='ucx_tksdz', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='migrate_e1qqa', sql_path=None, storage_credential_name=None, storage_location='dbfs:/user/hive/warehouse/migrate_e1qqa/ucx_tksdz', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 08:26 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_e1qqa.ucx_toulq: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_e1qqa/ucx_toulq 08:26 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_e1qqa.ucx_toulq', metastore_id=None, name='ucx_toulq', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='migrate_e1qqa', sql_path=None, storage_credential_name=None, storage_location='dbfs:/tmp/ucx_test_hqF7', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 08:26 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_e1qqa.ucx_tv3gb: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_e1qqa/ucx_tv3gb 08:26 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_e1qqa.ucx_tv3gb', metastore_id=None, name='ucx_tv3gb', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='migrate_e1qqa', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/b/VAKn', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 08:26 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_e1qqa.ucx_ttjfx: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_e1qqa/ucx_ttjfx 08:26 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_e1qqa.ucx_ttjfx', metastore_id=None, name='ucx_ttjfx', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='migrate_e1qqa', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_e1qqa.ucx_tksdz', view_dependencies=None) 08:26 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_e1qqa.ucx_tfnce: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_e1qqa/ucx_tfnce 08:26 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_e1qqa.ucx_tfnce', metastore_id=None, name='ucx_tfnce', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='migrate_e1qqa', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_e1qqa.ucx_ttjfx', view_dependencies=None) 08:26 DEBUG [databricks.labs.ucx.mixins.fixtures] added catalog fixture: CatalogInfo(browse_only=False, catalog_type=, comment='', connection_name=None, created_at=1719908806757, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_c1zcx', isolation_mode=, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_c1zcx', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1719908806757, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d') 08:26 INFO [databricks.labs.ucx.mixins.fixtures] Schema ucx_c1zcx.migrate_e1qqa: https://DATABRICKS_HOST/explore/data/ucx_c1zcx/migrate_e1qqa 08:26 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_c1zcx', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_c1zcx.migrate_e1qqa', metastore_id=None, name='migrate_e1qqa', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 08:26 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_shpgq: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_shpgq 08:26 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_shpgq', metastore_id=None, name='ucx_shpgq', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 08:27 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.lmaO/config.yml) doesn't exist. 08:27 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration 08:27 INFO [databricks.labs.ucx.installer.hms_lineage] HMS Lineage feature creates one system table named system.hms_to_uc_migration.table_access and helps in your migration process from HMS to UC by allowing you to programmatically query HMS lineage data. 08:27 INFO [databricks.labs.ucx.install] Fetching installations... 08:27 WARNING [databricks.labs.ucx.install] Existing installation at /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.lmaO is corrupted. Skipping... 08:27 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy. 08:27 DEBUG [tests.integration.conftest] Waiting for clusters to start... 08:27 DEBUG [tests.integration.conftest] Waiting for clusters to start... 08:27 INFO [databricks.labs.ucx.install] Installing UCX v0.27.2+1920240702082755 08:27 INFO [databricks.labs.ucx.install] Creating ucx schemas... 08:28 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables 08:28 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups 08:28 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions 08:28 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment 08:29 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-experimental 08:29 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups 08:29 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-tables-ctas 08:29 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=experimental-workflow-linter 08:29 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=scan-tables-in-mounts-experimental 08:29 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental 08:29 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=failing 08:29 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-data-reconciliation 08:29 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-hiveserde-tables-in-place-experimental 08:29 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.lmaO/README for the next steps. 08:29 DEBUG [databricks.labs.ucx.installer.workflows] starting migrate-tables job: https://DATABRICKS_HOST#job/3607200715084 08:26 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/VAKn 08:26 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.migrate_e1qqa: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_e1qqa 08:26 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.migrate_e1qqa', metastore_id=None, name='migrate_e1qqa', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 08:26 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_e1qqa.ucx_tksdz: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_e1qqa/ucx_tksdz 08:26 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_e1qqa.ucx_tksdz', metastore_id=None, name='ucx_tksdz', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='migrate_e1qqa', sql_path=None, storage_credential_name=None, storage_location='dbfs:/user/hive/warehouse/migrate_e1qqa/ucx_tksdz', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 08:26 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_e1qqa.ucx_toulq: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_e1qqa/ucx_toulq 08:26 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_e1qqa.ucx_toulq', metastore_id=None, name='ucx_toulq', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='migrate_e1qqa', sql_path=None, storage_credential_name=None, storage_location='dbfs:/tmp/ucx_test_hqF7', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 08:26 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_e1qqa.ucx_tv3gb: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_e1qqa/ucx_tv3gb 08:26 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_e1qqa.ucx_tv3gb', metastore_id=None, name='ucx_tv3gb', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='migrate_e1qqa', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/b/VAKn', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 08:26 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_e1qqa.ucx_ttjfx: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_e1qqa/ucx_ttjfx 08:26 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_e1qqa.ucx_ttjfx', metastore_id=None, name='ucx_ttjfx', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='migrate_e1qqa', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_e1qqa.ucx_tksdz', view_dependencies=None) 08:26 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_e1qqa.ucx_tfnce: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_e1qqa/ucx_tfnce 08:26 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_e1qqa.ucx_tfnce', metastore_id=None, name='ucx_tfnce', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='migrate_e1qqa', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_e1qqa.ucx_ttjfx', view_dependencies=None) 08:26 DEBUG [databricks.labs.ucx.mixins.fixtures] added catalog fixture: CatalogInfo(browse_only=False, catalog_type=, comment='', connection_name=None, created_at=1719908806757, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_c1zcx', isolation_mode=, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_c1zcx', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1719908806757, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d') 08:26 INFO [databricks.labs.ucx.mixins.fixtures] Schema ucx_c1zcx.migrate_e1qqa: https://DATABRICKS_HOST/explore/data/ucx_c1zcx/migrate_e1qqa 08:26 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_c1zcx', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_c1zcx.migrate_e1qqa', metastore_id=None, name='migrate_e1qqa', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 08:26 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_shpgq: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_shpgq 08:26 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_shpgq', metastore_id=None, name='ucx_shpgq', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 08:27 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.lmaO/config.yml) doesn't exist. 08:27 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration 08:27 INFO [databricks.labs.ucx.installer.hms_lineage] HMS Lineage feature creates one system table named system.hms_to_uc_migration.table_access and helps in your migration process from HMS to UC by allowing you to programmatically query HMS lineage data. 08:27 INFO [databricks.labs.ucx.install] Fetching installations... 08:27 WARNING [databricks.labs.ucx.install] Existing installation at /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.lmaO is corrupted. Skipping... 08:27 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy. 08:27 DEBUG [tests.integration.conftest] Waiting for clusters to start... 08:27 DEBUG [tests.integration.conftest] Waiting for clusters to start... 08:27 INFO [databricks.labs.ucx.install] Installing UCX v0.27.2+1920240702082755 08:27 INFO [databricks.labs.ucx.install] Creating ucx schemas... 08:28 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables 08:28 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups 08:28 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions 08:28 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment 08:29 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-experimental 08:29 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups 08:29 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-tables-ctas 08:29 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=experimental-workflow-linter 08:29 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=scan-tables-in-mounts-experimental 08:29 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental 08:29 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=failing 08:29 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-data-reconciliation 08:29 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-hiveserde-tables-in-place-experimental 08:29 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.lmaO/README for the next steps. 08:29 DEBUG [databricks.labs.ucx.installer.workflows] starting migrate-tables job: https://DATABRICKS_HOST#job/3607200715084 08:48 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 make_storage_dir fixtures 08:48 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 make_dbfs_data_copy fixtures 08:48 DEBUG [databricks.labs.ucx.mixins.fixtures] removing make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/VAKn 08:48 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 cluster fixtures 08:48 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 catalog fixtures 08:48 DEBUG [databricks.labs.ucx.mixins.fixtures] removing catalog fixture: CatalogInfo(browse_only=False, catalog_type=, comment='', connection_name=None, created_at=1719908806757, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_c1zcx', isolation_mode=, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_c1zcx', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1719908806757, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d') 08:48 INFO [databricks.labs.ucx.install] Deleting UCX v0.27.2+1920240702082755 from https://DATABRICKS_HOST 08:48 INFO [databricks.labs.ucx.install] Deleting inventory database ucx_shpgq 08:49 INFO [databricks.labs.ucx.install] Deleting jobs 08:49 INFO [databricks.labs.ucx.install] Deleting migrate-tables job_id=3607200715084. 08:49 INFO [databricks.labs.ucx.install] Deleting remove-workspace-local-backup-groups job_id=603586545779881. 08:49 INFO [databricks.labs.ucx.install] Deleting validate-groups-permissions job_id=87353882239896. 08:49 INFO [databricks.labs.ucx.install] Deleting assessment job_id=8284000797270. 08:49 INFO [databricks.labs.ucx.install] Deleting migrate-groups-experimental job_id=713932708131772. 08:49 INFO [databricks.labs.ucx.install] Deleting migrate-groups job_id=1099134119234777. 08:49 INFO [databricks.labs.ucx.install] Deleting migrate-external-tables-ctas job_id=916981644685126. 08:49 INFO [databricks.labs.ucx.install] Deleting experimental-workflow-linter job_id=212740737613385. 08:49 INFO [databricks.labs.ucx.install] Deleting scan-tables-in-mounts-experimental job_id=955849797870851. 08:49 INFO [databricks.labs.ucx.install] Deleting migrate-tables-in-mounts-experimental job_id=93904157616609. 08:49 INFO [databricks.labs.ucx.install] Deleting failing job_id=155855087385096. 08:49 INFO [databricks.labs.ucx.install] Deleting migrate-data-reconciliation job_id=700614556745144. 08:49 INFO [databricks.labs.ucx.install] Deleting migrate-external-hiveserde-tables-in-place-experimental job_id=144394863092255. 08:49 INFO [databricks.labs.ucx.install] Deleting cluster policy 08:49 INFO [databricks.labs.ucx.install] Deleting secret scope 08:49 INFO [databricks.labs.ucx.install] UnInstalling UCX complete 08:49 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 workspace user fixtures 08:49 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 account group fixtures 08:49 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 workspace group fixtures 08:49 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 table fixtures 08:49 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 5 table fixtures 08:49 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_e1qqa.ucx_tksdz', metastore_id=None, name='ucx_tksdz', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='migrate_e1qqa', sql_path=None, storage_credential_name=None, storage_location='dbfs:/user/hive/warehouse/migrate_e1qqa/ucx_tksdz', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 08:49 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_e1qqa.ucx_toulq', metastore_id=None, name='ucx_toulq', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='migrate_e1qqa', sql_path=None, storage_credential_name=None, storage_location='dbfs:/tmp/ucx_test_hqF7', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 08:49 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_e1qqa.ucx_tv3gb', metastore_id=None, name='ucx_tv3gb', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='migrate_e1qqa', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/b/VAKn', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 08:49 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_e1qqa.ucx_ttjfx', metastore_id=None, name='ucx_ttjfx', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='migrate_e1qqa', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_e1qqa.ucx_tksdz', view_dependencies=None) 08:49 DEBUG [databricks.labs.ucx.mixins.fixtures] ignoring error while table TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_e1qqa.ucx_ttjfx', metastore_id=None, name='ucx_ttjfx', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='migrate_e1qqa', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_e1qqa.ucx_tksdz', view_dependencies=None) teardown: [WRONG_COMMAND_FOR_OBJECT_TYPE] The operation DROP TABLE requires a EXTERNAL or MANAGED. But hive_metastore.migrate_e1qqa.ucx_ttjfx is a VIEW. Use DROP VIEW instead. SQLSTATE: 42809 08:49 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_e1qqa.ucx_tfnce', metastore_id=None, name='ucx_tfnce', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='migrate_e1qqa', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_e1qqa.ucx_ttjfx', view_dependencies=None) 08:49 DEBUG [databricks.labs.ucx.mixins.fixtures] ignoring error while table TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_e1qqa.ucx_tfnce', metastore_id=None, name='ucx_tfnce', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='migrate_e1qqa', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_e1qqa.ucx_ttjfx', view_dependencies=None) teardown: [WRONG_COMMAND_FOR_OBJECT_TYPE] The operation DROP TABLE requires a EXTERNAL or MANAGED. But hive_metastore.migrate_e1qqa.ucx_tfnce is a VIEW. Use DROP VIEW instead. SQLSTATE: 42809 08:49 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 3 schema fixtures 08:49 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.migrate_e1qqa', metastore_id=None, name='migrate_e1qqa', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 08:49 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_c1zcx', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_c1zcx.migrate_e1qqa', metastore_id=None, name='migrate_e1qqa', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 08:49 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_shpgq', metastore_id=None, name='ucx_shpgq', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) [gw5] linux -- Python 3.10.14 /home/runner/work/ucx/ucx/.venv/bin/python ```

❌ test_hiveserde_table_ctas_migration_job[hiveserde]: AssertionError: assert False (11m38.384s)

``` AssertionError: assert False + where False = >('migrate-external-tables-ctas') + where > = .validate_step + where = .deployed_workflows 08:41 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/2BT1 08:41 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.hiveserde_in_place_cygj0: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_cygj0 08:41 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.hiveserde_in_place_cygj0', metastore_id=None, name='hiveserde_in_place_cygj0', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 08:41 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_storage_dir fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_cygj0 08:41 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_cygj0.parquet_serde_cygj0: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_cygj0/parquet_serde_cygj0 08:41 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_cygj0.parquet_serde_cygj0', metastore_id=None, name='parquet_serde_cygj0', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='hiveserde_in_place_cygj0', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_cygj0/parquet_serde_cygj0', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 08:41 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_cygj0.orc_serde_cygj0: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_cygj0/orc_serde_cygj0 08:41 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_cygj0.orc_serde_cygj0', metastore_id=None, name='orc_serde_cygj0', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='hiveserde_in_place_cygj0', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_cygj0/orc_serde_cygj0', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 08:41 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_cygj0.avro_serde_cygj0: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_cygj0/avro_serde_cygj0 08:41 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_cygj0.avro_serde_cygj0', metastore_id=None, name='avro_serde_cygj0', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='hiveserde_in_place_cygj0', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_cygj0/avro_serde_cygj0', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 08:41 DEBUG [databricks.labs.ucx.mixins.fixtures] added catalog fixture: CatalogInfo(browse_only=False, catalog_type=, comment='', connection_name=None, created_at=1719909710449, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cipce', isolation_mode=, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_cipce', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1719909710449, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d') 08:41 INFO [databricks.labs.ucx.mixins.fixtures] Schema ucx_cipce.hiveserde_in_place_cygj0: https://DATABRICKS_HOST/explore/data/ucx_cipce/hiveserde_in_place_cygj0 08:41 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_cipce', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cipce.hiveserde_in_place_cygj0', metastore_id=None, name='hiveserde_in_place_cygj0', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 08:42 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_saauj: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_saauj 08:42 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_saauj', metastore_id=None, name='ucx_saauj', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) [gw3] linux -- Python 3.10.14 /home/runner/work/ucx/ucx/.venv/bin/python 08:41 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/2BT1 08:41 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.hiveserde_in_place_cygj0: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_cygj0 08:41 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.hiveserde_in_place_cygj0', metastore_id=None, name='hiveserde_in_place_cygj0', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 08:41 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_storage_dir fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_cygj0 08:41 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_cygj0.parquet_serde_cygj0: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_cygj0/parquet_serde_cygj0 08:41 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_cygj0.parquet_serde_cygj0', metastore_id=None, name='parquet_serde_cygj0', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='hiveserde_in_place_cygj0', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_cygj0/parquet_serde_cygj0', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 08:41 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_cygj0.orc_serde_cygj0: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_cygj0/orc_serde_cygj0 08:41 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_cygj0.orc_serde_cygj0', metastore_id=None, name='orc_serde_cygj0', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='hiveserde_in_place_cygj0', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_cygj0/orc_serde_cygj0', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 08:41 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_cygj0.avro_serde_cygj0: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_cygj0/avro_serde_cygj0 08:41 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_cygj0.avro_serde_cygj0', metastore_id=None, name='avro_serde_cygj0', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='hiveserde_in_place_cygj0', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_cygj0/avro_serde_cygj0', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 08:41 DEBUG [databricks.labs.ucx.mixins.fixtures] added catalog fixture: CatalogInfo(browse_only=False, catalog_type=, comment='', connection_name=None, created_at=1719909710449, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cipce', isolation_mode=, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_cipce', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1719909710449, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d') 08:41 INFO [databricks.labs.ucx.mixins.fixtures] Schema ucx_cipce.hiveserde_in_place_cygj0: https://DATABRICKS_HOST/explore/data/ucx_cipce/hiveserde_in_place_cygj0 08:41 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_cipce', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cipce.hiveserde_in_place_cygj0', metastore_id=None, name='hiveserde_in_place_cygj0', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 08:42 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_saauj: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_saauj 08:42 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_saauj', metastore_id=None, name='ucx_saauj', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 08:43 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.Rb5s/config.yml) doesn't exist. 08:43 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration 08:43 INFO [databricks.labs.ucx.installer.hms_lineage] HMS Lineage feature creates one system table named system.hms_to_uc_migration.table_access and helps in your migration process from HMS to UC by allowing you to programmatically query HMS lineage data. 08:43 INFO [databricks.labs.ucx.install] Fetching installations... 08:43 WARNING [databricks.labs.ucx.install] Existing installation at /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.Rb5s is corrupted. Skipping... 08:43 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy. 08:43 DEBUG [tests.integration.conftest] Waiting for clusters to start... 08:43 DEBUG [tests.integration.conftest] Waiting for clusters to start... 08:43 INFO [databricks.labs.ucx.install] Installing UCX v0.27.2+1920240702084322 08:43 INFO [databricks.labs.ucx.install] Creating ucx schemas... 08:44 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental 08:44 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=failing 08:44 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-data-reconciliation 08:44 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-tables-ctas 08:44 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-hiveserde-tables-in-place-experimental 08:44 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions 08:44 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-experimental 08:44 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=experimental-workflow-linter 08:44 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups 08:44 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment 08:44 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables 08:44 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=scan-tables-in-mounts-experimental 08:44 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups 08:44 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.Rb5s/README for the next steps. 08:44 DEBUG [databricks.labs.ucx.installer.workflows] starting migrate-external-tables-ctas job: https://DATABRICKS_HOST#job/5663333963439 08:52 DEBUG [databricks.labs.ucx.installer.workflows] Validating migrate-external-tables-ctas workflow: https://DATABRICKS_HOST#job/5663333963439 08:41 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/2BT1 08:41 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.hiveserde_in_place_cygj0: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_cygj0 08:41 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.hiveserde_in_place_cygj0', metastore_id=None, name='hiveserde_in_place_cygj0', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 08:41 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_storage_dir fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_cygj0 08:41 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_cygj0.parquet_serde_cygj0: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_cygj0/parquet_serde_cygj0 08:41 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_cygj0.parquet_serde_cygj0', metastore_id=None, name='parquet_serde_cygj0', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='hiveserde_in_place_cygj0', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_cygj0/parquet_serde_cygj0', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 08:41 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_cygj0.orc_serde_cygj0: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_cygj0/orc_serde_cygj0 08:41 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_cygj0.orc_serde_cygj0', metastore_id=None, name='orc_serde_cygj0', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='hiveserde_in_place_cygj0', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_cygj0/orc_serde_cygj0', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 08:41 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_cygj0.avro_serde_cygj0: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_cygj0/avro_serde_cygj0 08:41 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_cygj0.avro_serde_cygj0', metastore_id=None, name='avro_serde_cygj0', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='hiveserde_in_place_cygj0', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_cygj0/avro_serde_cygj0', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 08:41 DEBUG [databricks.labs.ucx.mixins.fixtures] added catalog fixture: CatalogInfo(browse_only=False, catalog_type=, comment='', connection_name=None, created_at=1719909710449, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cipce', isolation_mode=, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_cipce', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1719909710449, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d') 08:41 INFO [databricks.labs.ucx.mixins.fixtures] Schema ucx_cipce.hiveserde_in_place_cygj0: https://DATABRICKS_HOST/explore/data/ucx_cipce/hiveserde_in_place_cygj0 08:41 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_cipce', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cipce.hiveserde_in_place_cygj0', metastore_id=None, name='hiveserde_in_place_cygj0', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 08:42 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_saauj: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_saauj 08:42 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_saauj', metastore_id=None, name='ucx_saauj', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 08:43 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.Rb5s/config.yml) doesn't exist. 08:43 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration 08:43 INFO [databricks.labs.ucx.installer.hms_lineage] HMS Lineage feature creates one system table named system.hms_to_uc_migration.table_access and helps in your migration process from HMS to UC by allowing you to programmatically query HMS lineage data. 08:43 INFO [databricks.labs.ucx.install] Fetching installations... 08:43 WARNING [databricks.labs.ucx.install] Existing installation at /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.Rb5s is corrupted. Skipping... 08:43 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy. 08:43 DEBUG [tests.integration.conftest] Waiting for clusters to start... 08:43 DEBUG [tests.integration.conftest] Waiting for clusters to start... 08:43 INFO [databricks.labs.ucx.install] Installing UCX v0.27.2+1920240702084322 08:43 INFO [databricks.labs.ucx.install] Creating ucx schemas... 08:44 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental 08:44 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=failing 08:44 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-data-reconciliation 08:44 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-tables-ctas 08:44 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-hiveserde-tables-in-place-experimental 08:44 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions 08:44 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-experimental 08:44 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=experimental-workflow-linter 08:44 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups 08:44 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment 08:44 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables 08:44 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=scan-tables-in-mounts-experimental 08:44 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups 08:44 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.Rb5s/README for the next steps. 08:44 DEBUG [databricks.labs.ucx.installer.workflows] starting migrate-external-tables-ctas job: https://DATABRICKS_HOST#job/5663333963439 08:52 DEBUG [databricks.labs.ucx.installer.workflows] Validating migrate-external-tables-ctas workflow: https://DATABRICKS_HOST#job/5663333963439 08:52 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 make_storage_dir fixtures 08:52 DEBUG [databricks.labs.ucx.mixins.fixtures] removing make_storage_dir fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_cygj0 08:52 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 make_dbfs_data_copy fixtures 08:52 DEBUG [databricks.labs.ucx.mixins.fixtures] removing make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/2BT1 08:52 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 cluster fixtures 08:52 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 catalog fixtures 08:52 DEBUG [databricks.labs.ucx.mixins.fixtures] removing catalog fixture: CatalogInfo(browse_only=False, catalog_type=, comment='', connection_name=None, created_at=1719909710449, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cipce', isolation_mode=, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_cipce', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1719909710449, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d') 08:52 INFO [databricks.labs.ucx.install] Deleting UCX v0.27.2+1920240702084322 from https://DATABRICKS_HOST 08:52 INFO [databricks.labs.ucx.install] Deleting inventory database ucx_saauj 08:52 INFO [databricks.labs.ucx.install] Deleting jobs 08:52 INFO [databricks.labs.ucx.install] Deleting migrate-tables-in-mounts-experimental job_id=824210279577082. 08:52 INFO [databricks.labs.ucx.install] Deleting failing job_id=710878812724859. 08:52 INFO [databricks.labs.ucx.install] Deleting migrate-data-reconciliation job_id=230362270253763. 08:52 INFO [databricks.labs.ucx.install] Deleting migrate-external-tables-ctas job_id=5663333963439. 08:52 INFO [databricks.labs.ucx.install] Deleting migrate-external-hiveserde-tables-in-place-experimental job_id=895922783195419. 08:52 INFO [databricks.labs.ucx.install] Deleting validate-groups-permissions job_id=133524984557082. 08:52 INFO [databricks.labs.ucx.install] Deleting migrate-groups-experimental job_id=274251280803703. 08:52 INFO [databricks.labs.ucx.install] Deleting experimental-workflow-linter job_id=250963531627626. 08:52 INFO [databricks.labs.ucx.install] Deleting remove-workspace-local-backup-groups job_id=741312385250175. 08:52 INFO [databricks.labs.ucx.install] Deleting assessment job_id=979753242188797. 08:52 INFO [databricks.labs.ucx.install] Deleting migrate-tables job_id=819901583135974. 08:52 INFO [databricks.labs.ucx.install] Deleting scan-tables-in-mounts-experimental job_id=102462560190794. 08:52 INFO [databricks.labs.ucx.install] Deleting migrate-groups job_id=826178644458442. 08:52 INFO [databricks.labs.ucx.install] Deleting cluster policy 08:52 INFO [databricks.labs.ucx.install] Deleting secret scope 08:52 INFO [databricks.labs.ucx.install] UnInstalling UCX complete 08:52 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 workspace user fixtures 08:52 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 account group fixtures 08:52 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 workspace group fixtures 08:52 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 table fixtures 08:52 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 3 table fixtures 08:52 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_cygj0.parquet_serde_cygj0', metastore_id=None, name='parquet_serde_cygj0', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='hiveserde_in_place_cygj0', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_cygj0/parquet_serde_cygj0', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 08:52 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_cygj0.orc_serde_cygj0', metastore_id=None, name='orc_serde_cygj0', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='hiveserde_in_place_cygj0', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_cygj0/orc_serde_cygj0', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 08:52 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_cygj0.avro_serde_cygj0', metastore_id=None, name='avro_serde_cygj0', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='hiveserde_in_place_cygj0', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_cygj0/avro_serde_cygj0', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 08:52 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 3 schema fixtures 08:52 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.hiveserde_in_place_cygj0', metastore_id=None, name='hiveserde_in_place_cygj0', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 08:52 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_cipce', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cipce.hiveserde_in_place_cygj0', metastore_id=None, name='hiveserde_in_place_cygj0', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 08:53 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_saauj', metastore_id=None, name='ucx_saauj', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) [gw3] linux -- Python 3.10.14 /home/runner/work/ucx/ucx/.venv/bin/python ```

_{Running from acceptance #4094}