Test failure: `test_table_migration_job[regular]`

[github-actions[bot]]

❌ test_table_migration_job[regular]: TimeoutError: timed out after 0:20:00: (21m38.029s)

``` TimeoutError: timed out after 0:20:00: 07:14 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/ahcd 07:14 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.migrate_ohhn3: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_ohhn3 07:14 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.migrate_ohhn3', metastore_id=None, name='migrate_ohhn3', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 07:14 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_ohhn3.ucx_t250s: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_ohhn3/ucx_t250s 07:14 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_ohhn3.ucx_t250s', metastore_id=None, name='ucx_t250s', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024062508'}, row_filter=None, schema_name='migrate_ohhn3', sql_path=None, storage_credential_name=None, storage_location='dbfs:/user/hive/warehouse/migrate_ohhn3/ucx_t250s', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 07:14 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_ohhn3.ucx_t9aqt: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_ohhn3/ucx_t9aqt 07:14 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_ohhn3.ucx_t9aqt', metastore_id=None, name='ucx_t9aqt', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024062508'}, row_filter=None, schema_name='migrate_ohhn3', sql_path=None, storage_credential_name=None, storage_location='dbfs:/tmp/ucx_test_Plfl', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 07:14 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_ohhn3.ucx_tdvqv: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_ohhn3/ucx_tdvqv 07:14 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_ohhn3.ucx_tdvqv', metastore_id=None, name='ucx_tdvqv', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024062508'}, row_filter=None, schema_name='migrate_ohhn3', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/b/ahcd', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 07:14 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_ohhn3.ucx_thm8y: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_ohhn3/ucx_thm8y 07:14 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_ohhn3.ucx_thm8y', metastore_id=None, name='ucx_thm8y', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024062508'}, row_filter=None, schema_name='migrate_ohhn3', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_ohhn3.ucx_t250s', view_dependencies=None) 07:14 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_ohhn3.ucx_txoio: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_ohhn3/ucx_txoio 07:14 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_ohhn3.ucx_txoio', metastore_id=None, name='ucx_txoio', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024062508'}, row_filter=None, schema_name='migrate_ohhn3', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_ohhn3.ucx_thm8y', view_dependencies=None) 07:14 DEBUG [databricks.labs.ucx.mixins.fixtures] added catalog fixture: CatalogInfo(browse_only=False, catalog_type=, comment='', connection_name=None, created_at=1719299673339, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cx459', isolation_mode=, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_cx459', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1719299673339, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d') 07:14 INFO [databricks.labs.ucx.mixins.fixtures] Schema ucx_cx459.migrate_ohhn3: https://DATABRICKS_HOST/explore/data/ucx_cx459/migrate_ohhn3 07:14 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_cx459', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cx459.migrate_ohhn3', metastore_id=None, name='migrate_ohhn3', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 07:14 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_srutz: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_srutz 07:14 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_srutz', metastore_id=None, name='ucx_srutz', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) [gw4] linux -- Python 3.10.14 /home/runner/work/ucx/ucx/.venv/bin/python 07:14 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/ahcd 07:14 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.migrate_ohhn3: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_ohhn3 07:14 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.migrate_ohhn3', metastore_id=None, name='migrate_ohhn3', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 07:14 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_ohhn3.ucx_t250s: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_ohhn3/ucx_t250s 07:14 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_ohhn3.ucx_t250s', metastore_id=None, name='ucx_t250s', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024062508'}, row_filter=None, schema_name='migrate_ohhn3', sql_path=None, storage_credential_name=None, storage_location='dbfs:/user/hive/warehouse/migrate_ohhn3/ucx_t250s', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 07:14 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_ohhn3.ucx_t9aqt: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_ohhn3/ucx_t9aqt 07:14 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_ohhn3.ucx_t9aqt', metastore_id=None, name='ucx_t9aqt', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024062508'}, row_filter=None, schema_name='migrate_ohhn3', sql_path=None, storage_credential_name=None, storage_location='dbfs:/tmp/ucx_test_Plfl', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 07:14 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_ohhn3.ucx_tdvqv: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_ohhn3/ucx_tdvqv 07:14 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_ohhn3.ucx_tdvqv', metastore_id=None, name='ucx_tdvqv', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024062508'}, row_filter=None, schema_name='migrate_ohhn3', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/b/ahcd', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 07:14 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_ohhn3.ucx_thm8y: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_ohhn3/ucx_thm8y 07:14 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_ohhn3.ucx_thm8y', metastore_id=None, name='ucx_thm8y', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024062508'}, row_filter=None, schema_name='migrate_ohhn3', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_ohhn3.ucx_t250s', view_dependencies=None) 07:14 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_ohhn3.ucx_txoio: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_ohhn3/ucx_txoio 07:14 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_ohhn3.ucx_txoio', metastore_id=None, name='ucx_txoio', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024062508'}, row_filter=None, schema_name='migrate_ohhn3', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_ohhn3.ucx_thm8y', view_dependencies=None) 07:14 DEBUG [databricks.labs.ucx.mixins.fixtures] added catalog fixture: CatalogInfo(browse_only=False, catalog_type=, comment='', connection_name=None, created_at=1719299673339, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cx459', isolation_mode=, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_cx459', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1719299673339, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d') 07:14 INFO [databricks.labs.ucx.mixins.fixtures] Schema ucx_cx459.migrate_ohhn3: https://DATABRICKS_HOST/explore/data/ucx_cx459/migrate_ohhn3 07:14 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_cx459', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cx459.migrate_ohhn3', metastore_id=None, name='migrate_ohhn3', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 07:14 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_srutz: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_srutz 07:14 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_srutz', metastore_id=None, name='ucx_srutz', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 07:14 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.rSqp/config.yml) doesn't exist. 07:14 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration 07:14 INFO [databricks.labs.ucx.installer.hms_lineage] HMS Lineage feature creates one system table named system.hms_to_uc_migration.table_access and helps in your migration process from HMS to UC by allowing you to programmatically query HMS lineage data. 07:14 INFO [databricks.labs.ucx.install] Fetching installations... 07:14 WARNING [databricks.labs.ucx.install] Existing installation at /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.rSqp is corrupted. Skipping... 07:14 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy. 07:14 DEBUG [tests.integration.conftest] Waiting for clusters to start... 07:14 DEBUG [tests.integration.conftest] Waiting for clusters to start... 07:14 INFO [databricks.labs.ucx.install] Installing UCX v0.27.2+1120240625071456 07:14 INFO [databricks.labs.ucx.install] Creating ucx schemas... 07:15 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups 07:15 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-experimental 07:15 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-hiveserde-tables-in-place-experimental 07:15 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental 07:15 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=failing 07:15 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups 07:15 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=experimental-workflow-linter 07:15 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions 07:15 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-data-reconciliation 07:15 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-tables-ctas 07:15 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables 07:15 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=scan-tables-in-mounts-experimental 07:15 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment 07:15 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.rSqp/README for the next steps. 07:15 DEBUG [databricks.labs.ucx.installer.workflows] starting migrate-tables job: https://DATABRICKS_HOST#job/991667432963772 07:14 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/ahcd 07:14 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.migrate_ohhn3: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_ohhn3 07:14 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.migrate_ohhn3', metastore_id=None, name='migrate_ohhn3', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 07:14 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_ohhn3.ucx_t250s: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_ohhn3/ucx_t250s 07:14 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_ohhn3.ucx_t250s', metastore_id=None, name='ucx_t250s', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024062508'}, row_filter=None, schema_name='migrate_ohhn3', sql_path=None, storage_credential_name=None, storage_location='dbfs:/user/hive/warehouse/migrate_ohhn3/ucx_t250s', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 07:14 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_ohhn3.ucx_t9aqt: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_ohhn3/ucx_t9aqt 07:14 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_ohhn3.ucx_t9aqt', metastore_id=None, name='ucx_t9aqt', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024062508'}, row_filter=None, schema_name='migrate_ohhn3', sql_path=None, storage_credential_name=None, storage_location='dbfs:/tmp/ucx_test_Plfl', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 07:14 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_ohhn3.ucx_tdvqv: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_ohhn3/ucx_tdvqv 07:14 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_ohhn3.ucx_tdvqv', metastore_id=None, name='ucx_tdvqv', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024062508'}, row_filter=None, schema_name='migrate_ohhn3', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/b/ahcd', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 07:14 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_ohhn3.ucx_thm8y: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_ohhn3/ucx_thm8y 07:14 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_ohhn3.ucx_thm8y', metastore_id=None, name='ucx_thm8y', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024062508'}, row_filter=None, schema_name='migrate_ohhn3', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_ohhn3.ucx_t250s', view_dependencies=None) 07:14 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_ohhn3.ucx_txoio: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_ohhn3/ucx_txoio 07:14 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_ohhn3.ucx_txoio', metastore_id=None, name='ucx_txoio', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024062508'}, row_filter=None, schema_name='migrate_ohhn3', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_ohhn3.ucx_thm8y', view_dependencies=None) 07:14 DEBUG [databricks.labs.ucx.mixins.fixtures] added catalog fixture: CatalogInfo(browse_only=False, catalog_type=, comment='', connection_name=None, created_at=1719299673339, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cx459', isolation_mode=, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_cx459', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1719299673339, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d') 07:14 INFO [databricks.labs.ucx.mixins.fixtures] Schema ucx_cx459.migrate_ohhn3: https://DATABRICKS_HOST/explore/data/ucx_cx459/migrate_ohhn3 07:14 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_cx459', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cx459.migrate_ohhn3', metastore_id=None, name='migrate_ohhn3', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 07:14 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_srutz: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_srutz 07:14 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_srutz', metastore_id=None, name='ucx_srutz', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 07:14 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.rSqp/config.yml) doesn't exist. 07:14 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration 07:14 INFO [databricks.labs.ucx.installer.hms_lineage] HMS Lineage feature creates one system table named system.hms_to_uc_migration.table_access and helps in your migration process from HMS to UC by allowing you to programmatically query HMS lineage data. 07:14 INFO [databricks.labs.ucx.install] Fetching installations... 07:14 WARNING [databricks.labs.ucx.install] Existing installation at /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.rSqp is corrupted. Skipping... 07:14 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy. 07:14 DEBUG [tests.integration.conftest] Waiting for clusters to start... 07:14 DEBUG [tests.integration.conftest] Waiting for clusters to start... 07:14 INFO [databricks.labs.ucx.install] Installing UCX v0.27.2+1120240625071456 07:14 INFO [databricks.labs.ucx.install] Creating ucx schemas... 07:15 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups 07:15 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-experimental 07:15 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-hiveserde-tables-in-place-experimental 07:15 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental 07:15 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=failing 07:15 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups 07:15 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=experimental-workflow-linter 07:15 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions 07:15 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-data-reconciliation 07:15 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-tables-ctas 07:15 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables 07:15 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=scan-tables-in-mounts-experimental 07:15 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment 07:15 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.rSqp/README for the next steps. 07:15 DEBUG [databricks.labs.ucx.installer.workflows] starting migrate-tables job: https://DATABRICKS_HOST#job/991667432963772 07:35 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 make_storage_dir fixtures 07:35 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 make_dbfs_data_copy fixtures 07:35 DEBUG [databricks.labs.ucx.mixins.fixtures] removing make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/ahcd 07:35 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 cluster fixtures 07:35 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 catalog fixtures 07:35 DEBUG [databricks.labs.ucx.mixins.fixtures] removing catalog fixture: CatalogInfo(browse_only=False, catalog_type=, comment='', connection_name=None, created_at=1719299673339, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cx459', isolation_mode=, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_cx459', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1719299673339, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d') 07:35 INFO [databricks.labs.ucx.install] Deleting UCX v0.27.2+1120240625071456 from https://DATABRICKS_HOST 07:35 INFO [databricks.labs.ucx.install] Deleting inventory database ucx_srutz 07:35 INFO [databricks.labs.ucx.install] Deleting jobs 07:35 INFO [databricks.labs.ucx.install] Deleting migrate-groups job_id=1050128164411083. 07:35 INFO [databricks.labs.ucx.install] Deleting migrate-groups-experimental job_id=143260075558555. 07:35 INFO [databricks.labs.ucx.install] Deleting migrate-external-hiveserde-tables-in-place-experimental job_id=727894841111343. 07:35 INFO [databricks.labs.ucx.install] Deleting migrate-tables-in-mounts-experimental job_id=675554858758246. 07:35 INFO [databricks.labs.ucx.install] Deleting failing job_id=537078797864767. 07:35 INFO [databricks.labs.ucx.install] Deleting remove-workspace-local-backup-groups job_id=1047856248446924. 07:35 INFO [databricks.labs.ucx.install] Deleting experimental-workflow-linter job_id=266837503193869. 07:35 INFO [databricks.labs.ucx.install] Deleting validate-groups-permissions job_id=1001838687281421. 07:35 INFO [databricks.labs.ucx.install] Deleting migrate-data-reconciliation job_id=124070441881511. 07:35 INFO [databricks.labs.ucx.install] Deleting migrate-external-tables-ctas job_id=772012863748265. 07:35 INFO [databricks.labs.ucx.install] Deleting migrate-tables job_id=991667432963772. 07:35 INFO [databricks.labs.ucx.install] Deleting scan-tables-in-mounts-experimental job_id=978816194719827. 07:35 INFO [databricks.labs.ucx.install] Deleting assessment job_id=692370381514253. 07:35 INFO [databricks.labs.ucx.install] Deleting cluster policy 07:35 INFO [databricks.labs.ucx.install] Deleting secret scope 07:35 INFO [databricks.labs.ucx.install] UnInstalling UCX complete 07:35 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 workspace user fixtures 07:35 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 account group fixtures 07:35 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 workspace group fixtures 07:35 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 table fixtures 07:35 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 5 table fixtures 07:35 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_ohhn3.ucx_t250s', metastore_id=None, name='ucx_t250s', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024062508'}, row_filter=None, schema_name='migrate_ohhn3', sql_path=None, storage_credential_name=None, storage_location='dbfs:/user/hive/warehouse/migrate_ohhn3/ucx_t250s', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 07:35 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_ohhn3.ucx_t9aqt', metastore_id=None, name='ucx_t9aqt', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024062508'}, row_filter=None, schema_name='migrate_ohhn3', sql_path=None, storage_credential_name=None, storage_location='dbfs:/tmp/ucx_test_Plfl', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 07:35 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_ohhn3.ucx_tdvqv', metastore_id=None, name='ucx_tdvqv', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024062508'}, row_filter=None, schema_name='migrate_ohhn3', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/b/ahcd', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 07:35 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_ohhn3.ucx_thm8y', metastore_id=None, name='ucx_thm8y', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024062508'}, row_filter=None, schema_name='migrate_ohhn3', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_ohhn3.ucx_t250s', view_dependencies=None) 07:35 DEBUG [databricks.labs.ucx.mixins.fixtures] ignoring error while table TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_ohhn3.ucx_thm8y', metastore_id=None, name='ucx_thm8y', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024062508'}, row_filter=None, schema_name='migrate_ohhn3', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_ohhn3.ucx_t250s', view_dependencies=None) teardown: [WRONG_COMMAND_FOR_OBJECT_TYPE] The operation DROP TABLE requires a EXTERNAL or MANAGED. But hive_metastore.migrate_ohhn3.ucx_thm8y is a VIEW. Use DROP VIEW instead. SQLSTATE: 42809 07:35 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_ohhn3.ucx_txoio', metastore_id=None, name='ucx_txoio', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024062508'}, row_filter=None, schema_name='migrate_ohhn3', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_ohhn3.ucx_thm8y', view_dependencies=None) 07:35 DEBUG [databricks.labs.ucx.mixins.fixtures] ignoring error while table TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_ohhn3.ucx_txoio', metastore_id=None, name='ucx_txoio', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024062508'}, row_filter=None, schema_name='migrate_ohhn3', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_ohhn3.ucx_thm8y', view_dependencies=None) teardown: [WRONG_COMMAND_FOR_OBJECT_TYPE] The operation DROP TABLE requires a EXTERNAL or MANAGED. But hive_metastore.migrate_ohhn3.ucx_txoio is a VIEW. Use DROP VIEW instead. SQLSTATE: 42809 07:35 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 3 schema fixtures 07:35 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.migrate_ohhn3', metastore_id=None, name='migrate_ohhn3', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 07:35 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_cx459', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cx459.migrate_ohhn3', metastore_id=None, name='migrate_ohhn3', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 07:36 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_srutz', metastore_id=None, name='ucx_srutz', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) [gw4] linux -- Python 3.10.14 /home/runner/work/ucx/ucx/.venv/bin/python ```

_{Running from nightly #102}

[github-actions[bot]]

❌ test_table_migration_job[regular]: TimeoutError: timed out after 0:20:00: (21m52.317s)

``` TimeoutError: timed out after 0:20:00: 07:15 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/Vul0 07:15 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.migrate_qw4qe: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_qw4qe 07:15 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.migrate_qw4qe', metastore_id=None, name='migrate_qw4qe', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 07:15 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_qw4qe.ucx_tisrk: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_qw4qe/ucx_tisrk 07:15 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_qw4qe.ucx_tisrk', metastore_id=None, name='ucx_tisrk', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070108'}, row_filter=None, schema_name='migrate_qw4qe', sql_path=None, storage_credential_name=None, storage_location='dbfs:/user/hive/warehouse/migrate_qw4qe/ucx_tisrk', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 07:15 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_qw4qe.ucx_txfih: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_qw4qe/ucx_txfih 07:15 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_qw4qe.ucx_txfih', metastore_id=None, name='ucx_txfih', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070108'}, row_filter=None, schema_name='migrate_qw4qe', sql_path=None, storage_credential_name=None, storage_location='dbfs:/tmp/ucx_test_N144', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 07:15 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_qw4qe.ucx_te9ik: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_qw4qe/ucx_te9ik 07:15 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_qw4qe.ucx_te9ik', metastore_id=None, name='ucx_te9ik', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070108'}, row_filter=None, schema_name='migrate_qw4qe', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/b/Vul0', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 07:15 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_qw4qe.ucx_tlskj: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_qw4qe/ucx_tlskj 07:15 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_qw4qe.ucx_tlskj', metastore_id=None, name='ucx_tlskj', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070108'}, row_filter=None, schema_name='migrate_qw4qe', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_qw4qe.ucx_tisrk', view_dependencies=None) 07:15 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_qw4qe.ucx_tvirs: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_qw4qe/ucx_tvirs 07:15 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_qw4qe.ucx_tvirs', metastore_id=None, name='ucx_tvirs', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070108'}, row_filter=None, schema_name='migrate_qw4qe', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_qw4qe.ucx_tlskj', view_dependencies=None) 07:15 DEBUG [databricks.labs.ucx.mixins.fixtures] added catalog fixture: CatalogInfo(browse_only=False, catalog_type=, comment='', connection_name=None, created_at=1719818131757, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cwwto', isolation_mode=, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_cwwto', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1719818131757, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d') 07:15 INFO [databricks.labs.ucx.mixins.fixtures] Schema ucx_cwwto.migrate_qw4qe: https://DATABRICKS_HOST/explore/data/ucx_cwwto/migrate_qw4qe 07:15 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_cwwto', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cwwto.migrate_qw4qe', metastore_id=None, name='migrate_qw4qe', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 07:15 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_s9bvq: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_s9bvq 07:15 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_s9bvq', metastore_id=None, name='ucx_s9bvq', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) [gw6] linux -- Python 3.10.14 /home/runner/work/ucx/ucx/.venv/bin/python 07:15 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/Vul0 07:15 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.migrate_qw4qe: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_qw4qe 07:15 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.migrate_qw4qe', metastore_id=None, name='migrate_qw4qe', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 07:15 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_qw4qe.ucx_tisrk: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_qw4qe/ucx_tisrk 07:15 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_qw4qe.ucx_tisrk', metastore_id=None, name='ucx_tisrk', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070108'}, row_filter=None, schema_name='migrate_qw4qe', sql_path=None, storage_credential_name=None, storage_location='dbfs:/user/hive/warehouse/migrate_qw4qe/ucx_tisrk', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 07:15 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_qw4qe.ucx_txfih: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_qw4qe/ucx_txfih 07:15 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_qw4qe.ucx_txfih', metastore_id=None, name='ucx_txfih', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070108'}, row_filter=None, schema_name='migrate_qw4qe', sql_path=None, storage_credential_name=None, storage_location='dbfs:/tmp/ucx_test_N144', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 07:15 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_qw4qe.ucx_te9ik: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_qw4qe/ucx_te9ik 07:15 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_qw4qe.ucx_te9ik', metastore_id=None, name='ucx_te9ik', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070108'}, row_filter=None, schema_name='migrate_qw4qe', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/b/Vul0', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 07:15 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_qw4qe.ucx_tlskj: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_qw4qe/ucx_tlskj 07:15 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_qw4qe.ucx_tlskj', metastore_id=None, name='ucx_tlskj', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070108'}, row_filter=None, schema_name='migrate_qw4qe', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_qw4qe.ucx_tisrk', view_dependencies=None) 07:15 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_qw4qe.ucx_tvirs: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_qw4qe/ucx_tvirs 07:15 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_qw4qe.ucx_tvirs', metastore_id=None, name='ucx_tvirs', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070108'}, row_filter=None, schema_name='migrate_qw4qe', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_qw4qe.ucx_tlskj', view_dependencies=None) 07:15 DEBUG [databricks.labs.ucx.mixins.fixtures] added catalog fixture: CatalogInfo(browse_only=False, catalog_type=, comment='', connection_name=None, created_at=1719818131757, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cwwto', isolation_mode=, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_cwwto', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1719818131757, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d') 07:15 INFO [databricks.labs.ucx.mixins.fixtures] Schema ucx_cwwto.migrate_qw4qe: https://DATABRICKS_HOST/explore/data/ucx_cwwto/migrate_qw4qe 07:15 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_cwwto', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cwwto.migrate_qw4qe', metastore_id=None, name='migrate_qw4qe', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 07:15 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_s9bvq: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_s9bvq 07:15 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_s9bvq', metastore_id=None, name='ucx_s9bvq', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 07:15 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.XpIz/config.yml) doesn't exist. 07:15 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration 07:15 INFO [databricks.labs.ucx.installer.hms_lineage] HMS Lineage feature creates one system table named system.hms_to_uc_migration.table_access and helps in your migration process from HMS to UC by allowing you to programmatically query HMS lineage data. 07:15 INFO [databricks.labs.ucx.install] Fetching installations... 07:16 WARNING [databricks.labs.ucx.install] Existing installation at /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.XpIz is corrupted. Skipping... 07:16 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy. 07:16 DEBUG [tests.integration.conftest] Waiting for clusters to start... 07:16 DEBUG [tests.integration.conftest] Waiting for clusters to start... 07:16 INFO [databricks.labs.ucx.install] Installing UCX v0.27.2+1320240701071608 07:16 INFO [databricks.labs.ucx.install] Creating ucx schemas... 07:16 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-data-reconciliation 07:16 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables 07:16 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental 07:16 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-tables-ctas 07:16 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-experimental 07:16 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=experimental-workflow-linter 07:16 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=failing 07:16 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups 07:16 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-hiveserde-tables-in-place-experimental 07:16 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=scan-tables-in-mounts-experimental 07:16 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment 07:16 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions 07:16 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups 07:16 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.XpIz/README for the next steps. 07:16 DEBUG [databricks.labs.ucx.installer.workflows] starting migrate-tables job: https://DATABRICKS_HOST#job/191647837048877 07:15 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/Vul0 07:15 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.migrate_qw4qe: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_qw4qe 07:15 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.migrate_qw4qe', metastore_id=None, name='migrate_qw4qe', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 07:15 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_qw4qe.ucx_tisrk: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_qw4qe/ucx_tisrk 07:15 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_qw4qe.ucx_tisrk', metastore_id=None, name='ucx_tisrk', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070108'}, row_filter=None, schema_name='migrate_qw4qe', sql_path=None, storage_credential_name=None, storage_location='dbfs:/user/hive/warehouse/migrate_qw4qe/ucx_tisrk', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 07:15 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_qw4qe.ucx_txfih: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_qw4qe/ucx_txfih 07:15 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_qw4qe.ucx_txfih', metastore_id=None, name='ucx_txfih', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070108'}, row_filter=None, schema_name='migrate_qw4qe', sql_path=None, storage_credential_name=None, storage_location='dbfs:/tmp/ucx_test_N144', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 07:15 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_qw4qe.ucx_te9ik: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_qw4qe/ucx_te9ik 07:15 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_qw4qe.ucx_te9ik', metastore_id=None, name='ucx_te9ik', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070108'}, row_filter=None, schema_name='migrate_qw4qe', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/b/Vul0', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 07:15 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_qw4qe.ucx_tlskj: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_qw4qe/ucx_tlskj 07:15 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_qw4qe.ucx_tlskj', metastore_id=None, name='ucx_tlskj', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070108'}, row_filter=None, schema_name='migrate_qw4qe', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_qw4qe.ucx_tisrk', view_dependencies=None) 07:15 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_qw4qe.ucx_tvirs: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_qw4qe/ucx_tvirs 07:15 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_qw4qe.ucx_tvirs', metastore_id=None, name='ucx_tvirs', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070108'}, row_filter=None, schema_name='migrate_qw4qe', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_qw4qe.ucx_tlskj', view_dependencies=None) 07:15 DEBUG [databricks.labs.ucx.mixins.fixtures] added catalog fixture: CatalogInfo(browse_only=False, catalog_type=, comment='', connection_name=None, created_at=1719818131757, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cwwto', isolation_mode=, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_cwwto', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1719818131757, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d') 07:15 INFO [databricks.labs.ucx.mixins.fixtures] Schema ucx_cwwto.migrate_qw4qe: https://DATABRICKS_HOST/explore/data/ucx_cwwto/migrate_qw4qe 07:15 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_cwwto', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cwwto.migrate_qw4qe', metastore_id=None, name='migrate_qw4qe', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 07:15 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_s9bvq: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_s9bvq 07:15 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_s9bvq', metastore_id=None, name='ucx_s9bvq', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 07:15 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.XpIz/config.yml) doesn't exist. 07:15 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration 07:15 INFO [databricks.labs.ucx.installer.hms_lineage] HMS Lineage feature creates one system table named system.hms_to_uc_migration.table_access and helps in your migration process from HMS to UC by allowing you to programmatically query HMS lineage data. 07:15 INFO [databricks.labs.ucx.install] Fetching installations... 07:16 WARNING [databricks.labs.ucx.install] Existing installation at /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.XpIz is corrupted. Skipping... 07:16 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy. 07:16 DEBUG [tests.integration.conftest] Waiting for clusters to start... 07:16 DEBUG [tests.integration.conftest] Waiting for clusters to start... 07:16 INFO [databricks.labs.ucx.install] Installing UCX v0.27.2+1320240701071608 07:16 INFO [databricks.labs.ucx.install] Creating ucx schemas... 07:16 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-data-reconciliation 07:16 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables 07:16 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental 07:16 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-tables-ctas 07:16 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-experimental 07:16 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=experimental-workflow-linter 07:16 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=failing 07:16 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups 07:16 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-hiveserde-tables-in-place-experimental 07:16 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=scan-tables-in-mounts-experimental 07:16 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment 07:16 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions 07:16 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups 07:16 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.XpIz/README for the next steps. 07:16 DEBUG [databricks.labs.ucx.installer.workflows] starting migrate-tables job: https://DATABRICKS_HOST#job/191647837048877 07:36 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 make_storage_dir fixtures 07:36 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 make_dbfs_data_copy fixtures 07:36 DEBUG [databricks.labs.ucx.mixins.fixtures] removing make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/Vul0 07:36 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 cluster fixtures 07:36 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 catalog fixtures 07:36 DEBUG [databricks.labs.ucx.mixins.fixtures] removing catalog fixture: CatalogInfo(browse_only=False, catalog_type=, comment='', connection_name=None, created_at=1719818131757, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cwwto', isolation_mode=, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_cwwto', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1719818131757, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d') 07:36 INFO [databricks.labs.ucx.install] Deleting UCX v0.27.2+1320240701071608 from https://DATABRICKS_HOST 07:36 INFO [databricks.labs.ucx.install] Deleting inventory database ucx_s9bvq 07:36 INFO [databricks.labs.ucx.install] Deleting jobs 07:36 INFO [databricks.labs.ucx.install] Deleting migrate-data-reconciliation job_id=116638703378763. 07:36 INFO [databricks.labs.ucx.install] Deleting migrate-tables job_id=191647837048877. 07:36 INFO [databricks.labs.ucx.install] Deleting migrate-tables-in-mounts-experimental job_id=436242349524147. 07:36 INFO [databricks.labs.ucx.install] Deleting migrate-external-tables-ctas job_id=946006745284361. 07:36 INFO [databricks.labs.ucx.install] Deleting migrate-groups-experimental job_id=1018717195622275. 07:36 INFO [databricks.labs.ucx.install] Deleting experimental-workflow-linter job_id=438185648883023. 07:36 INFO [databricks.labs.ucx.install] Deleting failing job_id=87228517371129. 07:36 INFO [databricks.labs.ucx.install] Deleting remove-workspace-local-backup-groups job_id=1037126949287739. 07:36 INFO [databricks.labs.ucx.install] Deleting migrate-external-hiveserde-tables-in-place-experimental job_id=90409082879275. 07:36 INFO [databricks.labs.ucx.install] Deleting scan-tables-in-mounts-experimental job_id=240166311459885. 07:36 INFO [databricks.labs.ucx.install] Deleting assessment job_id=175475860435545. 07:36 INFO [databricks.labs.ucx.install] Deleting validate-groups-permissions job_id=982452748750080. 07:36 INFO [databricks.labs.ucx.install] Deleting migrate-groups job_id=232449754778088. 07:36 INFO [databricks.labs.ucx.install] Deleting cluster policy 07:36 INFO [databricks.labs.ucx.install] Deleting secret scope 07:37 INFO [databricks.labs.ucx.install] UnInstalling UCX complete 07:37 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 workspace user fixtures 07:37 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 account group fixtures 07:37 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 workspace group fixtures 07:37 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 table fixtures 07:37 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 5 table fixtures 07:37 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_qw4qe.ucx_tisrk', metastore_id=None, name='ucx_tisrk', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070108'}, row_filter=None, schema_name='migrate_qw4qe', sql_path=None, storage_credential_name=None, storage_location='dbfs:/user/hive/warehouse/migrate_qw4qe/ucx_tisrk', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 07:37 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_qw4qe.ucx_txfih', metastore_id=None, name='ucx_txfih', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070108'}, row_filter=None, schema_name='migrate_qw4qe', sql_path=None, storage_credential_name=None, storage_location='dbfs:/tmp/ucx_test_N144', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 07:37 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_qw4qe.ucx_te9ik', metastore_id=None, name='ucx_te9ik', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070108'}, row_filter=None, schema_name='migrate_qw4qe', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/b/Vul0', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 07:37 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_qw4qe.ucx_tlskj', metastore_id=None, name='ucx_tlskj', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070108'}, row_filter=None, schema_name='migrate_qw4qe', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_qw4qe.ucx_tisrk', view_dependencies=None) 07:37 DEBUG [databricks.labs.ucx.mixins.fixtures] ignoring error while table TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_qw4qe.ucx_tlskj', metastore_id=None, name='ucx_tlskj', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070108'}, row_filter=None, schema_name='migrate_qw4qe', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_qw4qe.ucx_tisrk', view_dependencies=None) teardown: [WRONG_COMMAND_FOR_OBJECT_TYPE] The operation DROP TABLE requires a EXTERNAL or MANAGED. But hive_metastore.migrate_qw4qe.ucx_tlskj is a VIEW. Use DROP VIEW instead. SQLSTATE: 42809 07:37 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_qw4qe.ucx_tvirs', metastore_id=None, name='ucx_tvirs', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070108'}, row_filter=None, schema_name='migrate_qw4qe', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_qw4qe.ucx_tlskj', view_dependencies=None) 07:37 DEBUG [databricks.labs.ucx.mixins.fixtures] ignoring error while table TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_qw4qe.ucx_tvirs', metastore_id=None, name='ucx_tvirs', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070108'}, row_filter=None, schema_name='migrate_qw4qe', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_qw4qe.ucx_tlskj', view_dependencies=None) teardown: [WRONG_COMMAND_FOR_OBJECT_TYPE] The operation DROP TABLE requires a EXTERNAL or MANAGED. But hive_metastore.migrate_qw4qe.ucx_tvirs is a VIEW. Use DROP VIEW instead. SQLSTATE: 42809 07:37 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 3 schema fixtures 07:37 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.migrate_qw4qe', metastore_id=None, name='migrate_qw4qe', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 07:37 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_cwwto', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cwwto.migrate_qw4qe', metastore_id=None, name='migrate_qw4qe', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 07:37 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_s9bvq', metastore_id=None, name='ucx_s9bvq', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) [gw6] linux -- Python 3.10.14 /home/runner/work/ucx/ucx/.venv/bin/python ```

_{Running from nightly #108}