whitelist databricks-arc

❌ 121/128 passed, 7 failed, 22 skipped, 5h4m23s total

❌ test_table_migration_job_refreshes_migration_status[regular-migrate-tables]: TimeoutError: Timed out after 0:05:00 (8m39.495s)

``` TimeoutError: Timed out after 0:05:00 17:46 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/kjL6 17:46 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.migrate_4qooh: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_4qooh 17:46 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.migrate_4qooh', metastore_id=None, name='migrate_4qooh', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 17:46 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_4qooh.ucx_tojef: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_4qooh/ucx_tojef 17:46 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_4qooh.ucx_tojef', metastore_id=None, name='ucx_tojef', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='migrate_4qooh', sql_path=None, storage_credential_name=None, storage_location='dbfs:/user/hive/warehouse/migrate_4qooh/ucx_tojef', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 17:46 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_4qooh.ucx_tdubs: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_4qooh/ucx_tdubs 17:46 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_4qooh.ucx_tdubs', metastore_id=None, name='ucx_tdubs', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='migrate_4qooh', sql_path=None, storage_credential_name=None, storage_location='dbfs:/tmp/ucx_test_CPYQ', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 17:46 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_4qooh.ucx_tmj98: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_4qooh/ucx_tmj98 17:46 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_4qooh.ucx_tmj98', metastore_id=None, name='ucx_tmj98', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='migrate_4qooh', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/b/kjL6', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 17:46 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_4qooh.ucx_tzpzx: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_4qooh/ucx_tzpzx 17:46 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_4qooh.ucx_tzpzx', metastore_id=None, name='ucx_tzpzx', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='migrate_4qooh', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_4qooh.ucx_tojef', view_dependencies=None) 17:47 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_4qooh.ucx_tar1x: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_4qooh/ucx_tar1x 17:47 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_4qooh.ucx_tar1x', metastore_id=None, name='ucx_tar1x', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='migrate_4qooh', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_4qooh.ucx_tzpzx', view_dependencies=None) 17:47 DEBUG [databricks.labs.ucx.mixins.fixtures] added catalog fixture: CatalogInfo(browse_only=False, catalog_type=, comment='', connection_name=None, created_at=1719942426195, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_chof1', isolation_mode=, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_chof1', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1719942426195, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d') 17:47 INFO [databricks.labs.ucx.mixins.fixtures] Schema ucx_chof1.migrate_4qooh: https://DATABRICKS_HOST/explore/data/ucx_chof1/migrate_4qooh 17:47 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_chof1', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_chof1.migrate_4qooh', metastore_id=None, name='migrate_4qooh', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 17:47 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_swlzx: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_swlzx 17:47 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_swlzx', metastore_id=None, name='ucx_swlzx', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) [gw0] linux -- Python 3.10.14 /home/runner/work/ucx/ucx/.venv/bin/python 17:46 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/kjL6 17:46 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.migrate_4qooh: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_4qooh 17:46 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.migrate_4qooh', metastore_id=None, name='migrate_4qooh', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 17:46 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_4qooh.ucx_tojef: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_4qooh/ucx_tojef 17:46 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_4qooh.ucx_tojef', metastore_id=None, name='ucx_tojef', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='migrate_4qooh', sql_path=None, storage_credential_name=None, storage_location='dbfs:/user/hive/warehouse/migrate_4qooh/ucx_tojef', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 17:46 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_4qooh.ucx_tdubs: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_4qooh/ucx_tdubs 17:46 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_4qooh.ucx_tdubs', metastore_id=None, name='ucx_tdubs', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='migrate_4qooh', sql_path=None, storage_credential_name=None, storage_location='dbfs:/tmp/ucx_test_CPYQ', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 17:46 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_4qooh.ucx_tmj98: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_4qooh/ucx_tmj98 17:46 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_4qooh.ucx_tmj98', metastore_id=None, name='ucx_tmj98', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='migrate_4qooh', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/b/kjL6', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 17:46 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_4qooh.ucx_tzpzx: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_4qooh/ucx_tzpzx 17:46 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_4qooh.ucx_tzpzx', metastore_id=None, name='ucx_tzpzx', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='migrate_4qooh', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_4qooh.ucx_tojef', view_dependencies=None) 17:47 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_4qooh.ucx_tar1x: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_4qooh/ucx_tar1x 17:47 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_4qooh.ucx_tar1x', metastore_id=None, name='ucx_tar1x', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='migrate_4qooh', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_4qooh.ucx_tzpzx', view_dependencies=None) 17:47 DEBUG [databricks.labs.ucx.mixins.fixtures] added catalog fixture: CatalogInfo(browse_only=False, catalog_type=, comment='', connection_name=None, created_at=1719942426195, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_chof1', isolation_mode=, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_chof1', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1719942426195, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d') 17:47 INFO [databricks.labs.ucx.mixins.fixtures] Schema ucx_chof1.migrate_4qooh: https://DATABRICKS_HOST/explore/data/ucx_chof1/migrate_4qooh 17:47 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_chof1', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_chof1.migrate_4qooh', metastore_id=None, name='migrate_4qooh', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 17:47 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_swlzx: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_swlzx 17:47 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_swlzx', metastore_id=None, name='ucx_swlzx', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 17:47 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.H6wL/config.yml) doesn't exist. 17:47 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration 17:47 INFO [databricks.labs.ucx.installer.hms_lineage] HMS Lineage feature creates one system table named system.hms_to_uc_migration.table_access and helps in your migration process from HMS to UC by allowing you to programmatically query HMS lineage data. 17:47 INFO [databricks.labs.ucx.install] Fetching installations... 17:47 WARNING [databricks.labs.ucx.install] Existing installation at /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.H6wL is corrupted. Skipping... 17:48 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy. 17:48 DEBUG [tests.integration.conftest] Waiting for clusters to start... 17:48 DEBUG [tests.integration.conftest] Waiting for clusters to start... 17:48 INFO [databricks.labs.ucx.install] Installing UCX v0.27.2+6120240702174859 17:48 INFO [databricks.labs.ucx.install] Creating ucx schemas... 17:50 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=experimental-workflow-linter 17:50 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-tables-ctas 17:50 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups 17:50 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions 17:50 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment 17:50 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-experimental 17:50 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-data-reconciliation 17:50 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups 17:50 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=failing 17:50 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-hiveserde-tables-in-place-experimental 17:50 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=scan-tables-in-mounts-experimental 17:50 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental 17:50 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables 17:50 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.H6wL/README for the next steps. 17:50 DEBUG [databricks.labs.ucx.installer.workflows] starting migrate-tables job: https://DATABRICKS_HOST#job/563698022716326 17:46 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/kjL6 17:46 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.migrate_4qooh: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_4qooh 17:46 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.migrate_4qooh', metastore_id=None, name='migrate_4qooh', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 17:46 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_4qooh.ucx_tojef: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_4qooh/ucx_tojef 17:46 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_4qooh.ucx_tojef', metastore_id=None, name='ucx_tojef', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='migrate_4qooh', sql_path=None, storage_credential_name=None, storage_location='dbfs:/user/hive/warehouse/migrate_4qooh/ucx_tojef', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 17:46 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_4qooh.ucx_tdubs: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_4qooh/ucx_tdubs 17:46 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_4qooh.ucx_tdubs', metastore_id=None, name='ucx_tdubs', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='migrate_4qooh', sql_path=None, storage_credential_name=None, storage_location='dbfs:/tmp/ucx_test_CPYQ', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 17:46 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_4qooh.ucx_tmj98: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_4qooh/ucx_tmj98 17:46 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_4qooh.ucx_tmj98', metastore_id=None, name='ucx_tmj98', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='migrate_4qooh', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/b/kjL6', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 17:46 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_4qooh.ucx_tzpzx: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_4qooh/ucx_tzpzx 17:46 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_4qooh.ucx_tzpzx', metastore_id=None, name='ucx_tzpzx', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='migrate_4qooh', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_4qooh.ucx_tojef', view_dependencies=None) 17:47 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_4qooh.ucx_tar1x: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_4qooh/ucx_tar1x 17:47 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_4qooh.ucx_tar1x', metastore_id=None, name='ucx_tar1x', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='migrate_4qooh', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_4qooh.ucx_tzpzx', view_dependencies=None) 17:47 DEBUG [databricks.labs.ucx.mixins.fixtures] added catalog fixture: CatalogInfo(browse_only=False, catalog_type=, comment='', connection_name=None, created_at=1719942426195, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_chof1', isolation_mode=, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_chof1', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1719942426195, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d') 17:47 INFO [databricks.labs.ucx.mixins.fixtures] Schema ucx_chof1.migrate_4qooh: https://DATABRICKS_HOST/explore/data/ucx_chof1/migrate_4qooh 17:47 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_chof1', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_chof1.migrate_4qooh', metastore_id=None, name='migrate_4qooh', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 17:47 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_swlzx: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_swlzx 17:47 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_swlzx', metastore_id=None, name='ucx_swlzx', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 17:47 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.H6wL/config.yml) doesn't exist. 17:47 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration 17:47 INFO [databricks.labs.ucx.installer.hms_lineage] HMS Lineage feature creates one system table named system.hms_to_uc_migration.table_access and helps in your migration process from HMS to UC by allowing you to programmatically query HMS lineage data. 17:47 INFO [databricks.labs.ucx.install] Fetching installations... 17:47 WARNING [databricks.labs.ucx.install] Existing installation at /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.H6wL is corrupted. Skipping... 17:48 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy. 17:48 DEBUG [tests.integration.conftest] Waiting for clusters to start... 17:48 DEBUG [tests.integration.conftest] Waiting for clusters to start... 17:48 INFO [databricks.labs.ucx.install] Installing UCX v0.27.2+6120240702174859 17:48 INFO [databricks.labs.ucx.install] Creating ucx schemas... 17:50 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=experimental-workflow-linter 17:50 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-tables-ctas 17:50 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups 17:50 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions 17:50 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment 17:50 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-experimental 17:50 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-data-reconciliation 17:50 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups 17:50 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=failing 17:50 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-hiveserde-tables-in-place-experimental 17:50 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=scan-tables-in-mounts-experimental 17:50 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental 17:50 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables 17:50 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.H6wL/README for the next steps. 17:50 DEBUG [databricks.labs.ucx.installer.workflows] starting migrate-tables job: https://DATABRICKS_HOST#job/563698022716326 17:54 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 make_storage_dir fixtures 17:54 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 make_dbfs_data_copy fixtures 17:54 DEBUG [databricks.labs.ucx.mixins.fixtures] removing make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/kjL6 17:54 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 cluster fixtures 17:54 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 catalog fixtures 17:54 DEBUG [databricks.labs.ucx.mixins.fixtures] removing catalog fixture: CatalogInfo(browse_only=False, catalog_type=, comment='', connection_name=None, created_at=1719942426195, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_chof1', isolation_mode=, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_chof1', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1719942426195, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d') 17:54 INFO [databricks.labs.ucx.install] Deleting UCX v0.27.2+6120240702174859 from https://DATABRICKS_HOST 17:54 INFO [databricks.labs.ucx.install] Deleting inventory database ucx_swlzx 17:54 INFO [databricks.labs.ucx.install] Deleting jobs 17:54 INFO [databricks.labs.ucx.install] Deleting experimental-workflow-linter job_id=993004641761400. 17:54 INFO [databricks.labs.ucx.install] Deleting migrate-external-tables-ctas job_id=331106918034683. 17:54 INFO [databricks.labs.ucx.install] Deleting remove-workspace-local-backup-groups job_id=125753809552937. 17:54 INFO [databricks.labs.ucx.install] Deleting validate-groups-permissions job_id=618864975277697. 17:54 INFO [databricks.labs.ucx.install] Deleting assessment job_id=475118655299658. 17:54 INFO [databricks.labs.ucx.install] Deleting migrate-groups-experimental job_id=269541441549654. 17:54 INFO [databricks.labs.ucx.install] Deleting migrate-data-reconciliation job_id=596130125652212. 17:54 INFO [databricks.labs.ucx.install] Deleting migrate-groups job_id=136942201692237. 17:54 INFO [databricks.labs.ucx.install] Deleting failing job_id=711142816773498. 17:54 INFO [databricks.labs.ucx.install] Deleting migrate-external-hiveserde-tables-in-place-experimental job_id=658850384626595. 17:54 INFO [databricks.labs.ucx.install] Deleting scan-tables-in-mounts-experimental job_id=735308146076402. 17:54 INFO [databricks.labs.ucx.install] Deleting migrate-tables-in-mounts-experimental job_id=44588353258969. 17:54 INFO [databricks.labs.ucx.install] Deleting migrate-tables job_id=563698022716326. 17:54 INFO [databricks.labs.ucx.install] Deleting cluster policy 17:54 INFO [databricks.labs.ucx.install] Deleting secret scope 17:54 INFO [databricks.labs.ucx.install] UnInstalling UCX complete 17:54 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 workspace user fixtures 17:54 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 account group fixtures 17:54 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 workspace group fixtures 17:54 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 table fixtures 17:54 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 5 table fixtures 17:54 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_4qooh.ucx_tojef', metastore_id=None, name='ucx_tojef', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='migrate_4qooh', sql_path=None, storage_credential_name=None, storage_location='dbfs:/user/hive/warehouse/migrate_4qooh/ucx_tojef', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 17:54 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_4qooh.ucx_tdubs', metastore_id=None, name='ucx_tdubs', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='migrate_4qooh', sql_path=None, storage_credential_name=None, storage_location='dbfs:/tmp/ucx_test_CPYQ', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 17:54 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_4qooh.ucx_tmj98', metastore_id=None, name='ucx_tmj98', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='migrate_4qooh', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/b/kjL6', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 17:54 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_4qooh.ucx_tzpzx', metastore_id=None, name='ucx_tzpzx', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='migrate_4qooh', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_4qooh.ucx_tojef', view_dependencies=None) 17:54 DEBUG [databricks.labs.ucx.mixins.fixtures] ignoring error while table TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_4qooh.ucx_tzpzx', metastore_id=None, name='ucx_tzpzx', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='migrate_4qooh', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_4qooh.ucx_tojef', view_dependencies=None) teardown: [WRONG_COMMAND_FOR_OBJECT_TYPE] The operation DROP TABLE requires a EXTERNAL or MANAGED. But hive_metastore.migrate_4qooh.ucx_tzpzx is a VIEW. Use DROP VIEW instead. SQLSTATE: 42809 17:54 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_4qooh.ucx_tar1x', metastore_id=None, name='ucx_tar1x', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='migrate_4qooh', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_4qooh.ucx_tzpzx', view_dependencies=None) 17:54 DEBUG [databricks.labs.ucx.mixins.fixtures] ignoring error while table TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_4qooh.ucx_tar1x', metastore_id=None, name='ucx_tar1x', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='migrate_4qooh', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_4qooh.ucx_tzpzx', view_dependencies=None) teardown: [WRONG_COMMAND_FOR_OBJECT_TYPE] The operation DROP TABLE requires a EXTERNAL or MANAGED. But hive_metastore.migrate_4qooh.ucx_tar1x is a VIEW. Use DROP VIEW instead. SQLSTATE: 42809 17:54 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 3 schema fixtures 17:54 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.migrate_4qooh', metastore_id=None, name='migrate_4qooh', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 17:54 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_chof1', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_chof1.migrate_4qooh', metastore_id=None, name='migrate_4qooh', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 17:55 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_swlzx', metastore_id=None, name='ucx_swlzx', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) [gw0] linux -- Python 3.10.14 /home/runner/work/ucx/ucx/.venv/bin/python ```

❌ test_running_real_workflow_linter_job: TimeoutError: Timed out after 0:02:00 (5m48.631s)

``` TimeoutError: Timed out after 0:02:00 [gw1] linux -- Python 3.10.14 /home/runner/work/ucx/ucx/.venv/bin/python 17:49 DEBUG [databricks.labs.ucx.mixins.fixtures] added directory fixture: /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/sdk-IkaZ 17:49 DEBUG [databricks.labs.ucx.mixins.fixtures] added notebook fixture: /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/sdk-IkaZ/notebook.ipynb 17:49 INFO [databricks.labs.ucx.mixins.fixtures] Job: https://DATABRICKS_HOST#job/981311523238270 17:49 DEBUG [databricks.labs.ucx.mixins.fixtures] added job fixture: CreateResponse(job_id=981311523238270) 17:49 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_sihvr: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_sihvr 17:49 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_sihvr', metastore_id=None, name='ucx_sihvr', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 17:49 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.diSQ/config.yml) doesn't exist. 17:49 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration 17:49 INFO [databricks.labs.ucx.installer.hms_lineage] HMS Lineage feature creates one system table named system.hms_to_uc_migration.table_access and helps in your migration process from HMS to UC by allowing you to programmatically query HMS lineage data. 17:49 INFO [databricks.labs.ucx.install] Fetching installations... 17:50 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy. 17:50 DEBUG [tests.integration.conftest] Waiting for clusters to start... 17:50 DEBUG [tests.integration.conftest] Waiting for clusters to start... 17:50 INFO [databricks.labs.ucx.install] Installing UCX v0.27.2+6120240702175004 17:50 INFO [databricks.labs.ucx.install] Creating ucx schemas... 17:51 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment 17:51 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-hiveserde-tables-in-place-experimental 17:51 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=scan-tables-in-mounts-experimental 17:51 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables 17:51 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-experimental 17:51 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups 17:51 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=experimental-workflow-linter 17:51 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups 17:51 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=failing 17:51 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental 17:51 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-data-reconciliation 17:51 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions 17:51 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-tables-ctas 17:51 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.diSQ/README for the next steps. 17:51 DEBUG [databricks.labs.ucx.installer.workflows] starting experimental-workflow-linter job: https://DATABRICKS_HOST#job/225242546441549 17:55 DEBUG [databricks.labs.ucx.installer.workflows] Validating experimental-workflow-linter workflow: https://DATABRICKS_HOST#job/225242546441549 17:49 DEBUG [databricks.labs.ucx.mixins.fixtures] added directory fixture: /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/sdk-IkaZ 17:49 DEBUG [databricks.labs.ucx.mixins.fixtures] added notebook fixture: /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/sdk-IkaZ/notebook.ipynb 17:49 INFO [databricks.labs.ucx.mixins.fixtures] Job: https://DATABRICKS_HOST#job/981311523238270 17:49 DEBUG [databricks.labs.ucx.mixins.fixtures] added job fixture: CreateResponse(job_id=981311523238270) 17:49 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_sihvr: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_sihvr 17:49 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_sihvr', metastore_id=None, name='ucx_sihvr', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 17:49 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.diSQ/config.yml) doesn't exist. 17:49 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration 17:49 INFO [databricks.labs.ucx.installer.hms_lineage] HMS Lineage feature creates one system table named system.hms_to_uc_migration.table_access and helps in your migration process from HMS to UC by allowing you to programmatically query HMS lineage data. 17:49 INFO [databricks.labs.ucx.install] Fetching installations... 17:50 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy. 17:50 DEBUG [tests.integration.conftest] Waiting for clusters to start... 17:50 DEBUG [tests.integration.conftest] Waiting for clusters to start... 17:50 INFO [databricks.labs.ucx.install] Installing UCX v0.27.2+6120240702175004 17:50 INFO [databricks.labs.ucx.install] Creating ucx schemas... 17:51 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment 17:51 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-hiveserde-tables-in-place-experimental 17:51 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=scan-tables-in-mounts-experimental 17:51 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables 17:51 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-experimental 17:51 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups 17:51 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=experimental-workflow-linter 17:51 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups 17:51 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=failing 17:51 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental 17:51 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-data-reconciliation 17:51 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions 17:51 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-tables-ctas 17:51 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.diSQ/README for the next steps. 17:51 DEBUG [databricks.labs.ucx.installer.workflows] starting experimental-workflow-linter job: https://DATABRICKS_HOST#job/225242546441549 17:55 DEBUG [databricks.labs.ucx.installer.workflows] Validating experimental-workflow-linter workflow: https://DATABRICKS_HOST#job/225242546441549 17:55 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 job fixtures 17:55 DEBUG [databricks.labs.ucx.mixins.fixtures] removing job fixture: CreateResponse(job_id=981311523238270) 17:55 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 directory fixtures 17:55 DEBUG [databricks.labs.ucx.mixins.fixtures] removing directory fixture: /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/sdk-IkaZ 17:55 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 notebook fixtures 17:55 DEBUG [databricks.labs.ucx.mixins.fixtures] removing notebook fixture: /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/sdk-IkaZ/notebook.ipynb 17:55 DEBUG [databricks.labs.ucx.mixins.fixtures] ignoring error while notebook /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/sdk-IkaZ/notebook.ipynb teardown: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/sdk-IkaZ/notebook.ipynb) doesn't exist. 17:55 INFO [databricks.labs.ucx.install] Deleting UCX v0.27.2+6120240702175004 from https://DATABRICKS_HOST 17:55 INFO [databricks.labs.ucx.install] Deleting inventory database ucx_sihvr 17:55 INFO [databricks.labs.ucx.install] Deleting jobs 17:55 INFO [databricks.labs.ucx.install] Deleting assessment job_id=667382249458692. 17:55 INFO [databricks.labs.ucx.install] Deleting migrate-external-hiveserde-tables-in-place-experimental job_id=909705811892354. 17:55 INFO [databricks.labs.ucx.install] Deleting scan-tables-in-mounts-experimental job_id=443280318551455. 17:55 INFO [databricks.labs.ucx.install] Deleting migrate-tables job_id=1062937002098159. 17:55 INFO [databricks.labs.ucx.install] Deleting migrate-groups-experimental job_id=422204731317234. 17:55 INFO [databricks.labs.ucx.install] Deleting migrate-groups job_id=1071413279038395. 17:55 INFO [databricks.labs.ucx.install] Deleting experimental-workflow-linter job_id=225242546441549. 17:55 INFO [databricks.labs.ucx.install] Deleting remove-workspace-local-backup-groups job_id=364024185390971. 17:55 INFO [databricks.labs.ucx.install] Deleting failing job_id=58698398079429. 17:55 INFO [databricks.labs.ucx.install] Deleting migrate-tables-in-mounts-experimental job_id=798353346758742. 17:55 INFO [databricks.labs.ucx.install] Deleting migrate-data-reconciliation job_id=176608622931145. 17:55 INFO [databricks.labs.ucx.install] Deleting validate-groups-permissions job_id=64164308309134. 17:55 INFO [databricks.labs.ucx.install] Deleting migrate-external-tables-ctas job_id=287220773509290. 17:55 INFO [databricks.labs.ucx.install] Deleting cluster policy 17:55 INFO [databricks.labs.ucx.install] Deleting secret scope 17:55 INFO [databricks.labs.ucx.install] UnInstalling UCX complete 17:55 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 workspace user fixtures 17:55 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 account group fixtures 17:55 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 workspace group fixtures 17:55 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 table fixtures 17:55 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 table fixtures 17:55 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 schema fixtures 17:55 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_sihvr', metastore_id=None, name='ucx_sihvr', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) [gw1] linux -- Python 3.10.14 /home/runner/work/ucx/ucx/.venv/bin/python ```

❌ test_table_migration_job_refreshes_migration_status[hiveserde-migrate-external-hiveserde-tables-in-place-experimental]: TimeoutError: Timed out after 0:05:00 (8m29.645s)

``` TimeoutError: Timed out after 0:05:00 17:55 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/3fDY 17:55 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.hiveserde_in_place_o60tt: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_o60tt 17:55 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.hiveserde_in_place_o60tt', metastore_id=None, name='hiveserde_in_place_o60tt', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 17:55 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_storage_dir fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_o60tt 17:55 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_o60tt.parquet_serde_o60tt: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_o60tt/parquet_serde_o60tt 17:55 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_o60tt.parquet_serde_o60tt', metastore_id=None, name='parquet_serde_o60tt', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='hiveserde_in_place_o60tt', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_o60tt/parquet_serde_o60tt', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 17:55 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_o60tt.orc_serde_o60tt: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_o60tt/orc_serde_o60tt 17:55 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_o60tt.orc_serde_o60tt', metastore_id=None, name='orc_serde_o60tt', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='hiveserde_in_place_o60tt', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_o60tt/orc_serde_o60tt', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 17:55 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_o60tt.avro_serde_o60tt: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_o60tt/avro_serde_o60tt 17:55 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_o60tt.avro_serde_o60tt', metastore_id=None, name='avro_serde_o60tt', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='hiveserde_in_place_o60tt', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_o60tt/avro_serde_o60tt', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 17:55 DEBUG [databricks.labs.ucx.mixins.fixtures] added catalog fixture: CatalogInfo(browse_only=False, catalog_type=, comment='', connection_name=None, created_at=1719942929253, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cvh0r', isolation_mode=, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_cvh0r', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1719942929253, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d') 17:55 INFO [databricks.labs.ucx.mixins.fixtures] Schema ucx_cvh0r.hiveserde_in_place_o60tt: https://DATABRICKS_HOST/explore/data/ucx_cvh0r/hiveserde_in_place_o60tt 17:55 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_cvh0r', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cvh0r.hiveserde_in_place_o60tt', metastore_id=None, name='hiveserde_in_place_o60tt', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 17:56 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_s77gl: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_s77gl 17:56 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_s77gl', metastore_id=None, name='ucx_s77gl', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) [gw0] linux -- Python 3.10.14 /home/runner/work/ucx/ucx/.venv/bin/python 17:55 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/3fDY 17:55 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.hiveserde_in_place_o60tt: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_o60tt 17:55 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.hiveserde_in_place_o60tt', metastore_id=None, name='hiveserde_in_place_o60tt', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 17:55 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_storage_dir fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_o60tt 17:55 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_o60tt.parquet_serde_o60tt: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_o60tt/parquet_serde_o60tt 17:55 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_o60tt.parquet_serde_o60tt', metastore_id=None, name='parquet_serde_o60tt', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='hiveserde_in_place_o60tt', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_o60tt/parquet_serde_o60tt', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 17:55 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_o60tt.orc_serde_o60tt: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_o60tt/orc_serde_o60tt 17:55 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_o60tt.orc_serde_o60tt', metastore_id=None, name='orc_serde_o60tt', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='hiveserde_in_place_o60tt', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_o60tt/orc_serde_o60tt', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 17:55 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_o60tt.avro_serde_o60tt: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_o60tt/avro_serde_o60tt 17:55 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_o60tt.avro_serde_o60tt', metastore_id=None, name='avro_serde_o60tt', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='hiveserde_in_place_o60tt', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_o60tt/avro_serde_o60tt', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 17:55 DEBUG [databricks.labs.ucx.mixins.fixtures] added catalog fixture: CatalogInfo(browse_only=False, catalog_type=, comment='', connection_name=None, created_at=1719942929253, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cvh0r', isolation_mode=, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_cvh0r', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1719942929253, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d') 17:55 INFO [databricks.labs.ucx.mixins.fixtures] Schema ucx_cvh0r.hiveserde_in_place_o60tt: https://DATABRICKS_HOST/explore/data/ucx_cvh0r/hiveserde_in_place_o60tt 17:55 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_cvh0r', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cvh0r.hiveserde_in_place_o60tt', metastore_id=None, name='hiveserde_in_place_o60tt', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 17:56 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_s77gl: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_s77gl 17:56 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_s77gl', metastore_id=None, name='ucx_s77gl', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 17:57 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.GHIN/config.yml) doesn't exist. 17:57 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration 17:57 INFO [databricks.labs.ucx.installer.hms_lineage] HMS Lineage feature creates one system table named system.hms_to_uc_migration.table_access and helps in your migration process from HMS to UC by allowing you to programmatically query HMS lineage data. 17:57 INFO [databricks.labs.ucx.install] Fetching installations... 17:57 WARNING [databricks.labs.ucx.install] Existing installation at /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.GHIN is corrupted. Skipping... 17:57 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy. 17:57 DEBUG [tests.integration.conftest] Waiting for clusters to start... 17:57 DEBUG [tests.integration.conftest] Waiting for clusters to start... 17:57 INFO [databricks.labs.ucx.install] Installing UCX v0.27.2+6120240702175745 17:57 INFO [databricks.labs.ucx.install] Creating ucx schemas... 17:58 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=experimental-workflow-linter 17:58 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-tables-ctas 17:58 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups 17:58 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions 17:58 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment 17:58 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-experimental 17:58 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-data-reconciliation 17:58 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups 17:58 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=failing 17:58 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-hiveserde-tables-in-place-experimental 17:58 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=scan-tables-in-mounts-experimental 17:58 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental 17:58 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables 17:58 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.GHIN/README for the next steps. 17:58 DEBUG [databricks.labs.ucx.installer.workflows] starting migrate-external-hiveserde-tables-in-place-experimental job: https://DATABRICKS_HOST#job/999123672015016 17:55 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/3fDY 17:55 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.hiveserde_in_place_o60tt: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_o60tt 17:55 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.hiveserde_in_place_o60tt', metastore_id=None, name='hiveserde_in_place_o60tt', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 17:55 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_storage_dir fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_o60tt 17:55 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_o60tt.parquet_serde_o60tt: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_o60tt/parquet_serde_o60tt 17:55 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_o60tt.parquet_serde_o60tt', metastore_id=None, name='parquet_serde_o60tt', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='hiveserde_in_place_o60tt', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_o60tt/parquet_serde_o60tt', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 17:55 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_o60tt.orc_serde_o60tt: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_o60tt/orc_serde_o60tt 17:55 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_o60tt.orc_serde_o60tt', metastore_id=None, name='orc_serde_o60tt', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='hiveserde_in_place_o60tt', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_o60tt/orc_serde_o60tt', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 17:55 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_o60tt.avro_serde_o60tt: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_o60tt/avro_serde_o60tt 17:55 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_o60tt.avro_serde_o60tt', metastore_id=None, name='avro_serde_o60tt', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='hiveserde_in_place_o60tt', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_o60tt/avro_serde_o60tt', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 17:55 DEBUG [databricks.labs.ucx.mixins.fixtures] added catalog fixture: CatalogInfo(browse_only=False, catalog_type=, comment='', connection_name=None, created_at=1719942929253, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cvh0r', isolation_mode=, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_cvh0r', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1719942929253, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d') 17:55 INFO [databricks.labs.ucx.mixins.fixtures] Schema ucx_cvh0r.hiveserde_in_place_o60tt: https://DATABRICKS_HOST/explore/data/ucx_cvh0r/hiveserde_in_place_o60tt 17:55 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_cvh0r', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cvh0r.hiveserde_in_place_o60tt', metastore_id=None, name='hiveserde_in_place_o60tt', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 17:56 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_s77gl: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_s77gl 17:56 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_s77gl', metastore_id=None, name='ucx_s77gl', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 17:57 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.GHIN/config.yml) doesn't exist. 17:57 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration 17:57 INFO [databricks.labs.ucx.installer.hms_lineage] HMS Lineage feature creates one system table named system.hms_to_uc_migration.table_access and helps in your migration process from HMS to UC by allowing you to programmatically query HMS lineage data. 17:57 INFO [databricks.labs.ucx.install] Fetching installations... 17:57 WARNING [databricks.labs.ucx.install] Existing installation at /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.GHIN is corrupted. Skipping... 17:57 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy. 17:57 DEBUG [tests.integration.conftest] Waiting for clusters to start... 17:57 DEBUG [tests.integration.conftest] Waiting for clusters to start... 17:57 INFO [databricks.labs.ucx.install] Installing UCX v0.27.2+6120240702175745 17:57 INFO [databricks.labs.ucx.install] Creating ucx schemas... 17:58 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=experimental-workflow-linter 17:58 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-tables-ctas 17:58 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups 17:58 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions 17:58 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment 17:58 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-experimental 17:58 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-data-reconciliation 17:58 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups 17:58 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=failing 17:58 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-hiveserde-tables-in-place-experimental 17:58 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=scan-tables-in-mounts-experimental 17:58 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental 17:58 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables 17:58 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.GHIN/README for the next steps. 17:58 DEBUG [databricks.labs.ucx.installer.workflows] starting migrate-external-hiveserde-tables-in-place-experimental job: https://DATABRICKS_HOST#job/999123672015016 18:02 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 make_storage_dir fixtures 18:02 DEBUG [databricks.labs.ucx.mixins.fixtures] removing make_storage_dir fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_o60tt 18:02 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 make_dbfs_data_copy fixtures 18:02 DEBUG [databricks.labs.ucx.mixins.fixtures] removing make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/3fDY 18:02 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 cluster fixtures 18:02 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 catalog fixtures 18:02 DEBUG [databricks.labs.ucx.mixins.fixtures] removing catalog fixture: CatalogInfo(browse_only=False, catalog_type=, comment='', connection_name=None, created_at=1719942929253, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cvh0r', isolation_mode=, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_cvh0r', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1719942929253, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d') 18:03 INFO [databricks.labs.ucx.install] Deleting UCX v0.27.2+6120240702175745 from https://DATABRICKS_HOST 18:03 INFO [databricks.labs.ucx.install] Deleting inventory database ucx_s77gl 18:03 INFO [databricks.labs.ucx.install] Deleting jobs 18:03 INFO [databricks.labs.ucx.install] Deleting experimental-workflow-linter job_id=91283665151496. 18:03 INFO [databricks.labs.ucx.install] Deleting migrate-external-tables-ctas job_id=719163415699692. 18:03 INFO [databricks.labs.ucx.install] Deleting remove-workspace-local-backup-groups job_id=422067017481168. 18:03 INFO [databricks.labs.ucx.install] Deleting validate-groups-permissions job_id=877010779308139. 18:03 INFO [databricks.labs.ucx.install] Deleting assessment job_id=245583768335527. 18:03 INFO [databricks.labs.ucx.install] Deleting migrate-groups-experimental job_id=1010889187019391. 18:03 INFO [databricks.labs.ucx.install] Deleting migrate-data-reconciliation job_id=265645415546926. 18:03 INFO [databricks.labs.ucx.install] Deleting migrate-groups job_id=147985197513675. 18:03 INFO [databricks.labs.ucx.install] Deleting failing job_id=526125005439175. 18:03 INFO [databricks.labs.ucx.install] Deleting migrate-external-hiveserde-tables-in-place-experimental job_id=999123672015016. 18:03 INFO [databricks.labs.ucx.install] Deleting scan-tables-in-mounts-experimental job_id=83821851529313. 18:03 INFO [databricks.labs.ucx.install] Deleting migrate-tables-in-mounts-experimental job_id=1039341828784563. 18:03 INFO [databricks.labs.ucx.install] Deleting migrate-tables job_id=155269688704050. 18:03 INFO [databricks.labs.ucx.install] Deleting cluster policy 18:03 INFO [databricks.labs.ucx.install] Deleting secret scope 18:03 INFO [databricks.labs.ucx.install] UnInstalling UCX complete 18:03 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 workspace user fixtures 18:03 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 account group fixtures 18:03 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 workspace group fixtures 18:03 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 table fixtures 18:03 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 3 table fixtures 18:03 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_o60tt.parquet_serde_o60tt', metastore_id=None, name='parquet_serde_o60tt', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='hiveserde_in_place_o60tt', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_o60tt/parquet_serde_o60tt', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 18:03 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_o60tt.orc_serde_o60tt', metastore_id=None, name='orc_serde_o60tt', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='hiveserde_in_place_o60tt', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_o60tt/orc_serde_o60tt', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 18:03 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_o60tt.avro_serde_o60tt', metastore_id=None, name='avro_serde_o60tt', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='hiveserde_in_place_o60tt', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_o60tt/avro_serde_o60tt', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 18:03 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 3 schema fixtures 18:03 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.hiveserde_in_place_o60tt', metastore_id=None, name='hiveserde_in_place_o60tt', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 18:03 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_cvh0r', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cvh0r.hiveserde_in_place_o60tt', metastore_id=None, name='hiveserde_in_place_o60tt', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 18:03 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_s77gl', metastore_id=None, name='ucx_s77gl', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) [gw0] linux -- Python 3.10.14 /home/runner/work/ucx/ucx/.venv/bin/python ```

❌ test_table_migration_job_refreshes_migration_status[hiveserde-migrate-external-tables-ctas]: TimeoutError: timed out after 0:20:00: (23m15.895s)

``` TimeoutError: timed out after 0:20:00: 17:43 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/SUVc 17:43 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.hiveserde_in_place_44gge: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_44gge 17:43 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.hiveserde_in_place_44gge', metastore_id=None, name='hiveserde_in_place_44gge', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 17:43 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_storage_dir fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_44gge 17:43 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_44gge.parquet_serde_44gge: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_44gge/parquet_serde_44gge 17:43 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_44gge.parquet_serde_44gge', metastore_id=None, name='parquet_serde_44gge', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070218'}, row_filter=None, schema_name='hiveserde_in_place_44gge', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_44gge/parquet_serde_44gge', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 17:43 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_44gge.orc_serde_44gge: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_44gge/orc_serde_44gge 17:43 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_44gge.orc_serde_44gge', metastore_id=None, name='orc_serde_44gge', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070218'}, row_filter=None, schema_name='hiveserde_in_place_44gge', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_44gge/orc_serde_44gge', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 17:43 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_44gge.avro_serde_44gge: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_44gge/avro_serde_44gge 17:43 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_44gge.avro_serde_44gge', metastore_id=None, name='avro_serde_44gge', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070218'}, row_filter=None, schema_name='hiveserde_in_place_44gge', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_44gge/avro_serde_44gge', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 17:43 DEBUG [databricks.labs.ucx.mixins.fixtures] added catalog fixture: CatalogInfo(browse_only=False, catalog_type=, comment='', connection_name=None, created_at=1719942222659, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cmofq', isolation_mode=, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_cmofq', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1719942222659, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d') 17:43 INFO [databricks.labs.ucx.mixins.fixtures] Schema ucx_cmofq.hiveserde_in_place_44gge: https://DATABRICKS_HOST/explore/data/ucx_cmofq/hiveserde_in_place_44gge 17:43 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_cmofq', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cmofq.hiveserde_in_place_44gge', metastore_id=None, name='hiveserde_in_place_44gge', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 17:43 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_smwbt: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_smwbt 17:43 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_smwbt', metastore_id=None, name='ucx_smwbt', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) [gw2] linux -- Python 3.10.14 /home/runner/work/ucx/ucx/.venv/bin/python 17:43 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/SUVc 17:43 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.hiveserde_in_place_44gge: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_44gge 17:43 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.hiveserde_in_place_44gge', metastore_id=None, name='hiveserde_in_place_44gge', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 17:43 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_storage_dir fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_44gge 17:43 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_44gge.parquet_serde_44gge: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_44gge/parquet_serde_44gge 17:43 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_44gge.parquet_serde_44gge', metastore_id=None, name='parquet_serde_44gge', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070218'}, row_filter=None, schema_name='hiveserde_in_place_44gge', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_44gge/parquet_serde_44gge', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 17:43 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_44gge.orc_serde_44gge: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_44gge/orc_serde_44gge 17:43 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_44gge.orc_serde_44gge', metastore_id=None, name='orc_serde_44gge', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070218'}, row_filter=None, schema_name='hiveserde_in_place_44gge', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_44gge/orc_serde_44gge', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 17:43 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_44gge.avro_serde_44gge: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_44gge/avro_serde_44gge 17:43 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_44gge.avro_serde_44gge', metastore_id=None, name='avro_serde_44gge', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070218'}, row_filter=None, schema_name='hiveserde_in_place_44gge', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_44gge/avro_serde_44gge', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 17:43 DEBUG [databricks.labs.ucx.mixins.fixtures] added catalog fixture: CatalogInfo(browse_only=False, catalog_type=, comment='', connection_name=None, created_at=1719942222659, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cmofq', isolation_mode=, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_cmofq', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1719942222659, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d') 17:43 INFO [databricks.labs.ucx.mixins.fixtures] Schema ucx_cmofq.hiveserde_in_place_44gge: https://DATABRICKS_HOST/explore/data/ucx_cmofq/hiveserde_in_place_44gge 17:43 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_cmofq', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cmofq.hiveserde_in_place_44gge', metastore_id=None, name='hiveserde_in_place_44gge', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 17:43 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_smwbt: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_smwbt 17:43 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_smwbt', metastore_id=None, name='ucx_smwbt', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 17:44 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.XRl2/config.yml) doesn't exist. 17:44 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration 17:44 INFO [databricks.labs.ucx.installer.hms_lineage] HMS Lineage feature creates one system table named system.hms_to_uc_migration.table_access and helps in your migration process from HMS to UC by allowing you to programmatically query HMS lineage data. 17:44 INFO [databricks.labs.ucx.install] Fetching installations... 17:44 WARNING [databricks.labs.ucx.install] Existing installation at /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.XRl2 is corrupted. Skipping... 17:44 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy. 17:44 DEBUG [tests.integration.conftest] Waiting for clusters to start... 17:44 DEBUG [tests.integration.conftest] Waiting for clusters to start... 17:44 INFO [databricks.labs.ucx.install] Installing UCX v0.27.2+6120240702174446 17:44 INFO [databricks.labs.ucx.install] Creating ucx schemas... 17:45 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables 17:45 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-experimental 17:45 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-hiveserde-tables-in-place-experimental 17:45 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental 17:45 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=experimental-workflow-linter 17:45 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=failing 17:45 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-tables-ctas 17:45 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment 17:45 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups 17:45 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=scan-tables-in-mounts-experimental 17:45 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions 17:45 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups 17:45 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-data-reconciliation 17:45 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.XRl2/README for the next steps. 17:45 DEBUG [databricks.labs.ucx.installer.workflows] starting migrate-external-tables-ctas job: https://DATABRICKS_HOST#job/982353922477807 17:43 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/SUVc 17:43 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.hiveserde_in_place_44gge: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_44gge 17:43 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.hiveserde_in_place_44gge', metastore_id=None, name='hiveserde_in_place_44gge', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 17:43 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_storage_dir fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_44gge 17:43 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_44gge.parquet_serde_44gge: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_44gge/parquet_serde_44gge 17:43 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_44gge.parquet_serde_44gge', metastore_id=None, name='parquet_serde_44gge', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070218'}, row_filter=None, schema_name='hiveserde_in_place_44gge', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_44gge/parquet_serde_44gge', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 17:43 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_44gge.orc_serde_44gge: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_44gge/orc_serde_44gge 17:43 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_44gge.orc_serde_44gge', metastore_id=None, name='orc_serde_44gge', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070218'}, row_filter=None, schema_name='hiveserde_in_place_44gge', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_44gge/orc_serde_44gge', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 17:43 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_44gge.avro_serde_44gge: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_44gge/avro_serde_44gge 17:43 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_44gge.avro_serde_44gge', metastore_id=None, name='avro_serde_44gge', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070218'}, row_filter=None, schema_name='hiveserde_in_place_44gge', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_44gge/avro_serde_44gge', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 17:43 DEBUG [databricks.labs.ucx.mixins.fixtures] added catalog fixture: CatalogInfo(browse_only=False, catalog_type=, comment='', connection_name=None, created_at=1719942222659, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cmofq', isolation_mode=, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_cmofq', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1719942222659, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d') 17:43 INFO [databricks.labs.ucx.mixins.fixtures] Schema ucx_cmofq.hiveserde_in_place_44gge: https://DATABRICKS_HOST/explore/data/ucx_cmofq/hiveserde_in_place_44gge 17:43 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_cmofq', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cmofq.hiveserde_in_place_44gge', metastore_id=None, name='hiveserde_in_place_44gge', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 17:43 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_smwbt: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_smwbt 17:43 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_smwbt', metastore_id=None, name='ucx_smwbt', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 17:44 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.XRl2/config.yml) doesn't exist. 17:44 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration 17:44 INFO [databricks.labs.ucx.installer.hms_lineage] HMS Lineage feature creates one system table named system.hms_to_uc_migration.table_access and helps in your migration process from HMS to UC by allowing you to programmatically query HMS lineage data. 17:44 INFO [databricks.labs.ucx.install] Fetching installations... 17:44 WARNING [databricks.labs.ucx.install] Existing installation at /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.XRl2 is corrupted. Skipping... 17:44 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy. 17:44 DEBUG [tests.integration.conftest] Waiting for clusters to start... 17:44 DEBUG [tests.integration.conftest] Waiting for clusters to start... 17:44 INFO [databricks.labs.ucx.install] Installing UCX v0.27.2+6120240702174446 17:44 INFO [databricks.labs.ucx.install] Creating ucx schemas... 17:45 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables 17:45 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-experimental 17:45 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-hiveserde-tables-in-place-experimental 17:45 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental 17:45 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=experimental-workflow-linter 17:45 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=failing 17:45 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-tables-ctas 17:45 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment 17:45 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups 17:45 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=scan-tables-in-mounts-experimental 17:45 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions 17:45 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups 17:45 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-data-reconciliation 17:45 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.XRl2/README for the next steps. 17:45 DEBUG [databricks.labs.ucx.installer.workflows] starting migrate-external-tables-ctas job: https://DATABRICKS_HOST#job/982353922477807 18:05 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 make_storage_dir fixtures 18:05 DEBUG [databricks.labs.ucx.mixins.fixtures] removing make_storage_dir fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_44gge 18:05 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 make_dbfs_data_copy fixtures 18:05 DEBUG [databricks.labs.ucx.mixins.fixtures] removing make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/SUVc 18:05 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 cluster fixtures 18:05 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 catalog fixtures 18:05 DEBUG [databricks.labs.ucx.mixins.fixtures] removing catalog fixture: CatalogInfo(browse_only=False, catalog_type=, comment='', connection_name=None, created_at=1719942222659, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cmofq', isolation_mode=, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_cmofq', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1719942222659, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d') 18:05 INFO [databricks.labs.ucx.install] Deleting UCX v0.27.2+6120240702174446 from https://DATABRICKS_HOST 18:05 INFO [databricks.labs.ucx.install] Deleting inventory database ucx_smwbt 18:05 INFO [databricks.labs.ucx.install] Deleting jobs 18:05 INFO [databricks.labs.ucx.install] Deleting migrate-tables job_id=411252755125119. 18:05 INFO [databricks.labs.ucx.install] Deleting migrate-groups-experimental job_id=869401243866394. 18:05 INFO [databricks.labs.ucx.install] Deleting migrate-external-hiveserde-tables-in-place-experimental job_id=288486754997024. 18:05 INFO [databricks.labs.ucx.install] Deleting migrate-tables-in-mounts-experimental job_id=1116807171374151. 18:05 INFO [databricks.labs.ucx.install] Deleting experimental-workflow-linter job_id=494359700466354. 18:05 INFO [databricks.labs.ucx.install] Deleting failing job_id=421267979038770. 18:05 INFO [databricks.labs.ucx.install] Deleting migrate-external-tables-ctas job_id=982353922477807. 18:05 INFO [databricks.labs.ucx.install] Deleting assessment job_id=116568637100872. 18:05 INFO [databricks.labs.ucx.install] Deleting remove-workspace-local-backup-groups job_id=997186938753506. 18:05 INFO [databricks.labs.ucx.install] Deleting scan-tables-in-mounts-experimental job_id=34290424489224. 18:05 INFO [databricks.labs.ucx.install] Deleting validate-groups-permissions job_id=432427653695645. 18:05 INFO [databricks.labs.ucx.install] Deleting migrate-groups job_id=1042635144037010. 18:05 INFO [databricks.labs.ucx.install] Deleting migrate-data-reconciliation job_id=456104485254763. 18:05 INFO [databricks.labs.ucx.install] Deleting cluster policy 18:05 INFO [databricks.labs.ucx.install] Deleting secret scope 18:05 INFO [databricks.labs.ucx.install] UnInstalling UCX complete 18:05 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 workspace user fixtures 18:05 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 account group fixtures 18:05 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 workspace group fixtures 18:05 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 table fixtures 18:05 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 3 table fixtures 18:05 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_44gge.parquet_serde_44gge', metastore_id=None, name='parquet_serde_44gge', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070218'}, row_filter=None, schema_name='hiveserde_in_place_44gge', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_44gge/parquet_serde_44gge', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 18:06 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_44gge.orc_serde_44gge', metastore_id=None, name='orc_serde_44gge', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070218'}, row_filter=None, schema_name='hiveserde_in_place_44gge', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_44gge/orc_serde_44gge', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 18:06 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_44gge.avro_serde_44gge', metastore_id=None, name='avro_serde_44gge', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070218'}, row_filter=None, schema_name='hiveserde_in_place_44gge', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_44gge/avro_serde_44gge', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 18:06 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 3 schema fixtures 18:06 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.hiveserde_in_place_44gge', metastore_id=None, name='hiveserde_in_place_44gge', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 18:06 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_cmofq', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cmofq.hiveserde_in_place_44gge', metastore_id=None, name='hiveserde_in_place_44gge', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 18:06 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_smwbt', metastore_id=None, name='ucx_smwbt', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) [gw2] linux -- Python 3.10.14 /home/runner/work/ucx/ucx/.venv/bin/python ```

❌ test_table_migration_job_cluster_override[regular]: AssertionError: assert False (7m52.431s)

``` AssertionError: assert False + where False = >('migrate-tables') + where > = .validate_step + where = .deployed_workflows 18:16 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/TVeo 18:16 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.migrate_o0uls: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_o0uls 18:16 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.migrate_o0uls', metastore_id=None, name='migrate_o0uls', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 18:16 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_o0uls.ucx_t4fke: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_o0uls/ucx_t4fke 18:16 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_o0uls.ucx_t4fke', metastore_id=None, name='ucx_t4fke', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='migrate_o0uls', sql_path=None, storage_credential_name=None, storage_location='dbfs:/user/hive/warehouse/migrate_o0uls/ucx_t4fke', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 18:16 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_o0uls.ucx_t41u7: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_o0uls/ucx_t41u7 18:16 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_o0uls.ucx_t41u7', metastore_id=None, name='ucx_t41u7', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='migrate_o0uls', sql_path=None, storage_credential_name=None, storage_location='dbfs:/tmp/ucx_test_9stL', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 18:16 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_o0uls.ucx_thaxp: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_o0uls/ucx_thaxp 18:16 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_o0uls.ucx_thaxp', metastore_id=None, name='ucx_thaxp', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='migrate_o0uls', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/b/TVeo', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 18:17 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_o0uls.ucx_tiirq: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_o0uls/ucx_tiirq 18:17 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_o0uls.ucx_tiirq', metastore_id=None, name='ucx_tiirq', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='migrate_o0uls', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_o0uls.ucx_t4fke', view_dependencies=None) 18:17 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_o0uls.ucx_ta78y: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_o0uls/ucx_ta78y 18:17 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_o0uls.ucx_ta78y', metastore_id=None, name='ucx_ta78y', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='migrate_o0uls', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_o0uls.ucx_tiirq', view_dependencies=None) 18:17 DEBUG [databricks.labs.ucx.mixins.fixtures] added catalog fixture: CatalogInfo(browse_only=False, catalog_type=, comment='', connection_name=None, created_at=1719944230423, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_c45k9', isolation_mode=, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_c45k9', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1719944230423, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d') 18:17 INFO [databricks.labs.ucx.mixins.fixtures] Schema ucx_c45k9.migrate_o0uls: https://DATABRICKS_HOST/explore/data/ucx_c45k9/migrate_o0uls 18:17 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_c45k9', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_c45k9.migrate_o0uls', metastore_id=None, name='migrate_o0uls', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 18:17 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_stynp: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_stynp 18:17 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_stynp', metastore_id=None, name='ucx_stynp', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) [gw7] linux -- Python 3.10.14 /home/runner/work/ucx/ucx/.venv/bin/python 18:16 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/TVeo 18:16 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.migrate_o0uls: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_o0uls 18:16 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.migrate_o0uls', metastore_id=None, name='migrate_o0uls', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 18:16 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_o0uls.ucx_t4fke: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_o0uls/ucx_t4fke 18:16 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_o0uls.ucx_t4fke', metastore_id=None, name='ucx_t4fke', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='migrate_o0uls', sql_path=None, storage_credential_name=None, storage_location='dbfs:/user/hive/warehouse/migrate_o0uls/ucx_t4fke', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 18:16 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_o0uls.ucx_t41u7: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_o0uls/ucx_t41u7 18:16 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_o0uls.ucx_t41u7', metastore_id=None, name='ucx_t41u7', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='migrate_o0uls', sql_path=None, storage_credential_name=None, storage_location='dbfs:/tmp/ucx_test_9stL', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 18:16 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_o0uls.ucx_thaxp: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_o0uls/ucx_thaxp 18:16 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_o0uls.ucx_thaxp', metastore_id=None, name='ucx_thaxp', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='migrate_o0uls', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/b/TVeo', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 18:17 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_o0uls.ucx_tiirq: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_o0uls/ucx_tiirq 18:17 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_o0uls.ucx_tiirq', metastore_id=None, name='ucx_tiirq', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='migrate_o0uls', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_o0uls.ucx_t4fke', view_dependencies=None) 18:17 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_o0uls.ucx_ta78y: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_o0uls/ucx_ta78y 18:17 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_o0uls.ucx_ta78y', metastore_id=None, name='ucx_ta78y', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='migrate_o0uls', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_o0uls.ucx_tiirq', view_dependencies=None) 18:17 DEBUG [databricks.labs.ucx.mixins.fixtures] added catalog fixture: CatalogInfo(browse_only=False, catalog_type=, comment='', connection_name=None, created_at=1719944230423, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_c45k9', isolation_mode=, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_c45k9', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1719944230423, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d') 18:17 INFO [databricks.labs.ucx.mixins.fixtures] Schema ucx_c45k9.migrate_o0uls: https://DATABRICKS_HOST/explore/data/ucx_c45k9/migrate_o0uls 18:17 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_c45k9', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_c45k9.migrate_o0uls', metastore_id=None, name='migrate_o0uls', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 18:17 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_stynp: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_stynp 18:17 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_stynp', metastore_id=None, name='ucx_stynp', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 18:17 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.rieb/config.yml) doesn't exist. 18:17 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration 18:18 INFO [databricks.labs.ucx.installer.hms_lineage] HMS Lineage feature creates one system table named system.hms_to_uc_migration.table_access and helps in your migration process from HMS to UC by allowing you to programmatically query HMS lineage data. 18:18 INFO [databricks.labs.ucx.install] Fetching installations... 18:18 WARNING [databricks.labs.ucx.install] Existing installation at /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.rieb is corrupted. Skipping... 18:18 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy. 18:18 DEBUG [tests.integration.conftest] Waiting for clusters to start... 18:18 DEBUG [tests.integration.conftest] Waiting for clusters to start... 18:18 INFO [databricks.labs.ucx.install] Installing UCX v0.27.2+6120240702181858 18:18 INFO [databricks.labs.ucx.install] Creating ucx schemas... 18:19 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=scan-tables-in-mounts-experimental 18:19 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions 18:19 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-hiveserde-tables-in-place-experimental 18:19 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental 18:19 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=experimental-workflow-linter 18:19 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables 18:19 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-tables-ctas 18:19 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-experimental 18:19 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups 18:19 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment 18:19 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-data-reconciliation 18:19 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=failing 18:19 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups 18:19 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.rieb/README for the next steps. 18:19 DEBUG [databricks.labs.ucx.installer.workflows] starting migrate-tables job: https://DATABRICKS_HOST#job/8439732705999 18:24 DEBUG [databricks.labs.ucx.installer.workflows] Validating migrate-tables workflow: https://DATABRICKS_HOST#job/8439732705999 18:16 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/TVeo 18:16 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.migrate_o0uls: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_o0uls 18:16 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.migrate_o0uls', metastore_id=None, name='migrate_o0uls', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 18:16 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_o0uls.ucx_t4fke: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_o0uls/ucx_t4fke 18:16 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_o0uls.ucx_t4fke', metastore_id=None, name='ucx_t4fke', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='migrate_o0uls', sql_path=None, storage_credential_name=None, storage_location='dbfs:/user/hive/warehouse/migrate_o0uls/ucx_t4fke', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 18:16 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_o0uls.ucx_t41u7: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_o0uls/ucx_t41u7 18:16 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_o0uls.ucx_t41u7', metastore_id=None, name='ucx_t41u7', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='migrate_o0uls', sql_path=None, storage_credential_name=None, storage_location='dbfs:/tmp/ucx_test_9stL', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 18:16 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_o0uls.ucx_thaxp: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_o0uls/ucx_thaxp 18:16 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_o0uls.ucx_thaxp', metastore_id=None, name='ucx_thaxp', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='migrate_o0uls', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/b/TVeo', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 18:17 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_o0uls.ucx_tiirq: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_o0uls/ucx_tiirq 18:17 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_o0uls.ucx_tiirq', metastore_id=None, name='ucx_tiirq', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='migrate_o0uls', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_o0uls.ucx_t4fke', view_dependencies=None) 18:17 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_o0uls.ucx_ta78y: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_o0uls/ucx_ta78y 18:17 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_o0uls.ucx_ta78y', metastore_id=None, name='ucx_ta78y', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='migrate_o0uls', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_o0uls.ucx_tiirq', view_dependencies=None) 18:17 DEBUG [databricks.labs.ucx.mixins.fixtures] added catalog fixture: CatalogInfo(browse_only=False, catalog_type=, comment='', connection_name=None, created_at=1719944230423, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_c45k9', isolation_mode=, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_c45k9', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1719944230423, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d') 18:17 INFO [databricks.labs.ucx.mixins.fixtures] Schema ucx_c45k9.migrate_o0uls: https://DATABRICKS_HOST/explore/data/ucx_c45k9/migrate_o0uls 18:17 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_c45k9', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_c45k9.migrate_o0uls', metastore_id=None, name='migrate_o0uls', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 18:17 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_stynp: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_stynp 18:17 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_stynp', metastore_id=None, name='ucx_stynp', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 18:17 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.rieb/config.yml) doesn't exist. 18:17 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration 18:18 INFO [databricks.labs.ucx.installer.hms_lineage] HMS Lineage feature creates one system table named system.hms_to_uc_migration.table_access and helps in your migration process from HMS to UC by allowing you to programmatically query HMS lineage data. 18:18 INFO [databricks.labs.ucx.install] Fetching installations... 18:18 WARNING [databricks.labs.ucx.install] Existing installation at /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.rieb is corrupted. Skipping... 18:18 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy. 18:18 DEBUG [tests.integration.conftest] Waiting for clusters to start... 18:18 DEBUG [tests.integration.conftest] Waiting for clusters to start... 18:18 INFO [databricks.labs.ucx.install] Installing UCX v0.27.2+6120240702181858 18:18 INFO [databricks.labs.ucx.install] Creating ucx schemas... 18:19 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=scan-tables-in-mounts-experimental 18:19 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions 18:19 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-hiveserde-tables-in-place-experimental 18:19 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental 18:19 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=experimental-workflow-linter 18:19 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables 18:19 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-tables-ctas 18:19 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-experimental 18:19 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups 18:19 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment 18:19 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-data-reconciliation 18:19 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=failing 18:19 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups 18:19 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.rieb/README for the next steps. 18:19 DEBUG [databricks.labs.ucx.installer.workflows] starting migrate-tables job: https://DATABRICKS_HOST#job/8439732705999 18:24 DEBUG [databricks.labs.ucx.installer.workflows] Validating migrate-tables workflow: https://DATABRICKS_HOST#job/8439732705999 18:24 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 make_storage_dir fixtures 18:24 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 make_dbfs_data_copy fixtures 18:24 DEBUG [databricks.labs.ucx.mixins.fixtures] removing make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/TVeo 18:24 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 cluster fixtures 18:24 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 catalog fixtures 18:24 DEBUG [databricks.labs.ucx.mixins.fixtures] removing catalog fixture: CatalogInfo(browse_only=False, catalog_type=, comment='', connection_name=None, created_at=1719944230423, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_c45k9', isolation_mode=, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_c45k9', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1719944230423, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d') 18:24 INFO [databricks.labs.ucx.install] Deleting UCX v0.27.2+6120240702181858 from https://DATABRICKS_HOST 18:24 INFO [databricks.labs.ucx.install] Deleting inventory database ucx_stynp 18:24 INFO [databricks.labs.ucx.install] Deleting jobs 18:24 INFO [databricks.labs.ucx.install] Deleting scan-tables-in-mounts-experimental job_id=354625562519659. 18:24 INFO [databricks.labs.ucx.install] Deleting validate-groups-permissions job_id=480990776972961. 18:24 INFO [databricks.labs.ucx.install] Deleting migrate-external-hiveserde-tables-in-place-experimental job_id=512749568308837. 18:24 INFO [databricks.labs.ucx.install] Deleting migrate-tables-in-mounts-experimental job_id=138451192062401. 18:24 INFO [databricks.labs.ucx.install] Deleting experimental-workflow-linter job_id=109895326413557. 18:24 INFO [databricks.labs.ucx.install] Deleting migrate-tables job_id=8439732705999. 18:24 INFO [databricks.labs.ucx.install] Deleting migrate-external-tables-ctas job_id=1023777000423032. 18:24 INFO [databricks.labs.ucx.install] Deleting migrate-groups-experimental job_id=72020903116075. 18:24 INFO [databricks.labs.ucx.install] Deleting remove-workspace-local-backup-groups job_id=429908059645622. 18:24 INFO [databricks.labs.ucx.install] Deleting assessment job_id=170809141889086. 18:24 INFO [databricks.labs.ucx.install] Deleting migrate-data-reconciliation job_id=934895421292712. 18:24 INFO [databricks.labs.ucx.install] Deleting failing job_id=101154916887925. 18:24 INFO [databricks.labs.ucx.install] Deleting migrate-groups job_id=1036233271416156. 18:24 INFO [databricks.labs.ucx.install] Deleting cluster policy 18:24 INFO [databricks.labs.ucx.install] Deleting secret scope 18:24 INFO [databricks.labs.ucx.install] UnInstalling UCX complete 18:24 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 workspace user fixtures 18:24 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 account group fixtures 18:24 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 workspace group fixtures 18:24 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 table fixtures 18:24 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 5 table fixtures 18:24 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_o0uls.ucx_t4fke', metastore_id=None, name='ucx_t4fke', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='migrate_o0uls', sql_path=None, storage_credential_name=None, storage_location='dbfs:/user/hive/warehouse/migrate_o0uls/ucx_t4fke', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 18:24 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_o0uls.ucx_t41u7', metastore_id=None, name='ucx_t41u7', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='migrate_o0uls', sql_path=None, storage_credential_name=None, storage_location='dbfs:/tmp/ucx_test_9stL', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 18:24 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_o0uls.ucx_thaxp', metastore_id=None, name='ucx_thaxp', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='migrate_o0uls', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/b/TVeo', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 18:24 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_o0uls.ucx_tiirq', metastore_id=None, name='ucx_tiirq', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='migrate_o0uls', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_o0uls.ucx_t4fke', view_dependencies=None) 18:24 DEBUG [databricks.labs.ucx.mixins.fixtures] ignoring error while table TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_o0uls.ucx_tiirq', metastore_id=None, name='ucx_tiirq', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='migrate_o0uls', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_o0uls.ucx_t4fke', view_dependencies=None) teardown: [WRONG_COMMAND_FOR_OBJECT_TYPE] The operation DROP TABLE requires a EXTERNAL or MANAGED. But hive_metastore.migrate_o0uls.ucx_tiirq is a VIEW. Use DROP VIEW instead. SQLSTATE: 42809 18:24 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_o0uls.ucx_ta78y', metastore_id=None, name='ucx_ta78y', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='migrate_o0uls', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_o0uls.ucx_tiirq', view_dependencies=None) 18:24 DEBUG [databricks.labs.ucx.mixins.fixtures] ignoring error while table TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_o0uls.ucx_ta78y', metastore_id=None, name='ucx_ta78y', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='migrate_o0uls', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_o0uls.ucx_tiirq', view_dependencies=None) teardown: [WRONG_COMMAND_FOR_OBJECT_TYPE] The operation DROP TABLE requires a EXTERNAL or MANAGED. But hive_metastore.migrate_o0uls.ucx_ta78y is a VIEW. Use DROP VIEW instead. SQLSTATE: 42809 18:24 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 3 schema fixtures 18:24 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.migrate_o0uls', metastore_id=None, name='migrate_o0uls', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 18:24 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_c45k9', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_c45k9.migrate_o0uls', metastore_id=None, name='migrate_o0uls', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 18:24 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_stynp', metastore_id=None, name='ucx_stynp', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) [gw7] linux -- Python 3.10.14 /home/runner/work/ucx/ucx/.venv/bin/python ```

❌ test_hiveserde_table_ctas_migration_job[hiveserde]: AssertionError: assert False (8m33.846s)

``` AssertionError: assert False + where False = >('migrate-external-tables-ctas') + where > = .validate_step + where = .deployed_workflows 18:16 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/7tE4 18:16 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.hiveserde_in_place_hubnk: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_hubnk 18:16 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.hiveserde_in_place_hubnk', metastore_id=None, name='hiveserde_in_place_hubnk', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 18:16 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_storage_dir fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_hubnk 18:16 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_hubnk.parquet_serde_hubnk: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_hubnk/parquet_serde_hubnk 18:16 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_hubnk.parquet_serde_hubnk', metastore_id=None, name='parquet_serde_hubnk', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='hiveserde_in_place_hubnk', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_hubnk/parquet_serde_hubnk', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 18:16 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_hubnk.orc_serde_hubnk: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_hubnk/orc_serde_hubnk 18:16 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_hubnk.orc_serde_hubnk', metastore_id=None, name='orc_serde_hubnk', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='hiveserde_in_place_hubnk', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_hubnk/orc_serde_hubnk', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 18:16 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_hubnk.avro_serde_hubnk: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_hubnk/avro_serde_hubnk 18:16 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_hubnk.avro_serde_hubnk', metastore_id=None, name='avro_serde_hubnk', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='hiveserde_in_place_hubnk', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_hubnk/avro_serde_hubnk', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 18:16 DEBUG [databricks.labs.ucx.mixins.fixtures] added catalog fixture: CatalogInfo(browse_only=False, catalog_type=, comment='', connection_name=None, created_at=1719944182453, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_c6sfo', isolation_mode=, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_c6sfo', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1719944182453, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d') 18:16 INFO [databricks.labs.ucx.mixins.fixtures] Schema ucx_c6sfo.hiveserde_in_place_hubnk: https://DATABRICKS_HOST/explore/data/ucx_c6sfo/hiveserde_in_place_hubnk 18:16 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_c6sfo', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_c6sfo.hiveserde_in_place_hubnk', metastore_id=None, name='hiveserde_in_place_hubnk', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 18:16 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_syzol: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_syzol 18:16 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_syzol', metastore_id=None, name='ucx_syzol', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) [gw9] linux -- Python 3.10.14 /home/runner/work/ucx/ucx/.venv/bin/python 18:16 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/7tE4 18:16 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.hiveserde_in_place_hubnk: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_hubnk 18:16 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.hiveserde_in_place_hubnk', metastore_id=None, name='hiveserde_in_place_hubnk', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 18:16 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_storage_dir fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_hubnk 18:16 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_hubnk.parquet_serde_hubnk: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_hubnk/parquet_serde_hubnk 18:16 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_hubnk.parquet_serde_hubnk', metastore_id=None, name='parquet_serde_hubnk', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='hiveserde_in_place_hubnk', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_hubnk/parquet_serde_hubnk', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 18:16 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_hubnk.orc_serde_hubnk: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_hubnk/orc_serde_hubnk 18:16 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_hubnk.orc_serde_hubnk', metastore_id=None, name='orc_serde_hubnk', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='hiveserde_in_place_hubnk', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_hubnk/orc_serde_hubnk', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 18:16 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_hubnk.avro_serde_hubnk: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_hubnk/avro_serde_hubnk 18:16 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_hubnk.avro_serde_hubnk', metastore_id=None, name='avro_serde_hubnk', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='hiveserde_in_place_hubnk', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_hubnk/avro_serde_hubnk', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 18:16 DEBUG [databricks.labs.ucx.mixins.fixtures] added catalog fixture: CatalogInfo(browse_only=False, catalog_type=, comment='', connection_name=None, created_at=1719944182453, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_c6sfo', isolation_mode=, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_c6sfo', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1719944182453, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d') 18:16 INFO [databricks.labs.ucx.mixins.fixtures] Schema ucx_c6sfo.hiveserde_in_place_hubnk: https://DATABRICKS_HOST/explore/data/ucx_c6sfo/hiveserde_in_place_hubnk 18:16 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_c6sfo', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_c6sfo.hiveserde_in_place_hubnk', metastore_id=None, name='hiveserde_in_place_hubnk', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 18:16 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_syzol: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_syzol 18:16 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_syzol', metastore_id=None, name='ucx_syzol', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 18:16 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.pFPy/config.yml) doesn't exist. 18:16 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration 18:18 INFO [databricks.labs.ucx.installer.hms_lineage] HMS Lineage feature creates one system table named system.hms_to_uc_migration.table_access and helps in your migration process from HMS to UC by allowing you to programmatically query HMS lineage data. 18:18 INFO [databricks.labs.ucx.install] Fetching installations... 18:19 WARNING [databricks.labs.ucx.install] Existing installation at /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.pFPy is corrupted. Skipping... 18:19 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy. 18:19 DEBUG [tests.integration.conftest] Waiting for clusters to start... 18:19 DEBUG [tests.integration.conftest] Waiting for clusters to start... 18:19 INFO [databricks.labs.ucx.install] Installing UCX v0.27.2+6120240702181916 18:19 INFO [databricks.labs.ucx.install] Creating ucx schemas... 18:19 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment 18:19 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions 18:19 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-tables-ctas 18:19 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-experimental 18:19 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=experimental-workflow-linter 18:19 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-data-reconciliation 18:19 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=failing 18:19 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups 18:19 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables 18:19 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=scan-tables-in-mounts-experimental 18:19 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental 18:19 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-hiveserde-tables-in-place-experimental 18:19 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups 18:19 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.pFPy/README for the next steps. 18:19 DEBUG [databricks.labs.ucx.installer.workflows] starting migrate-external-tables-ctas job: https://DATABRICKS_HOST#job/64328659555822 18:24 DEBUG [databricks.labs.ucx.installer.workflows] Validating migrate-external-tables-ctas workflow: https://DATABRICKS_HOST#job/64328659555822 18:16 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/7tE4 18:16 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.hiveserde_in_place_hubnk: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_hubnk 18:16 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.hiveserde_in_place_hubnk', metastore_id=None, name='hiveserde_in_place_hubnk', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 18:16 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_storage_dir fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_hubnk 18:16 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_hubnk.parquet_serde_hubnk: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_hubnk/parquet_serde_hubnk 18:16 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_hubnk.parquet_serde_hubnk', metastore_id=None, name='parquet_serde_hubnk', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='hiveserde_in_place_hubnk', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_hubnk/parquet_serde_hubnk', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 18:16 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_hubnk.orc_serde_hubnk: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_hubnk/orc_serde_hubnk 18:16 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_hubnk.orc_serde_hubnk', metastore_id=None, name='orc_serde_hubnk', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='hiveserde_in_place_hubnk', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_hubnk/orc_serde_hubnk', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 18:16 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_hubnk.avro_serde_hubnk: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_hubnk/avro_serde_hubnk 18:16 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_hubnk.avro_serde_hubnk', metastore_id=None, name='avro_serde_hubnk', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='hiveserde_in_place_hubnk', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_hubnk/avro_serde_hubnk', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 18:16 DEBUG [databricks.labs.ucx.mixins.fixtures] added catalog fixture: CatalogInfo(browse_only=False, catalog_type=, comment='', connection_name=None, created_at=1719944182453, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_c6sfo', isolation_mode=, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_c6sfo', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1719944182453, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d') 18:16 INFO [databricks.labs.ucx.mixins.fixtures] Schema ucx_c6sfo.hiveserde_in_place_hubnk: https://DATABRICKS_HOST/explore/data/ucx_c6sfo/hiveserde_in_place_hubnk 18:16 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_c6sfo', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_c6sfo.hiveserde_in_place_hubnk', metastore_id=None, name='hiveserde_in_place_hubnk', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 18:16 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_syzol: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_syzol 18:16 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_syzol', metastore_id=None, name='ucx_syzol', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 18:16 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.pFPy/config.yml) doesn't exist. 18:16 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration 18:18 INFO [databricks.labs.ucx.installer.hms_lineage] HMS Lineage feature creates one system table named system.hms_to_uc_migration.table_access and helps in your migration process from HMS to UC by allowing you to programmatically query HMS lineage data. 18:18 INFO [databricks.labs.ucx.install] Fetching installations... 18:19 WARNING [databricks.labs.ucx.install] Existing installation at /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.pFPy is corrupted. Skipping... 18:19 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy. 18:19 DEBUG [tests.integration.conftest] Waiting for clusters to start... 18:19 DEBUG [tests.integration.conftest] Waiting for clusters to start... 18:19 INFO [databricks.labs.ucx.install] Installing UCX v0.27.2+6120240702181916 18:19 INFO [databricks.labs.ucx.install] Creating ucx schemas... 18:19 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment 18:19 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions 18:19 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-tables-ctas 18:19 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-experimental 18:19 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=experimental-workflow-linter 18:19 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-data-reconciliation 18:19 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=failing 18:19 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups 18:19 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables 18:19 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=scan-tables-in-mounts-experimental 18:19 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental 18:19 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-hiveserde-tables-in-place-experimental 18:19 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups 18:19 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.pFPy/README for the next steps. 18:19 DEBUG [databricks.labs.ucx.installer.workflows] starting migrate-external-tables-ctas job: https://DATABRICKS_HOST#job/64328659555822 18:24 DEBUG [databricks.labs.ucx.installer.workflows] Validating migrate-external-tables-ctas workflow: https://DATABRICKS_HOST#job/64328659555822 18:24 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 make_storage_dir fixtures 18:24 DEBUG [databricks.labs.ucx.mixins.fixtures] removing make_storage_dir fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_hubnk 18:24 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 make_dbfs_data_copy fixtures 18:24 DEBUG [databricks.labs.ucx.mixins.fixtures] removing make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/7tE4 18:24 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 cluster fixtures 18:24 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 catalog fixtures 18:24 DEBUG [databricks.labs.ucx.mixins.fixtures] removing catalog fixture: CatalogInfo(browse_only=False, catalog_type=, comment='', connection_name=None, created_at=1719944182453, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_c6sfo', isolation_mode=, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_c6sfo', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1719944182453, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d') 18:24 INFO [databricks.labs.ucx.install] Deleting UCX v0.27.2+6120240702181916 from https://DATABRICKS_HOST 18:24 INFO [databricks.labs.ucx.install] Deleting inventory database ucx_syzol 18:24 INFO [databricks.labs.ucx.install] Deleting jobs 18:24 INFO [databricks.labs.ucx.install] Deleting assessment job_id=188095318638174. 18:24 INFO [databricks.labs.ucx.install] Deleting validate-groups-permissions job_id=296087141794887. 18:24 INFO [databricks.labs.ucx.install] Deleting migrate-external-tables-ctas job_id=64328659555822. 18:24 INFO [databricks.labs.ucx.install] Deleting migrate-groups-experimental job_id=1094288873993953. 18:24 INFO [databricks.labs.ucx.install] Deleting experimental-workflow-linter job_id=262655746236330. 18:24 INFO [databricks.labs.ucx.install] Deleting migrate-data-reconciliation job_id=1101776232887340. 18:24 INFO [databricks.labs.ucx.install] Deleting failing job_id=461715561651511. 18:24 INFO [databricks.labs.ucx.install] Deleting migrate-groups job_id=519242336514480. 18:24 INFO [databricks.labs.ucx.install] Deleting migrate-tables job_id=393621674354792. 18:24 INFO [databricks.labs.ucx.install] Deleting scan-tables-in-mounts-experimental job_id=50337631169735. 18:24 INFO [databricks.labs.ucx.install] Deleting migrate-tables-in-mounts-experimental job_id=299511088513197. 18:24 INFO [databricks.labs.ucx.install] Deleting migrate-external-hiveserde-tables-in-place-experimental job_id=379509302246030. 18:24 INFO [databricks.labs.ucx.install] Deleting remove-workspace-local-backup-groups job_id=964405286792555. 18:24 INFO [databricks.labs.ucx.install] Deleting cluster policy 18:24 INFO [databricks.labs.ucx.install] Deleting secret scope 18:24 INFO [databricks.labs.ucx.install] UnInstalling UCX complete 18:24 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 workspace user fixtures 18:24 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 account group fixtures 18:24 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 workspace group fixtures 18:24 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 table fixtures 18:24 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 3 table fixtures 18:24 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_hubnk.parquet_serde_hubnk', metastore_id=None, name='parquet_serde_hubnk', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='hiveserde_in_place_hubnk', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_hubnk/parquet_serde_hubnk', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 18:24 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_hubnk.orc_serde_hubnk', metastore_id=None, name='orc_serde_hubnk', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='hiveserde_in_place_hubnk', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_hubnk/orc_serde_hubnk', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 18:24 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_hubnk.avro_serde_hubnk', metastore_id=None, name='avro_serde_hubnk', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='hiveserde_in_place_hubnk', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_hubnk/avro_serde_hubnk', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 18:24 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 3 schema fixtures 18:24 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.hiveserde_in_place_hubnk', metastore_id=None, name='hiveserde_in_place_hubnk', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 18:24 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_c6sfo', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_c6sfo.hiveserde_in_place_hubnk', metastore_id=None, name='hiveserde_in_place_hubnk', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 18:24 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_syzol', metastore_id=None, name='ucx_syzol', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) [gw9] linux -- Python 3.10.14 /home/runner/work/ucx/ucx/.venv/bin/python ```

❌ test_running_real_migrate_groups_job: KeyError: 'rename-RG6E-ucx_GOWpN' (12m48.085s)

``` KeyError: 'rename-RG6E-ucx_GOWpN' 18:12 INFO [databricks.sdk] Using Databricks Metadata Service authentication 18:12 INFO [databricks.sdk] Using Databricks Metadata Service authentication [gw6] linux -- Python 3.10.14 /home/runner/work/ucx/ucx/.venv/bin/python 18:12 INFO [databricks.sdk] Using Databricks Metadata Service authentication 18:12 INFO [databricks.sdk] Using Databricks Metadata Service authentication 18:12 DEBUG [databricks.labs.ucx.mixins.fixtures] added workspace user fixture: User(active=True, display_name='sdk-znormwcj@example.com', emails=[ComplexValue(display=None, primary=True, ref=None, type='work', value='sdk-znormwcj@example.com')], entitlements=[], external_id=None, groups=[], id='5376749525407869', name=Name(family_name=None, given_name='sdk-znormwcj@example.com'), roles=[], schemas=[, ], user_name='sdk-znormwcj@example.com') 18:12 INFO [databricks.labs.ucx.mixins.fixtures] Workspace group ucx_GOWpN: https://DATABRICKS_HOST#setting/accounts/groups/209305275978201 18:12 DEBUG [databricks.labs.ucx.mixins.fixtures] added workspace group fixture: Group(display_name='ucx_GOWpN', entitlements=[ComplexValue(display=None, primary=None, ref=None, type=None, value='allow-cluster-create')], external_id=None, groups=[], id='209305275978201', members=[ComplexValue(display='sdk-znormwcj@example.com', primary=None, ref='Users/5376749525407869', type=None, value='5376749525407869')], meta=ResourceMeta(resource_type='WorkspaceGroup'), roles=[], schemas=[]) 18:12 INFO [databricks.labs.ucx.mixins.fixtures] Account group ucx_GOWpN: https://accounts.CLOUD_ENVdatabricks.net/users/groups/958336894013053/members 18:12 DEBUG [databricks.labs.ucx.mixins.fixtures] added account group fixture: Group(display_name='ucx_GOWpN', entitlements=[], external_id=None, groups=[], id='958336894013053', members=[ComplexValue(display='sdk-znormwcj@example.com', primary=None, ref='Users/5376749525407869', type=None, value='5376749525407869')], meta=None, roles=[], schemas=[]) 18:12 INFO [databricks.labs.ucx.mixins.fixtures] Cluster policy: https://DATABRICKS_HOST#setting/clusters/cluster-policies/view/00097345E4D32506 18:12 DEBUG [databricks.labs.ucx.mixins.fixtures] added cluster policy fixture: CreatePolicyResponse(policy_id='00097345E4D32506') 18:12 DEBUG [databricks.labs.ucx.mixins.fixtures] added cluster_policy permissions fixture: 00097345E4D32506 [group_name admins CAN_USE] -> [group_name ucx_GOWpN CAN_USE] 18:12 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_sg2sf: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_sg2sf 18:12 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_sg2sf', metastore_id=None, name='ucx_sg2sf', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 18:12 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.ucx_sg2sf.ucx_t8aav: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_sg2sf/ucx_t8aav 18:12 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.ucx_sg2sf.ucx_t8aav', metastore_id=None, name='ucx_t8aav', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='ucx_sg2sf', sql_path=None, storage_credential_name=None, storage_location='dbfs:/user/hive/warehouse/ucx_sg2sf/ucx_t8aav', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 18:12 DEBUG [databricks.labs.ucx.mixins.fixtures] added secret scope fixture: sdk-qAAn 18:12 DEBUG [databricks.labs.ucx.mixins.fixtures] added secret scope acl fixture: ('sdk-qAAn', 'ucx_GOWpN') 18:12 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_sz3u0: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_sz3u0 18:12 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_sz3u0', metastore_id=None, name='ucx_sz3u0', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 18:12 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.RG6E/config.yml) doesn't exist. 18:12 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration 18:12 INFO [databricks.labs.blueprint.tui] Asking prompt: Inventory Database stored in hive_metastore 18:12 INFO [databricks.labs.blueprint.tui] Asking prompt: Log level 18:12 INFO [databricks.labs.blueprint.tui] Asking prompt: Number of threads 18:12 INFO [databricks.labs.blueprint.tui] Asking prompt: Backup prefix 18:12 INFO [databricks.labs.blueprint.tui] Asking prompt: [1mChoose how to map the workspace groups:[0m [1m[0][0m [36mMatch by Name[0m [1m[1][0m [36mApply a Prefix[0m [1m[2][0m [36mApply a Suffix[0m [1m[3][0m [36mMatch by External ID[0m [1m[4][0m [36mRegex Substitution[0m [1m[5][0m [36mRegex Matching[0m Enter a number between 0 and 5 18:12 INFO [databricks.labs.blueprint.tui] Asking prompt: Enter a prefix to add to the workspace group name 18:12 INFO [databricks.labs.blueprint.tui] Asking prompt: Comma-separated list of workspace group names to migrate. If not specified, we'll use all account-level groups with matching names to workspace-level groups 18:12 INFO [databricks.labs.blueprint.tui] Asking prompt: Comma-separated list of databases to migrate. If not specified, we'll use all databases in hive_metastore 18:12 INFO [databricks.labs.blueprint.tui] Asking prompt: Does given workspace 7342989205138882 block Internet access? 18:12 INFO [databricks.labs.blueprint.tui] Asking prompt: Do you want to trigger assessment job after installation? 18:12 INFO [databricks.labs.blueprint.tui] Asking prompt: Reconciliation threshold, in percentage 18:12 INFO [databricks.labs.ucx.installer.hms_lineage] HMS Lineage feature creates one system table named system.hms_to_uc_migration.table_access and helps in your migration process from HMS to UC by allowing you to programmatically query HMS lineage data. 18:12 INFO [databricks.labs.blueprint.tui] Asking prompt: No HMS lineage collection init script exists, do you want to create one? 18:12 INFO [databricks.labs.ucx.install] Fetching installations... 18:12 INFO [databricks.labs.blueprint.parallel] finding RG6E installations 100/325, rps: 212.433/sec 18:12 INFO [databricks.labs.blueprint.parallel] finding RG6E installations 200/325, rps: 231.814/sec 18:12 INFO [databricks.labs.blueprint.parallel] finding RG6E installations 300/325, rps: 236.696/sec 18:12 INFO [databricks.labs.blueprint.parallel] finding RG6E installations 325/325, rps: 101.759/sec 18:12 INFO [databricks.labs.blueprint.parallel] Finished 'finding RG6E installations' tasks: 0% results available (0/325). Took 0:00:03.195769 18:12 INFO [databricks.labs.blueprint.tui] Asking prompt: [1mSelect PRO or SERVERLESS SQL warehouse to run assessment dashboards on[0m [1m[0][0m [36m[Create new PRO SQL warehouse][0m [1m[1][0m [36mDEFAULT Test Warehouse (TEST_DEFAULT_WAREHOUSE_ID, SERVERLESS, RUNNING)[0m Enter a number between 0 and 1 18:12 INFO [databricks.labs.blueprint.tui] Asking prompt: Instance pool id to be set in cluster policy for all workflow clusters 18:12 INFO [databricks.labs.blueprint.tui] Asking prompt: We have identified one or more cluster policies set up for an external metastore. Would you like to set UCX to connect to the external metastore? 18:12 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy. 18:12 INFO [databricks.labs.blueprint.tui] Asking prompt: Parallelism for migrating dbfs root delta tables with deep clone 18:12 INFO [databricks.labs.blueprint.tui] Asking prompt: Min workers for auto-scale job cluster for table migration 18:12 INFO [databricks.labs.blueprint.tui] Asking prompt: Max workers for auto-scale job cluster for table migration 18:12 INFO [databricks.labs.blueprint.tui] Asking prompt: Open config file in the browser and continue installing? https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.RG6E/config.yml 18:12 DEBUG [tests.integration.conftest] Waiting for clusters to start... 18:12 INFO [databricks.labs.blueprint.parallel] ensure clusters running 3/3, rps: 55.123/sec 18:12 INFO [databricks.labs.blueprint.parallel] Finished 'ensure clusters running' tasks: 0% results available (0/3). Took 0:00:00.054709 18:12 DEBUG [tests.integration.conftest] Waiting for clusters to start... 18:12 INFO [databricks.labs.ucx.install] Installing UCX v0.27.2+6120240702181252 18:12 INFO [databricks.labs.ucx.install] Creating ucx schemas... 18:12 INFO [databricks.labs.lsql.deployment] Ensuring ucx_sz3u0 database exists 18:12 INFO [databricks.labs.lsql.deployment] Ensuring ucx_sz3u0.CLOUD_ENV_service_principals table exists 18:12 INFO [databricks.labs.lsql.deployment] Ensuring ucx_sz3u0.clusters table exists 18:12 INFO [databricks.labs.lsql.deployment] Ensuring ucx_sz3u0.global_init_scripts table exists 18:12 INFO [databricks.labs.lsql.deployment] Ensuring ucx_sz3u0.jobs table exists 18:12 INFO [databricks.labs.lsql.deployment] Ensuring ucx_sz3u0.pipelines table exists 18:12 INFO [databricks.labs.lsql.deployment] Ensuring ucx_sz3u0.external_locations table exists 18:12 INFO [databricks.labs.lsql.deployment] Ensuring ucx_sz3u0.mounts table exists 18:12 INFO [databricks.labs.lsql.deployment] Ensuring ucx_sz3u0.grants table exists 18:12 INFO [databricks.labs.lsql.deployment] Ensuring ucx_sz3u0.groups table exists 18:12 INFO [databricks.labs.lsql.deployment] Ensuring ucx_sz3u0.tables table exists 18:12 INFO [databricks.labs.lsql.deployment] Ensuring ucx_sz3u0.table_size table exists 18:12 INFO [databricks.labs.lsql.deployment] Ensuring ucx_sz3u0.table_failures table exists 18:12 INFO [databricks.labs.lsql.deployment] Ensuring ucx_sz3u0.workspace_objects table exists 18:12 INFO [databricks.labs.lsql.deployment] Ensuring ucx_sz3u0.permissions table exists 18:12 INFO [databricks.labs.lsql.deployment] Ensuring ucx_sz3u0.submit_runs table exists 18:12 INFO [databricks.labs.lsql.deployment] Ensuring ucx_sz3u0.policies table exists 18:12 INFO [databricks.labs.lsql.deployment] Ensuring ucx_sz3u0.migration_status table exists 18:12 INFO [databricks.labs.lsql.deployment] Ensuring ucx_sz3u0.workflow_problems table exists 18:12 INFO [databricks.labs.lsql.deployment] Ensuring ucx_sz3u0.udfs table exists 18:12 INFO [databricks.labs.lsql.deployment] Ensuring ucx_sz3u0.logs table exists 18:12 INFO [databricks.labs.lsql.deployment] Ensuring ucx_sz3u0.recon_results table exists 18:12 INFO [databricks.labs.blueprint.parallel] deploy tables 10/21, rps: 2.046/sec 18:13 INFO [databricks.labs.blueprint.parallel] deploy tables 20/21, rps: 1.710/sec 18:13 INFO [databricks.labs.blueprint.parallel] deploy tables 21/21, rps: 1.795/sec 18:13 INFO [databricks.labs.blueprint.parallel] Finished 'deploy tables' tasks: 0% results available (0/21). Took 0:00:11.699970 18:13 INFO [databricks.labs.lsql.deployment] Ensuring ucx_sz3u0.grant_detail view matches queries/views/grant_detail.sql contents 18:13 INFO [databricks.labs.lsql.deployment] Ensuring ucx_sz3u0.objects view matches queries/views/objects.sql contents 18:13 INFO [databricks.labs.lsql.deployment] Ensuring ucx_sz3u0.table_estimates view matches queries/views/table_estimates.sql contents 18:13 INFO [databricks.labs.lsql.deployment] Ensuring ucx_sz3u0.misc_patterns view matches queries/views/misc_patterns.sql contents 18:13 INFO [databricks.labs.lsql.deployment] Ensuring ucx_sz3u0.code_patterns view matches queries/views/code_patterns.sql contents 18:13 INFO [databricks.labs.lsql.deployment] Ensuring ucx_sz3u0.reconciliation_results view matches queries/views/reconciliation_results.sql contents 18:13 INFO [databricks.labs.blueprint.parallel] installing components 1/1, rps: 0.027/sec 18:13 INFO [databricks.labs.blueprint.parallel] Finished 'installing components' tasks: 0% results available (0/1). Took 0:00:37.584231 18:13 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=failing 18:13 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-data-reconciliation 18:13 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-experimental 18:13 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups 18:13 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=experimental-workflow-linter 18:13 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment 18:13 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-hiveserde-tables-in-place-experimental 18:13 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables 18:13 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-tables-ctas 18:13 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=scan-tables-in-mounts-experimental 18:13 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental 18:13 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions 18:13 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups 18:13 INFO [databricks.labs.blueprint.tui] Asking prompt: Open job overview in your browser? https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.RG6E/README 18:13 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.RG6E/README for the next steps. 18:18 DEBUG [databricks.labs.ucx.workspace_access.manager] Crawling permissions 18:18 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.ucx_sz3u0.grants] fetching grants inventory 18:18 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.ucx_sz3u0.grants] crawling new batch for grants 18:18 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.ucx_sz3u0.tables] fetching tables inventory 18:18 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.ucx_sz3u0.tables] crawling new batch for tables 18:18 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sg2sf] listing tables 18:18 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sg2sf.ucx_t8aav] fetching table metadata 18:18 INFO [databricks.labs.blueprint.parallel] listing tables in hive_metastore 1/1, rps: 1.191/sec 18:18 INFO [databricks.labs.blueprint.parallel] Finished 'listing tables in hive_metastore' tasks: 100% results available (1/1). Took 0:00:00.840086 18:18 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.ucx_sz3u0.tables] found 1 new records for tables 18:18 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.ucx_sz3u0.udfs] fetching udfs inventory 18:18 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.ucx_sz3u0.udfs] crawling new batch for udfs 18:18 DEBUG [databricks.labs.ucx.hive_metastore.udfs] [hive_metastore.ucx_sg2sf] listing udfs 18:18 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.ucx_sz3u0.udfs] found 0 new records for udfs 18:20 INFO [databricks.labs.blueprint.parallel] listing grants for hive_metastore 5/5, rps: 0.079/sec 18:20 INFO [databricks.labs.blueprint.parallel] Finished 'listing grants for hive_metastore' tasks: 100% results available (5/5). Took 0:01:03.272368 18:20 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.ucx_sz3u0.grants] found 283 new records for grants 18:20 INFO [databricks.labs.ucx.workspace_access.manager] Starting to crawl permissions. Total tasks: 4 18:20 INFO [databricks.labs.blueprint.parallel] crawl permissions 4/4, rps: 17.887/sec 18:20 INFO [databricks.labs.blueprint.parallel] Finished 'crawl permissions' tasks: 100% results available (4/4). Took 0:00:00.223882 18:20 INFO [databricks.labs.ucx.workspace_access.manager] Total crawled permissions: 4 18:20 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.ucx_sz3u0.permissions] found 4 new records for permissions 18:20 INFO [databricks.labs.ucx.workspace_access.manager] Successfully saved the items to inventory table 18:20 INFO [databricks.labs.ucx.workspace_access.manager] Saved 4 to hive_metastore.ucx_sz3u0.permissions 18:20 DEBUG [databricks.labs.ucx.installer.workflows] starting migrate-groups job: https://DATABRICKS_HOST#job/210507514958487 18:12 INFO [databricks.sdk] Using Databricks Metadata Service authentication 18:12 INFO [databricks.sdk] Using Databricks Metadata Service authentication 18:12 DEBUG [databricks.labs.ucx.mixins.fixtures] added workspace user fixture: User(active=True, display_name='sdk-znormwcj@example.com', emails=[ComplexValue(display=None, primary=True, ref=None, type='work', value='sdk-znormwcj@example.com')], entitlements=[], external_id=None, groups=[], id='5376749525407869', name=Name(family_name=None, given_name='sdk-znormwcj@example.com'), roles=[], schemas=[, ], user_name='sdk-znormwcj@example.com') 18:12 INFO [databricks.labs.ucx.mixins.fixtures] Workspace group ucx_GOWpN: https://DATABRICKS_HOST#setting/accounts/groups/209305275978201 18:12 DEBUG [databricks.labs.ucx.mixins.fixtures] added workspace group fixture: Group(display_name='ucx_GOWpN', entitlements=[ComplexValue(display=None, primary=None, ref=None, type=None, value='allow-cluster-create')], external_id=None, groups=[], id='209305275978201', members=[ComplexValue(display='sdk-znormwcj@example.com', primary=None, ref='Users/5376749525407869', type=None, value='5376749525407869')], meta=ResourceMeta(resource_type='WorkspaceGroup'), roles=[], schemas=[]) 18:12 INFO [databricks.labs.ucx.mixins.fixtures] Account group ucx_GOWpN: https://accounts.CLOUD_ENVdatabricks.net/users/groups/958336894013053/members 18:12 DEBUG [databricks.labs.ucx.mixins.fixtures] added account group fixture: Group(display_name='ucx_GOWpN', entitlements=[], external_id=None, groups=[], id='958336894013053', members=[ComplexValue(display='sdk-znormwcj@example.com', primary=None, ref='Users/5376749525407869', type=None, value='5376749525407869')], meta=None, roles=[], schemas=[]) 18:12 INFO [databricks.labs.ucx.mixins.fixtures] Cluster policy: https://DATABRICKS_HOST#setting/clusters/cluster-policies/view/00097345E4D32506 18:12 DEBUG [databricks.labs.ucx.mixins.fixtures] added cluster policy fixture: CreatePolicyResponse(policy_id='00097345E4D32506') 18:12 DEBUG [databricks.labs.ucx.mixins.fixtures] added cluster_policy permissions fixture: 00097345E4D32506 [group_name admins CAN_USE] -> [group_name ucx_GOWpN CAN_USE] 18:12 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_sg2sf: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_sg2sf 18:12 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_sg2sf', metastore_id=None, name='ucx_sg2sf', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 18:12 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.ucx_sg2sf.ucx_t8aav: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_sg2sf/ucx_t8aav 18:12 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.ucx_sg2sf.ucx_t8aav', metastore_id=None, name='ucx_t8aav', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='ucx_sg2sf', sql_path=None, storage_credential_name=None, storage_location='dbfs:/user/hive/warehouse/ucx_sg2sf/ucx_t8aav', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 18:12 DEBUG [databricks.labs.ucx.mixins.fixtures] added secret scope fixture: sdk-qAAn 18:12 DEBUG [databricks.labs.ucx.mixins.fixtures] added secret scope acl fixture: ('sdk-qAAn', 'ucx_GOWpN') 18:12 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_sz3u0: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_sz3u0 18:12 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_sz3u0', metastore_id=None, name='ucx_sz3u0', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 18:12 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.RG6E/config.yml) doesn't exist. 18:12 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration 18:12 INFO [databricks.labs.blueprint.tui] Asking prompt: Inventory Database stored in hive_metastore 18:12 INFO [databricks.labs.blueprint.tui] Asking prompt: Log level 18:12 INFO [databricks.labs.blueprint.tui] Asking prompt: Number of threads 18:12 INFO [databricks.labs.blueprint.tui] Asking prompt: Backup prefix 18:12 INFO [databricks.labs.blueprint.tui] Asking prompt: [1mChoose how to map the workspace groups:[0m [1m[0][0m [36mMatch by Name[0m [1m[1][0m [36mApply a Prefix[0m [1m[2][0m [36mApply a Suffix[0m [1m[3][0m [36mMatch by External ID[0m [1m[4][0m [36mRegex Substitution[0m [1m[5][0m [36mRegex Matching[0m Enter a number between 0 and 5 18:12 INFO [databricks.labs.blueprint.tui] Asking prompt: Enter a prefix to add to the workspace group name 18:12 INFO [databricks.labs.blueprint.tui] Asking prompt: Comma-separated list of workspace group names to migrate. If not specified, we'll use all account-level groups with matching names to workspace-level groups 18:12 INFO [databricks.labs.blueprint.tui] Asking prompt: Comma-separated list of databases to migrate. If not specified, we'll use all databases in hive_metastore 18:12 INFO [databricks.labs.blueprint.tui] Asking prompt: Does given workspace 7342989205138882 block Internet access? 18:12 INFO [databricks.labs.blueprint.tui] Asking prompt: Do you want to trigger assessment job after installation? 18:12 INFO [databricks.labs.blueprint.tui] Asking prompt: Reconciliation threshold, in percentage 18:12 INFO [databricks.labs.ucx.installer.hms_lineage] HMS Lineage feature creates one system table named system.hms_to_uc_migration.table_access and helps in your migration process from HMS to UC by allowing you to programmatically query HMS lineage data. 18:12 INFO [databricks.labs.blueprint.tui] Asking prompt: No HMS lineage collection init script exists, do you want to create one? 18:12 INFO [databricks.labs.ucx.install] Fetching installations... 18:12 INFO [databricks.labs.blueprint.parallel] finding RG6E installations 100/325, rps: 212.433/sec 18:12 INFO [databricks.labs.blueprint.parallel] finding RG6E installations 200/325, rps: 231.814/sec 18:12 INFO [databricks.labs.blueprint.parallel] finding RG6E installations 300/325, rps: 236.696/sec 18:12 INFO [databricks.labs.blueprint.parallel] finding RG6E installations 325/325, rps: 101.759/sec 18:12 INFO [databricks.labs.blueprint.parallel] Finished 'finding RG6E installations' tasks: 0% results available (0/325). Took 0:00:03.195769 18:12 INFO [databricks.labs.blueprint.tui] Asking prompt: [1mSelect PRO or SERVERLESS SQL warehouse to run assessment dashboards on[0m [1m[0][0m [36m[Create new PRO SQL warehouse][0m [1m[1][0m [36mDEFAULT Test Warehouse (TEST_DEFAULT_WAREHOUSE_ID, SERVERLESS, RUNNING)[0m Enter a number between 0 and 1 18:12 INFO [databricks.labs.blueprint.tui] Asking prompt: Instance pool id to be set in cluster policy for all workflow clusters 18:12 INFO [databricks.labs.blueprint.tui] Asking prompt: We have identified one or more cluster policies set up for an external metastore. Would you like to set UCX to connect to the external metastore? 18:12 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy. 18:12 INFO [databricks.labs.blueprint.tui] Asking prompt: Parallelism for migrating dbfs root delta tables with deep clone 18:12 INFO [databricks.labs.blueprint.tui] Asking prompt: Min workers for auto-scale job cluster for table migration 18:12 INFO [databricks.labs.blueprint.tui] Asking prompt: Max workers for auto-scale job cluster for table migration 18:12 INFO [databricks.labs.blueprint.tui] Asking prompt: Open config file in the browser and continue installing? https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.RG6E/config.yml 18:12 DEBUG [tests.integration.conftest] Waiting for clusters to start... 18:12 INFO [databricks.labs.blueprint.parallel] ensure clusters running 3/3, rps: 55.123/sec 18:12 INFO [databricks.labs.blueprint.parallel] Finished 'ensure clusters running' tasks: 0% results available (0/3). Took 0:00:00.054709 18:12 DEBUG [tests.integration.conftest] Waiting for clusters to start... 18:12 INFO [databricks.labs.ucx.install] Installing UCX v0.27.2+6120240702181252 18:12 INFO [databricks.labs.ucx.install] Creating ucx schemas... 18:12 INFO [databricks.labs.lsql.deployment] Ensuring ucx_sz3u0 database exists 18:12 INFO [databricks.labs.lsql.deployment] Ensuring ucx_sz3u0.CLOUD_ENV_service_principals table exists 18:12 INFO [databricks.labs.lsql.deployment] Ensuring ucx_sz3u0.clusters table exists 18:12 INFO [databricks.labs.lsql.deployment] Ensuring ucx_sz3u0.global_init_scripts table exists 18:12 INFO [databricks.labs.lsql.deployment] Ensuring ucx_sz3u0.jobs table exists 18:12 INFO [databricks.labs.lsql.deployment] Ensuring ucx_sz3u0.pipelines table exists 18:12 INFO [databricks.labs.lsql.deployment] Ensuring ucx_sz3u0.external_locations table exists 18:12 INFO [databricks.labs.lsql.deployment] Ensuring ucx_sz3u0.mounts table exists 18:12 INFO [databricks.labs.lsql.deployment] Ensuring ucx_sz3u0.grants table exists 18:12 INFO [databricks.labs.lsql.deployment] Ensuring ucx_sz3u0.groups table exists 18:12 INFO [databricks.labs.lsql.deployment] Ensuring ucx_sz3u0.tables table exists 18:12 INFO [databricks.labs.lsql.deployment] Ensuring ucx_sz3u0.table_size table exists 18:12 INFO [databricks.labs.lsql.deployment] Ensuring ucx_sz3u0.table_failures table exists 18:12 INFO [databricks.labs.lsql.deployment] Ensuring ucx_sz3u0.workspace_objects table exists 18:12 INFO [databricks.labs.lsql.deployment] Ensuring ucx_sz3u0.permissions table exists 18:12 INFO [databricks.labs.lsql.deployment] Ensuring ucx_sz3u0.submit_runs table exists 18:12 INFO [databricks.labs.lsql.deployment] Ensuring ucx_sz3u0.policies table exists 18:12 INFO [databricks.labs.lsql.deployment] Ensuring ucx_sz3u0.migration_status table exists 18:12 INFO [databricks.labs.lsql.deployment] Ensuring ucx_sz3u0.workflow_problems table exists 18:12 INFO [databricks.labs.lsql.deployment] Ensuring ucx_sz3u0.udfs table exists 18:12 INFO [databricks.labs.lsql.deployment] Ensuring ucx_sz3u0.logs table exists 18:12 INFO [databricks.labs.lsql.deployment] Ensuring ucx_sz3u0.recon_results table exists 18:12 INFO [databricks.labs.blueprint.parallel] deploy tables 10/21, rps: 2.046/sec 18:13 INFO [databricks.labs.blueprint.parallel] deploy tables 20/21, rps: 1.710/sec 18:13 INFO [databricks.labs.blueprint.parallel] deploy tables 21/21, rps: 1.795/sec 18:13 INFO [databricks.labs.blueprint.parallel] Finished 'deploy tables' tasks: 0% results available (0/21). Took 0:00:11.699970 18:13 INFO [databricks.labs.lsql.deployment] Ensuring ucx_sz3u0.grant_detail view matches queries/views/grant_detail.sql contents 18:13 INFO [databricks.labs.lsql.deployment] Ensuring ucx_sz3u0.objects view matches queries/views/objects.sql contents 18:13 INFO [databricks.labs.lsql.deployment] Ensuring ucx_sz3u0.table_estimates view matches queries/views/table_estimates.sql contents 18:13 INFO [databricks.labs.lsql.deployment] Ensuring ucx_sz3u0.misc_patterns view matches queries/views/misc_patterns.sql contents 18:13 INFO [databricks.labs.lsql.deployment] Ensuring ucx_sz3u0.code_patterns view matches queries/views/code_patterns.sql contents 18:13 INFO [databricks.labs.lsql.deployment] Ensuring ucx_sz3u0.reconciliation_results view matches queries/views/reconciliation_results.sql contents 18:13 INFO [databricks.labs.blueprint.parallel] installing components 1/1, rps: 0.027/sec 18:13 INFO [databricks.labs.blueprint.parallel] Finished 'installing components' tasks: 0% results available (0/1). Took 0:00:37.584231 18:13 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=failing 18:13 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-data-reconciliation 18:13 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-experimental 18:13 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups 18:13 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=experimental-workflow-linter 18:13 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment 18:13 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-hiveserde-tables-in-place-experimental 18:13 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables 18:13 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-tables-ctas 18:13 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=scan-tables-in-mounts-experimental 18:13 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental 18:13 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions 18:13 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups 18:13 INFO [databricks.labs.blueprint.tui] Asking prompt: Open job overview in your browser? https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.RG6E/README 18:13 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.RG6E/README for the next steps. 18:18 DEBUG [databricks.labs.ucx.workspace_access.manager] Crawling permissions 18:18 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.ucx_sz3u0.grants] fetching grants inventory 18:18 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.ucx_sz3u0.grants] crawling new batch for grants 18:18 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.ucx_sz3u0.tables] fetching tables inventory 18:18 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.ucx_sz3u0.tables] crawling new batch for tables 18:18 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sg2sf] listing tables 18:18 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sg2sf.ucx_t8aav] fetching table metadata 18:18 INFO [databricks.labs.blueprint.parallel] listing tables in hive_metastore 1/1, rps: 1.191/sec 18:18 INFO [databricks.labs.blueprint.parallel] Finished 'listing tables in hive_metastore' tasks: 100% results available (1/1). Took 0:00:00.840086 18:18 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.ucx_sz3u0.tables] found 1 new records for tables 18:18 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.ucx_sz3u0.udfs] fetching udfs inventory 18:18 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.ucx_sz3u0.udfs] crawling new batch for udfs 18:18 DEBUG [databricks.labs.ucx.hive_metastore.udfs] [hive_metastore.ucx_sg2sf] listing udfs 18:18 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.ucx_sz3u0.udfs] found 0 new records for udfs 18:20 INFO [databricks.labs.blueprint.parallel] listing grants for hive_metastore 5/5, rps: 0.079/sec 18:20 INFO [databricks.labs.blueprint.parallel] Finished 'listing grants for hive_metastore' tasks: 100% results available (5/5). Took 0:01:03.272368 18:20 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.ucx_sz3u0.grants] found 283 new records for grants 18:20 INFO [databricks.labs.ucx.workspace_access.manager] Starting to crawl permissions. Total tasks: 4 18:20 INFO [databricks.labs.blueprint.parallel] crawl permissions 4/4, rps: 17.887/sec 18:20 INFO [databricks.labs.blueprint.parallel] Finished 'crawl permissions' tasks: 100% results available (4/4). Took 0:00:00.223882 18:20 INFO [databricks.labs.ucx.workspace_access.manager] Total crawled permissions: 4 18:20 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.ucx_sz3u0.permissions] found 4 new records for permissions 18:20 INFO [databricks.labs.ucx.workspace_access.manager] Successfully saved the items to inventory table 18:20 INFO [databricks.labs.ucx.workspace_access.manager] Saved 4 to hive_metastore.ucx_sz3u0.permissions 18:20 DEBUG [databricks.labs.ucx.installer.workflows] starting migrate-groups job: https://DATABRICKS_HOST#job/210507514958487 18:24 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 secret scope acl fixtures 18:24 DEBUG [databricks.labs.ucx.mixins.fixtures] removing secret scope acl fixture: ('sdk-qAAn', 'ucx_GOWpN') 18:24 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 secret scope fixtures 18:24 DEBUG [databricks.labs.ucx.mixins.fixtures] removing secret scope fixture: sdk-qAAn 18:24 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 cluster_policy permissions fixtures 18:24 DEBUG [databricks.labs.ucx.mixins.fixtures] removing cluster_policy permissions fixture: 00097345E4D32506 [group_name admins CAN_USE] -> [group_name ucx_GOWpN CAN_USE] 18:24 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 cluster policy fixtures 18:24 DEBUG [databricks.labs.ucx.mixins.fixtures] removing cluster policy fixture: CreatePolicyResponse(policy_id='00097345E4D32506') 18:24 INFO [databricks.labs.blueprint.tui] Asking prompt: Do you want to uninstall ucx from the workspace too, this would remove ucx project folder, dashboards, queries and jobs 18:24 INFO [databricks.labs.ucx.install] Deleting UCX v0.27.2+6120240702181252 from https://DATABRICKS_HOST 18:24 INFO [databricks.labs.blueprint.tui] Asking prompt: Do you want to delete the inventory database ucx_sz3u0 too? 18:24 INFO [databricks.labs.ucx.install] Deleting inventory database ucx_sz3u0 18:24 INFO [databricks.labs.lsql.deployment] deleting ucx_sz3u0 database 18:24 INFO [databricks.labs.ucx.install] Deleting jobs 18:24 INFO [databricks.labs.ucx.install] Deleting failing job_id=488326450001790. 18:24 INFO [databricks.labs.ucx.install] Deleting migrate-data-reconciliation job_id=84235835109929. 18:24 INFO [databricks.labs.ucx.install] Deleting migrate-groups-experimental job_id=370513978741320. 18:24 INFO [databricks.labs.ucx.install] Deleting migrate-groups job_id=210507514958487. 18:24 INFO [databricks.labs.ucx.install] Deleting experimental-workflow-linter job_id=318314071454682. 18:24 INFO [databricks.labs.ucx.install] Deleting assessment job_id=1066910867013473. 18:24 INFO [databricks.labs.ucx.install] Deleting migrate-external-hiveserde-tables-in-place-experimental job_id=715634396313262. 18:24 INFO [databricks.labs.ucx.install] Deleting migrate-tables job_id=1100426852178255. 18:24 INFO [databricks.labs.ucx.install] Deleting migrate-external-tables-ctas job_id=223925866835119. 18:24 INFO [databricks.labs.ucx.install] Deleting scan-tables-in-mounts-experimental job_id=343488086659759. 18:24 INFO [databricks.labs.ucx.install] Deleting migrate-tables-in-mounts-experimental job_id=847305307013329. 18:24 INFO [databricks.labs.ucx.install] Deleting validate-groups-permissions job_id=666296602338732. 18:24 INFO [databricks.labs.ucx.install] Deleting remove-workspace-local-backup-groups job_id=73557030762793. 18:24 INFO [databricks.labs.ucx.install] Deleting cluster policy 18:24 INFO [databricks.labs.ucx.install] Deleting secret scope 18:24 INFO [databricks.labs.ucx.install] UnInstalling UCX complete 18:24 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 workspace user fixtures 18:24 DEBUG [databricks.labs.ucx.mixins.fixtures] removing workspace user fixture: User(active=True, display_name='sdk-znormwcj@example.com', emails=[ComplexValue(display=None, primary=True, ref=None, type='work', value='sdk-znormwcj@example.com')], entitlements=[], external_id=None, groups=[], id='5376749525407869', name=Name(family_name=None, given_name='sdk-znormwcj@example.com'), roles=[], schemas=[, ], user_name='sdk-znormwcj@example.com') 18:24 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 account group fixtures 18:24 DEBUG [databricks.labs.ucx.mixins.fixtures] removing account group fixture: Group(display_name='ucx_GOWpN', entitlements=[], external_id=None, groups=[], id='958336894013053', members=[ComplexValue(display='sdk-znormwcj@example.com', primary=None, ref='Users/5376749525407869', type=None, value='5376749525407869')], meta=None, roles=[], schemas=[]) 18:24 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 workspace group fixtures 18:24 DEBUG [databricks.labs.ucx.mixins.fixtures] removing workspace group fixture: Group(display_name='ucx_GOWpN', entitlements=[ComplexValue(display=None, primary=None, ref=None, type=None, value='allow-cluster-create')], external_id=None, groups=[], id='209305275978201', members=[ComplexValue(display='sdk-znormwcj@example.com', primary=None, ref='Users/5376749525407869', type=None, value='5376749525407869')], meta=ResourceMeta(resource_type='WorkspaceGroup'), roles=[], schemas=[]) 18:24 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 table fixtures 18:24 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 table fixtures 18:24 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.ucx_sg2sf.ucx_t8aav', metastore_id=None, name='ucx_t8aav', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='ucx_sg2sf', sql_path=None, storage_credential_name=None, storage_location='dbfs:/user/hive/warehouse/ucx_sg2sf/ucx_t8aav', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 18:24 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 2 schema fixtures 18:24 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_sg2sf', metastore_id=None, name='ucx_sg2sf', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 18:24 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_sz3u0', metastore_id=None, name='ucx_sz3u0', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) [gw6] linux -- Python 3.10.14 /home/runner/work/ucx/ucx/.venv/bin/python ```

_{Running from acceptance #4209}

databrickslabs / ucx

whitelist databricks-arc #2004

Changes

Linked issues

Functionality

Tests