search

databrickslabs / ucx

Your best companion for upgrading to Unity Catalog. UCX will guide you, the Databricks customer, through the process of upgrading your account, groups, workspaces, jobs etc. to Unity Catalog.
Other
193 stars 69 forks source link

whitelist spark-ocr #2011

Closed ericvergnaud closed 2 days ago

ericvergnaud commented 2 days ago

Changes

whitelist spark-ocr

Linked issues

Progresses #1901

Functionality

None

Tests

github-actions[bot] commented 2 days ago

❌ 124/129 passed, 5 failed, 22 skipped, 4h45m36s total

❌ test_table_migration_job_refreshes_migration_status[hiveserde-migrate-external-tables-ctas]: TimeoutError: Timed out after 0:05:00 (7m22.242s) ``` TimeoutError: Timed out after 0:05:00 17:48 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/YcOI 17:48 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.hiveserde_in_place_6t5ff: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_6t5ff 17:48 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.hiveserde_in_place_6t5ff', metastore_id=None, name='hiveserde_in_place_6t5ff', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 17:48 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_storage_dir fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_6t5ff 17:48 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_6t5ff.parquet_serde_6t5ff: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_6t5ff/parquet_serde_6t5ff 17:48 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_6t5ff.parquet_serde_6t5ff', metastore_id=None, name='parquet_serde_6t5ff', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='hiveserde_in_place_6t5ff', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_6t5ff/parquet_serde_6t5ff', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 17:48 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_6t5ff.orc_serde_6t5ff: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_6t5ff/orc_serde_6t5ff 17:48 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_6t5ff.orc_serde_6t5ff', metastore_id=None, name='orc_serde_6t5ff', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='hiveserde_in_place_6t5ff', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_6t5ff/orc_serde_6t5ff', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 17:48 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_6t5ff.avro_serde_6t5ff: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_6t5ff/avro_serde_6t5ff 17:48 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_6t5ff.avro_serde_6t5ff', metastore_id=None, name='avro_serde_6t5ff', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='hiveserde_in_place_6t5ff', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_6t5ff/avro_serde_6t5ff', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 17:48 DEBUG [databricks.labs.ucx.mixins.fixtures] added catalog fixture: CatalogInfo(browse_only=False, catalog_type=, comment='', connection_name=None, created_at=1719942509759, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cjahf', isolation_mode=, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_cjahf', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1719942509759, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d') 17:48 INFO [databricks.labs.ucx.mixins.fixtures] Schema ucx_cjahf.hiveserde_in_place_6t5ff: https://DATABRICKS_HOST/explore/data/ucx_cjahf/hiveserde_in_place_6t5ff 17:48 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_cjahf', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cjahf.hiveserde_in_place_6t5ff', metastore_id=None, name='hiveserde_in_place_6t5ff', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 17:48 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_shmwd: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_shmwd 17:48 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_shmwd', metastore_id=None, name='ucx_shmwd', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) [gw0] linux -- Python 3.10.14 /home/runner/work/ucx/ucx/.venv/bin/python 17:48 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/YcOI 17:48 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.hiveserde_in_place_6t5ff: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_6t5ff 17:48 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.hiveserde_in_place_6t5ff', metastore_id=None, name='hiveserde_in_place_6t5ff', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 17:48 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_storage_dir fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_6t5ff 17:48 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_6t5ff.parquet_serde_6t5ff: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_6t5ff/parquet_serde_6t5ff 17:48 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_6t5ff.parquet_serde_6t5ff', metastore_id=None, name='parquet_serde_6t5ff', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='hiveserde_in_place_6t5ff', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_6t5ff/parquet_serde_6t5ff', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 17:48 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_6t5ff.orc_serde_6t5ff: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_6t5ff/orc_serde_6t5ff 17:48 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_6t5ff.orc_serde_6t5ff', metastore_id=None, name='orc_serde_6t5ff', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='hiveserde_in_place_6t5ff', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_6t5ff/orc_serde_6t5ff', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 17:48 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_6t5ff.avro_serde_6t5ff: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_6t5ff/avro_serde_6t5ff 17:48 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_6t5ff.avro_serde_6t5ff', metastore_id=None, name='avro_serde_6t5ff', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='hiveserde_in_place_6t5ff', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_6t5ff/avro_serde_6t5ff', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 17:48 DEBUG [databricks.labs.ucx.mixins.fixtures] added catalog fixture: CatalogInfo(browse_only=False, catalog_type=, comment='', connection_name=None, created_at=1719942509759, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cjahf', isolation_mode=, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_cjahf', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1719942509759, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d') 17:48 INFO [databricks.labs.ucx.mixins.fixtures] Schema ucx_cjahf.hiveserde_in_place_6t5ff: https://DATABRICKS_HOST/explore/data/ucx_cjahf/hiveserde_in_place_6t5ff 17:48 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_cjahf', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cjahf.hiveserde_in_place_6t5ff', metastore_id=None, name='hiveserde_in_place_6t5ff', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 17:48 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_shmwd: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_shmwd 17:48 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_shmwd', metastore_id=None, name='ucx_shmwd', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 17:49 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.QjX0/config.yml) doesn't exist. 17:49 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration 17:49 INFO [databricks.labs.ucx.installer.hms_lineage] HMS Lineage feature creates one system table named system.hms_to_uc_migration.table_access and helps in your migration process from HMS to UC by allowing you to programmatically query HMS lineage data. 17:49 INFO [databricks.labs.ucx.install] Fetching installations... 17:49 WARNING [databricks.labs.ucx.install] Existing installation at /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.QjX0 is corrupted. Skipping... 17:49 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy. 17:49 DEBUG [tests.integration.conftest] Waiting for clusters to start... 17:49 DEBUG [tests.integration.conftest] Waiting for clusters to start... 17:49 INFO [databricks.labs.ucx.install] Installing UCX v0.27.2+6020240702174951 17:49 INFO [databricks.labs.ucx.install] Creating ucx schemas... 17:50 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups 17:50 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=failing 17:50 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-tables-ctas 17:50 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables 17:50 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=experimental-workflow-linter 17:51 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=scan-tables-in-mounts-experimental 17:51 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-hiveserde-tables-in-place-experimental 17:51 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-experimental 17:51 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups 17:51 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment 17:51 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental 17:51 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions 17:51 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-data-reconciliation 17:51 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.QjX0/README for the next steps. 17:51 DEBUG [databricks.labs.ucx.installer.workflows] starting migrate-external-tables-ctas job: https://DATABRICKS_HOST#job/23952184139366 17:48 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/YcOI 17:48 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.hiveserde_in_place_6t5ff: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_6t5ff 17:48 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.hiveserde_in_place_6t5ff', metastore_id=None, name='hiveserde_in_place_6t5ff', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 17:48 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_storage_dir fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_6t5ff 17:48 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_6t5ff.parquet_serde_6t5ff: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_6t5ff/parquet_serde_6t5ff 17:48 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_6t5ff.parquet_serde_6t5ff', metastore_id=None, name='parquet_serde_6t5ff', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='hiveserde_in_place_6t5ff', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_6t5ff/parquet_serde_6t5ff', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 17:48 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_6t5ff.orc_serde_6t5ff: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_6t5ff/orc_serde_6t5ff 17:48 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_6t5ff.orc_serde_6t5ff', metastore_id=None, name='orc_serde_6t5ff', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='hiveserde_in_place_6t5ff', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_6t5ff/orc_serde_6t5ff', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 17:48 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_6t5ff.avro_serde_6t5ff: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_6t5ff/avro_serde_6t5ff 17:48 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_6t5ff.avro_serde_6t5ff', metastore_id=None, name='avro_serde_6t5ff', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='hiveserde_in_place_6t5ff', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_6t5ff/avro_serde_6t5ff', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 17:48 DEBUG [databricks.labs.ucx.mixins.fixtures] added catalog fixture: CatalogInfo(browse_only=False, catalog_type=, comment='', connection_name=None, created_at=1719942509759, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cjahf', isolation_mode=, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_cjahf', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1719942509759, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d') 17:48 INFO [databricks.labs.ucx.mixins.fixtures] Schema ucx_cjahf.hiveserde_in_place_6t5ff: https://DATABRICKS_HOST/explore/data/ucx_cjahf/hiveserde_in_place_6t5ff 17:48 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_cjahf', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cjahf.hiveserde_in_place_6t5ff', metastore_id=None, name='hiveserde_in_place_6t5ff', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 17:48 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_shmwd: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_shmwd 17:48 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_shmwd', metastore_id=None, name='ucx_shmwd', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 17:49 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.QjX0/config.yml) doesn't exist. 17:49 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration 17:49 INFO [databricks.labs.ucx.installer.hms_lineage] HMS Lineage feature creates one system table named system.hms_to_uc_migration.table_access and helps in your migration process from HMS to UC by allowing you to programmatically query HMS lineage data. 17:49 INFO [databricks.labs.ucx.install] Fetching installations... 17:49 WARNING [databricks.labs.ucx.install] Existing installation at /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.QjX0 is corrupted. Skipping... 17:49 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy. 17:49 DEBUG [tests.integration.conftest] Waiting for clusters to start... 17:49 DEBUG [tests.integration.conftest] Waiting for clusters to start... 17:49 INFO [databricks.labs.ucx.install] Installing UCX v0.27.2+6020240702174951 17:49 INFO [databricks.labs.ucx.install] Creating ucx schemas... 17:50 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups 17:50 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=failing 17:50 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-tables-ctas 17:50 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables 17:50 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=experimental-workflow-linter 17:51 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=scan-tables-in-mounts-experimental 17:51 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-hiveserde-tables-in-place-experimental 17:51 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-experimental 17:51 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups 17:51 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment 17:51 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental 17:51 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions 17:51 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-data-reconciliation 17:51 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.QjX0/README for the next steps. 17:51 DEBUG [databricks.labs.ucx.installer.workflows] starting migrate-external-tables-ctas job: https://DATABRICKS_HOST#job/23952184139366 17:55 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 make_storage_dir fixtures 17:55 DEBUG [databricks.labs.ucx.mixins.fixtures] removing make_storage_dir fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_6t5ff 17:55 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 make_dbfs_data_copy fixtures 17:55 DEBUG [databricks.labs.ucx.mixins.fixtures] removing make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/YcOI 17:55 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 cluster fixtures 17:55 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 catalog fixtures 17:55 DEBUG [databricks.labs.ucx.mixins.fixtures] removing catalog fixture: CatalogInfo(browse_only=False, catalog_type=, comment='', connection_name=None, created_at=1719942509759, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cjahf', isolation_mode=, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_cjahf', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1719942509759, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d') 17:55 INFO [databricks.labs.ucx.install] Deleting UCX v0.27.2+6020240702174951 from https://DATABRICKS_HOST 17:55 INFO [databricks.labs.ucx.install] Deleting inventory database ucx_shmwd 17:55 INFO [databricks.labs.ucx.install] Deleting jobs 17:55 INFO [databricks.labs.ucx.install] Deleting migrate-groups job_id=428261278624795. 17:55 INFO [databricks.labs.ucx.install] Deleting failing job_id=374269735619002. 17:55 INFO [databricks.labs.ucx.install] Deleting migrate-external-tables-ctas job_id=23952184139366. 17:55 INFO [databricks.labs.ucx.install] Deleting migrate-tables job_id=759463619732757. 17:55 INFO [databricks.labs.ucx.install] Deleting experimental-workflow-linter job_id=109164610140258. 17:55 INFO [databricks.labs.ucx.install] Deleting scan-tables-in-mounts-experimental job_id=777018830777958. 17:55 INFO [databricks.labs.ucx.install] Deleting migrate-external-hiveserde-tables-in-place-experimental job_id=762485489857241. 17:55 INFO [databricks.labs.ucx.install] Deleting migrate-groups-experimental job_id=611377751327803. 17:55 INFO [databricks.labs.ucx.install] Deleting remove-workspace-local-backup-groups job_id=945700470938895. 17:55 INFO [databricks.labs.ucx.install] Deleting assessment job_id=36902091907465. 17:55 INFO [databricks.labs.ucx.install] Deleting migrate-tables-in-mounts-experimental job_id=312323251048590. 17:55 INFO [databricks.labs.ucx.install] Deleting validate-groups-permissions job_id=263479418244073. 17:55 INFO [databricks.labs.ucx.install] Deleting migrate-data-reconciliation job_id=163790819183564. 17:55 INFO [databricks.labs.ucx.install] Deleting cluster policy 17:55 INFO [databricks.labs.ucx.install] Deleting secret scope 17:55 INFO [databricks.labs.ucx.install] UnInstalling UCX complete 17:55 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 workspace user fixtures 17:55 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 account group fixtures 17:55 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 workspace group fixtures 17:55 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 table fixtures 17:55 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 3 table fixtures 17:55 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_6t5ff.parquet_serde_6t5ff', metastore_id=None, name='parquet_serde_6t5ff', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='hiveserde_in_place_6t5ff', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_6t5ff/parquet_serde_6t5ff', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 17:55 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_6t5ff.orc_serde_6t5ff', metastore_id=None, name='orc_serde_6t5ff', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='hiveserde_in_place_6t5ff', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_6t5ff/orc_serde_6t5ff', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 17:55 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_6t5ff.avro_serde_6t5ff', metastore_id=None, name='avro_serde_6t5ff', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='hiveserde_in_place_6t5ff', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_6t5ff/avro_serde_6t5ff', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 17:55 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 3 schema fixtures 17:55 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.hiveserde_in_place_6t5ff', metastore_id=None, name='hiveserde_in_place_6t5ff', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 17:55 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_cjahf', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cjahf.hiveserde_in_place_6t5ff', metastore_id=None, name='hiveserde_in_place_6t5ff', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 17:55 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_shmwd', metastore_id=None, name='ucx_shmwd', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) [gw0] linux -- Python 3.10.14 /home/runner/work/ucx/ucx/.venv/bin/python ```
❌ test_table_migration_job_refreshes_migration_status[regular-migrate-tables]: TimeoutError: Timed out after 0:05:00 (9m4.951s) ``` TimeoutError: Timed out after 0:05:00 17:51 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/zwjg 17:52 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.migrate_lilpe: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_lilpe 17:52 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.migrate_lilpe', metastore_id=None, name='migrate_lilpe', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 17:52 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_lilpe.ucx_tusd6: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_lilpe/ucx_tusd6 17:52 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_lilpe.ucx_tusd6', metastore_id=None, name='ucx_tusd6', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='migrate_lilpe', sql_path=None, storage_credential_name=None, storage_location='dbfs:/user/hive/warehouse/migrate_lilpe/ucx_tusd6', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 17:52 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_lilpe.ucx_todn9: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_lilpe/ucx_todn9 17:52 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_lilpe.ucx_todn9', metastore_id=None, name='ucx_todn9', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='migrate_lilpe', sql_path=None, storage_credential_name=None, storage_location='dbfs:/tmp/ucx_test_AUqi', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 17:52 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_lilpe.ucx_theos: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_lilpe/ucx_theos 17:52 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_lilpe.ucx_theos', metastore_id=None, name='ucx_theos', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='migrate_lilpe', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/b/zwjg', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 17:52 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_lilpe.ucx_ts43c: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_lilpe/ucx_ts43c 17:52 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_lilpe.ucx_ts43c', metastore_id=None, name='ucx_ts43c', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='migrate_lilpe', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_lilpe.ucx_tusd6', view_dependencies=None) 17:52 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_lilpe.ucx_tritn: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_lilpe/ucx_tritn 17:52 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_lilpe.ucx_tritn', metastore_id=None, name='ucx_tritn', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='migrate_lilpe', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_lilpe.ucx_ts43c', view_dependencies=None) 17:52 DEBUG [databricks.labs.ucx.mixins.fixtures] added catalog fixture: CatalogInfo(browse_only=False, catalog_type=, comment='', connection_name=None, created_at=1719942775359, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cxrhe', isolation_mode=, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_cxrhe', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1719942775359, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d') 17:53 INFO [databricks.labs.ucx.mixins.fixtures] Schema ucx_cxrhe.migrate_lilpe: https://DATABRICKS_HOST/explore/data/ucx_cxrhe/migrate_lilpe 17:53 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_cxrhe', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cxrhe.migrate_lilpe', metastore_id=None, name='migrate_lilpe', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 17:53 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_sffhx: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_sffhx 17:53 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_sffhx', metastore_id=None, name='ucx_sffhx', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) [gw1] linux -- Python 3.10.14 /home/runner/work/ucx/ucx/.venv/bin/python 17:51 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/zwjg 17:52 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.migrate_lilpe: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_lilpe 17:52 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.migrate_lilpe', metastore_id=None, name='migrate_lilpe', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 17:52 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_lilpe.ucx_tusd6: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_lilpe/ucx_tusd6 17:52 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_lilpe.ucx_tusd6', metastore_id=None, name='ucx_tusd6', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='migrate_lilpe', sql_path=None, storage_credential_name=None, storage_location='dbfs:/user/hive/warehouse/migrate_lilpe/ucx_tusd6', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 17:52 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_lilpe.ucx_todn9: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_lilpe/ucx_todn9 17:52 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_lilpe.ucx_todn9', metastore_id=None, name='ucx_todn9', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='migrate_lilpe', sql_path=None, storage_credential_name=None, storage_location='dbfs:/tmp/ucx_test_AUqi', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 17:52 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_lilpe.ucx_theos: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_lilpe/ucx_theos 17:52 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_lilpe.ucx_theos', metastore_id=None, name='ucx_theos', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='migrate_lilpe', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/b/zwjg', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 17:52 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_lilpe.ucx_ts43c: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_lilpe/ucx_ts43c 17:52 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_lilpe.ucx_ts43c', metastore_id=None, name='ucx_ts43c', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='migrate_lilpe', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_lilpe.ucx_tusd6', view_dependencies=None) 17:52 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_lilpe.ucx_tritn: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_lilpe/ucx_tritn 17:52 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_lilpe.ucx_tritn', metastore_id=None, name='ucx_tritn', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='migrate_lilpe', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_lilpe.ucx_ts43c', view_dependencies=None) 17:52 DEBUG [databricks.labs.ucx.mixins.fixtures] added catalog fixture: CatalogInfo(browse_only=False, catalog_type=, comment='', connection_name=None, created_at=1719942775359, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cxrhe', isolation_mode=, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_cxrhe', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1719942775359, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d') 17:53 INFO [databricks.labs.ucx.mixins.fixtures] Schema ucx_cxrhe.migrate_lilpe: https://DATABRICKS_HOST/explore/data/ucx_cxrhe/migrate_lilpe 17:53 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_cxrhe', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cxrhe.migrate_lilpe', metastore_id=None, name='migrate_lilpe', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 17:53 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_sffhx: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_sffhx 17:53 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_sffhx', metastore_id=None, name='ucx_sffhx', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 17:53 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.lj9n/config.yml) doesn't exist. 17:53 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration 17:53 INFO [databricks.labs.ucx.installer.hms_lineage] HMS Lineage feature creates one system table named system.hms_to_uc_migration.table_access and helps in your migration process from HMS to UC by allowing you to programmatically query HMS lineage data. 17:53 INFO [databricks.labs.ucx.install] Fetching installations... 17:54 WARNING [databricks.labs.ucx.install] Existing installation at /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.lj9n is corrupted. Skipping... 17:54 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy. 17:54 DEBUG [tests.integration.conftest] Waiting for clusters to start... 17:54 DEBUG [tests.integration.conftest] Waiting for clusters to start... 17:54 INFO [databricks.labs.ucx.install] Installing UCX v0.27.2+6020240702175405 17:54 INFO [databricks.labs.ucx.install] Creating ucx schemas... 17:55 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-hiveserde-tables-in-place-experimental 17:55 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups 17:55 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=experimental-workflow-linter 17:55 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-experimental 17:55 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=scan-tables-in-mounts-experimental 17:55 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental 17:55 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables 17:55 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions 17:55 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment 17:55 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-data-reconciliation 17:55 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups 17:55 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-tables-ctas 17:55 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=failing 17:55 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.lj9n/README for the next steps. 17:55 DEBUG [databricks.labs.ucx.installer.workflows] starting migrate-tables job: https://DATABRICKS_HOST#job/469321550771330 17:51 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/zwjg 17:52 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.migrate_lilpe: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_lilpe 17:52 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.migrate_lilpe', metastore_id=None, name='migrate_lilpe', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 17:52 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_lilpe.ucx_tusd6: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_lilpe/ucx_tusd6 17:52 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_lilpe.ucx_tusd6', metastore_id=None, name='ucx_tusd6', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='migrate_lilpe', sql_path=None, storage_credential_name=None, storage_location='dbfs:/user/hive/warehouse/migrate_lilpe/ucx_tusd6', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 17:52 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_lilpe.ucx_todn9: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_lilpe/ucx_todn9 17:52 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_lilpe.ucx_todn9', metastore_id=None, name='ucx_todn9', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='migrate_lilpe', sql_path=None, storage_credential_name=None, storage_location='dbfs:/tmp/ucx_test_AUqi', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 17:52 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_lilpe.ucx_theos: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_lilpe/ucx_theos 17:52 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_lilpe.ucx_theos', metastore_id=None, name='ucx_theos', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='migrate_lilpe', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/b/zwjg', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 17:52 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_lilpe.ucx_ts43c: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_lilpe/ucx_ts43c 17:52 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_lilpe.ucx_ts43c', metastore_id=None, name='ucx_ts43c', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='migrate_lilpe', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_lilpe.ucx_tusd6', view_dependencies=None) 17:52 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_lilpe.ucx_tritn: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_lilpe/ucx_tritn 17:52 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_lilpe.ucx_tritn', metastore_id=None, name='ucx_tritn', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='migrate_lilpe', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_lilpe.ucx_ts43c', view_dependencies=None) 17:52 DEBUG [databricks.labs.ucx.mixins.fixtures] added catalog fixture: CatalogInfo(browse_only=False, catalog_type=, comment='', connection_name=None, created_at=1719942775359, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cxrhe', isolation_mode=, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_cxrhe', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1719942775359, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d') 17:53 INFO [databricks.labs.ucx.mixins.fixtures] Schema ucx_cxrhe.migrate_lilpe: https://DATABRICKS_HOST/explore/data/ucx_cxrhe/migrate_lilpe 17:53 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_cxrhe', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cxrhe.migrate_lilpe', metastore_id=None, name='migrate_lilpe', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 17:53 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_sffhx: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_sffhx 17:53 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_sffhx', metastore_id=None, name='ucx_sffhx', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 17:53 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.lj9n/config.yml) doesn't exist. 17:53 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration 17:53 INFO [databricks.labs.ucx.installer.hms_lineage] HMS Lineage feature creates one system table named system.hms_to_uc_migration.table_access and helps in your migration process from HMS to UC by allowing you to programmatically query HMS lineage data. 17:53 INFO [databricks.labs.ucx.install] Fetching installations... 17:54 WARNING [databricks.labs.ucx.install] Existing installation at /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.lj9n is corrupted. Skipping... 17:54 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy. 17:54 DEBUG [tests.integration.conftest] Waiting for clusters to start... 17:54 DEBUG [tests.integration.conftest] Waiting for clusters to start... 17:54 INFO [databricks.labs.ucx.install] Installing UCX v0.27.2+6020240702175405 17:54 INFO [databricks.labs.ucx.install] Creating ucx schemas... 17:55 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-hiveserde-tables-in-place-experimental 17:55 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups 17:55 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=experimental-workflow-linter 17:55 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-experimental 17:55 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=scan-tables-in-mounts-experimental 17:55 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental 17:55 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables 17:55 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions 17:55 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment 17:55 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-data-reconciliation 17:55 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups 17:55 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-tables-ctas 17:55 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=failing 17:55 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.lj9n/README for the next steps. 17:55 DEBUG [databricks.labs.ucx.installer.workflows] starting migrate-tables job: https://DATABRICKS_HOST#job/469321550771330 17:59 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 make_storage_dir fixtures 17:59 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 make_dbfs_data_copy fixtures 17:59 DEBUG [databricks.labs.ucx.mixins.fixtures] removing make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/zwjg 17:59 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 cluster fixtures 17:59 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 catalog fixtures 17:59 DEBUG [databricks.labs.ucx.mixins.fixtures] removing catalog fixture: CatalogInfo(browse_only=False, catalog_type=, comment='', connection_name=None, created_at=1719942775359, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cxrhe', isolation_mode=, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_cxrhe', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1719942775359, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d') 17:59 INFO [databricks.labs.ucx.install] Deleting UCX v0.27.2+6020240702175405 from https://DATABRICKS_HOST 17:59 INFO [databricks.labs.ucx.install] Deleting inventory database ucx_sffhx 18:00 INFO [databricks.labs.ucx.install] Deleting jobs 18:00 INFO [databricks.labs.ucx.install] Deleting migrate-external-hiveserde-tables-in-place-experimental job_id=578732733126829. 18:00 INFO [databricks.labs.ucx.install] Deleting remove-workspace-local-backup-groups job_id=510314082423273. 18:00 INFO [databricks.labs.ucx.install] Deleting experimental-workflow-linter job_id=1049622057312608. 18:00 INFO [databricks.labs.ucx.install] Deleting migrate-groups-experimental job_id=514774481550798. 18:00 INFO [databricks.labs.ucx.install] Deleting scan-tables-in-mounts-experimental job_id=400838333319470. 18:00 INFO [databricks.labs.ucx.install] Deleting migrate-tables-in-mounts-experimental job_id=445405214863248. 18:00 INFO [databricks.labs.ucx.install] Deleting migrate-tables job_id=469321550771330. 18:00 INFO [databricks.labs.ucx.install] Deleting validate-groups-permissions job_id=756341847046412. 18:00 INFO [databricks.labs.ucx.install] Deleting assessment job_id=714797026487582. 18:00 INFO [databricks.labs.ucx.install] Deleting migrate-data-reconciliation job_id=697695757430281. 18:00 INFO [databricks.labs.ucx.install] Deleting migrate-groups job_id=882334689422919. 18:00 INFO [databricks.labs.ucx.install] Deleting migrate-external-tables-ctas job_id=660661617317079. 18:00 INFO [databricks.labs.ucx.install] Deleting failing job_id=105670801735570. 18:00 INFO [databricks.labs.ucx.install] Deleting cluster policy 18:00 INFO [databricks.labs.ucx.install] Deleting secret scope 18:00 INFO [databricks.labs.ucx.install] UnInstalling UCX complete 18:00 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 workspace user fixtures 18:00 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 account group fixtures 18:00 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 workspace group fixtures 18:00 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 table fixtures 18:00 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 5 table fixtures 18:00 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_lilpe.ucx_tusd6', metastore_id=None, name='ucx_tusd6', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='migrate_lilpe', sql_path=None, storage_credential_name=None, storage_location='dbfs:/user/hive/warehouse/migrate_lilpe/ucx_tusd6', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 18:00 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_lilpe.ucx_todn9', metastore_id=None, name='ucx_todn9', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='migrate_lilpe', sql_path=None, storage_credential_name=None, storage_location='dbfs:/tmp/ucx_test_AUqi', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 18:00 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_lilpe.ucx_theos', metastore_id=None, name='ucx_theos', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='migrate_lilpe', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/b/zwjg', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 18:00 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_lilpe.ucx_ts43c', metastore_id=None, name='ucx_ts43c', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='migrate_lilpe', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_lilpe.ucx_tusd6', view_dependencies=None) 18:00 DEBUG [databricks.labs.ucx.mixins.fixtures] ignoring error while table TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_lilpe.ucx_ts43c', metastore_id=None, name='ucx_ts43c', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='migrate_lilpe', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_lilpe.ucx_tusd6', view_dependencies=None) teardown: [WRONG_COMMAND_FOR_OBJECT_TYPE] The operation DROP TABLE requires a EXTERNAL or MANAGED. But hive_metastore.migrate_lilpe.ucx_ts43c is a VIEW. Use DROP VIEW instead. SQLSTATE: 42809 18:00 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_lilpe.ucx_tritn', metastore_id=None, name='ucx_tritn', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='migrate_lilpe', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_lilpe.ucx_ts43c', view_dependencies=None) 18:00 DEBUG [databricks.labs.ucx.mixins.fixtures] ignoring error while table TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_lilpe.ucx_tritn', metastore_id=None, name='ucx_tritn', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='migrate_lilpe', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_lilpe.ucx_ts43c', view_dependencies=None) teardown: [WRONG_COMMAND_FOR_OBJECT_TYPE] The operation DROP TABLE requires a EXTERNAL or MANAGED. But hive_metastore.migrate_lilpe.ucx_tritn is a VIEW. Use DROP VIEW instead. SQLSTATE: 42809 18:00 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 3 schema fixtures 18:00 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.migrate_lilpe', metastore_id=None, name='migrate_lilpe', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 18:00 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_cxrhe', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cxrhe.migrate_lilpe', metastore_id=None, name='migrate_lilpe', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 18:00 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_sffhx', metastore_id=None, name='ucx_sffhx', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) [gw1] linux -- Python 3.10.14 /home/runner/work/ucx/ucx/.venv/bin/python ```
❌ test_running_real_workflow_linter_job: TimeoutError: Timed out after 0:02:00 (6m57.857s) ``` TimeoutError: Timed out after 0:02:00 [gw3] linux -- Python 3.10.14 /home/runner/work/ucx/ucx/.venv/bin/python 17:56 DEBUG [databricks.labs.ucx.mixins.fixtures] added directory fixture: /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/sdk-Smuk 17:56 DEBUG [databricks.labs.ucx.mixins.fixtures] added notebook fixture: /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/sdk-Smuk/notebook.ipynb 17:56 INFO [databricks.labs.ucx.mixins.fixtures] Job: https://DATABRICKS_HOST#job/41906151760368 17:56 DEBUG [databricks.labs.ucx.mixins.fixtures] added job fixture: CreateResponse(job_id=41906151760368) 17:56 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_sa2px: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_sa2px 17:56 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_sa2px', metastore_id=None, name='ucx_sa2px', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 17:56 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.NOFK/config.yml) doesn't exist. 17:56 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration 17:56 INFO [databricks.labs.ucx.installer.hms_lineage] HMS Lineage feature creates one system table named system.hms_to_uc_migration.table_access and helps in your migration process from HMS to UC by allowing you to programmatically query HMS lineage data. 17:56 INFO [databricks.labs.ucx.install] Fetching installations... 17:57 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy. 17:57 DEBUG [tests.integration.conftest] Waiting for clusters to start... 17:57 DEBUG [tests.integration.conftest] Waiting for clusters to start... 17:57 INFO [databricks.labs.ucx.install] Installing UCX v0.27.2+6020240702175710 17:57 INFO [databricks.labs.ucx.install] Creating ucx schemas... 17:58 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-hiveserde-tables-in-place-experimental 17:58 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups 17:58 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions 17:58 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups 17:58 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=scan-tables-in-mounts-experimental 17:58 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-tables-ctas 17:58 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-experimental 17:58 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-data-reconciliation 17:58 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental 17:58 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=failing 17:58 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=experimental-workflow-linter 17:58 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment 17:58 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables 17:58 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.NOFK/README for the next steps. 17:58 DEBUG [databricks.labs.ucx.installer.workflows] starting experimental-workflow-linter job: https://DATABRICKS_HOST#job/918498142361115 18:02 DEBUG [databricks.labs.ucx.installer.workflows] Validating experimental-workflow-linter workflow: https://DATABRICKS_HOST#job/918498142361115 17:56 DEBUG [databricks.labs.ucx.mixins.fixtures] added directory fixture: /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/sdk-Smuk 17:56 DEBUG [databricks.labs.ucx.mixins.fixtures] added notebook fixture: /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/sdk-Smuk/notebook.ipynb 17:56 INFO [databricks.labs.ucx.mixins.fixtures] Job: https://DATABRICKS_HOST#job/41906151760368 17:56 DEBUG [databricks.labs.ucx.mixins.fixtures] added job fixture: CreateResponse(job_id=41906151760368) 17:56 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_sa2px: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_sa2px 17:56 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_sa2px', metastore_id=None, name='ucx_sa2px', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 17:56 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.NOFK/config.yml) doesn't exist. 17:56 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration 17:56 INFO [databricks.labs.ucx.installer.hms_lineage] HMS Lineage feature creates one system table named system.hms_to_uc_migration.table_access and helps in your migration process from HMS to UC by allowing you to programmatically query HMS lineage data. 17:56 INFO [databricks.labs.ucx.install] Fetching installations... 17:57 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy. 17:57 DEBUG [tests.integration.conftest] Waiting for clusters to start... 17:57 DEBUG [tests.integration.conftest] Waiting for clusters to start... 17:57 INFO [databricks.labs.ucx.install] Installing UCX v0.27.2+6020240702175710 17:57 INFO [databricks.labs.ucx.install] Creating ucx schemas... 17:58 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-hiveserde-tables-in-place-experimental 17:58 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups 17:58 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions 17:58 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups 17:58 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=scan-tables-in-mounts-experimental 17:58 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-tables-ctas 17:58 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-experimental 17:58 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-data-reconciliation 17:58 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental 17:58 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=failing 17:58 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=experimental-workflow-linter 17:58 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment 17:58 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables 17:58 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.NOFK/README for the next steps. 17:58 DEBUG [databricks.labs.ucx.installer.workflows] starting experimental-workflow-linter job: https://DATABRICKS_HOST#job/918498142361115 18:02 DEBUG [databricks.labs.ucx.installer.workflows] Validating experimental-workflow-linter workflow: https://DATABRICKS_HOST#job/918498142361115 18:02 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 job fixtures 18:02 DEBUG [databricks.labs.ucx.mixins.fixtures] removing job fixture: CreateResponse(job_id=41906151760368) 18:02 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 directory fixtures 18:02 DEBUG [databricks.labs.ucx.mixins.fixtures] removing directory fixture: /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/sdk-Smuk 18:02 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 notebook fixtures 18:02 DEBUG [databricks.labs.ucx.mixins.fixtures] removing notebook fixture: /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/sdk-Smuk/notebook.ipynb 18:02 DEBUG [databricks.labs.ucx.mixins.fixtures] ignoring error while notebook /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/sdk-Smuk/notebook.ipynb teardown: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/sdk-Smuk/notebook.ipynb) doesn't exist. 18:02 INFO [databricks.labs.ucx.install] Deleting UCX v0.27.2+6020240702175710 from https://DATABRICKS_HOST 18:02 INFO [databricks.labs.ucx.install] Deleting inventory database ucx_sa2px 18:02 INFO [databricks.labs.ucx.install] Deleting jobs 18:02 INFO [databricks.labs.ucx.install] Deleting migrate-external-hiveserde-tables-in-place-experimental job_id=353394773761480. 18:02 INFO [databricks.labs.ucx.install] Deleting migrate-groups job_id=176379954974343. 18:02 INFO [databricks.labs.ucx.install] Deleting validate-groups-permissions job_id=319264795755023. 18:02 INFO [databricks.labs.ucx.install] Deleting remove-workspace-local-backup-groups job_id=339744475483630. 18:02 INFO [databricks.labs.ucx.install] Deleting scan-tables-in-mounts-experimental job_id=163646298786054. 18:02 INFO [databricks.labs.ucx.install] Deleting migrate-external-tables-ctas job_id=1044887071305233. 18:02 INFO [databricks.labs.ucx.install] Deleting migrate-groups-experimental job_id=692669968642146. 18:02 INFO [databricks.labs.ucx.install] Deleting migrate-data-reconciliation job_id=674893043524535. 18:02 INFO [databricks.labs.ucx.install] Deleting migrate-tables-in-mounts-experimental job_id=881092475408172. 18:02 INFO [databricks.labs.ucx.install] Deleting failing job_id=1094722933137523. 18:02 INFO [databricks.labs.ucx.install] Deleting experimental-workflow-linter job_id=918498142361115. 18:02 INFO [databricks.labs.ucx.install] Deleting assessment job_id=899234107741021. 18:02 INFO [databricks.labs.ucx.install] Deleting migrate-tables job_id=945796353633556. 18:02 INFO [databricks.labs.ucx.install] Deleting cluster policy 18:02 INFO [databricks.labs.ucx.install] Deleting secret scope 18:02 INFO [databricks.labs.ucx.install] UnInstalling UCX complete 18:02 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 workspace user fixtures 18:02 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 account group fixtures 18:02 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 workspace group fixtures 18:02 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 table fixtures 18:02 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 table fixtures 18:02 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 schema fixtures 18:02 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_sa2px', metastore_id=None, name='ucx_sa2px', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) [gw3] linux -- Python 3.10.14 /home/runner/work/ucx/ucx/.venv/bin/python ```
❌ test_table_migration_job_refreshes_migration_status[hiveserde-migrate-external-hiveserde-tables-in-place-experimental]: TimeoutError: Timed out after 0:05:00 (10m29.637s) ``` TimeoutError: Timed out after 0:05:00 18:01 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/YvGn 18:01 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.hiveserde_in_place_rxmso: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_rxmso 18:01 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.hiveserde_in_place_rxmso', metastore_id=None, name='hiveserde_in_place_rxmso', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 18:01 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_storage_dir fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_rxmso 18:01 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_rxmso.parquet_serde_rxmso: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_rxmso/parquet_serde_rxmso 18:01 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_rxmso.parquet_serde_rxmso', metastore_id=None, name='parquet_serde_rxmso', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='hiveserde_in_place_rxmso', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_rxmso/parquet_serde_rxmso', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 18:01 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_rxmso.orc_serde_rxmso: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_rxmso/orc_serde_rxmso 18:01 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_rxmso.orc_serde_rxmso', metastore_id=None, name='orc_serde_rxmso', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='hiveserde_in_place_rxmso', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_rxmso/orc_serde_rxmso', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 18:01 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_rxmso.avro_serde_rxmso: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_rxmso/avro_serde_rxmso 18:01 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_rxmso.avro_serde_rxmso', metastore_id=None, name='avro_serde_rxmso', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='hiveserde_in_place_rxmso', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_rxmso/avro_serde_rxmso', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 18:01 DEBUG [databricks.labs.ucx.mixins.fixtures] added catalog fixture: CatalogInfo(browse_only=False, catalog_type=, comment='', connection_name=None, created_at=1719943290870, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_ciszo', isolation_mode=, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_ciszo', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1719943290870, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d') 18:01 INFO [databricks.labs.ucx.mixins.fixtures] Schema ucx_ciszo.hiveserde_in_place_rxmso: https://DATABRICKS_HOST/explore/data/ucx_ciszo/hiveserde_in_place_rxmso 18:01 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_ciszo', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_ciszo.hiveserde_in_place_rxmso', metastore_id=None, name='hiveserde_in_place_rxmso', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 18:01 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_sknuk: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_sknuk 18:01 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_sknuk', metastore_id=None, name='ucx_sknuk', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) [gw1] linux -- Python 3.10.14 /home/runner/work/ucx/ucx/.venv/bin/python 18:01 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/YvGn 18:01 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.hiveserde_in_place_rxmso: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_rxmso 18:01 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.hiveserde_in_place_rxmso', metastore_id=None, name='hiveserde_in_place_rxmso', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 18:01 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_storage_dir fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_rxmso 18:01 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_rxmso.parquet_serde_rxmso: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_rxmso/parquet_serde_rxmso 18:01 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_rxmso.parquet_serde_rxmso', metastore_id=None, name='parquet_serde_rxmso', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='hiveserde_in_place_rxmso', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_rxmso/parquet_serde_rxmso', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 18:01 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_rxmso.orc_serde_rxmso: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_rxmso/orc_serde_rxmso 18:01 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_rxmso.orc_serde_rxmso', metastore_id=None, name='orc_serde_rxmso', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='hiveserde_in_place_rxmso', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_rxmso/orc_serde_rxmso', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 18:01 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_rxmso.avro_serde_rxmso: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_rxmso/avro_serde_rxmso 18:01 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_rxmso.avro_serde_rxmso', metastore_id=None, name='avro_serde_rxmso', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='hiveserde_in_place_rxmso', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_rxmso/avro_serde_rxmso', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 18:01 DEBUG [databricks.labs.ucx.mixins.fixtures] added catalog fixture: CatalogInfo(browse_only=False, catalog_type=, comment='', connection_name=None, created_at=1719943290870, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_ciszo', isolation_mode=, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_ciszo', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1719943290870, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d') 18:01 INFO [databricks.labs.ucx.mixins.fixtures] Schema ucx_ciszo.hiveserde_in_place_rxmso: https://DATABRICKS_HOST/explore/data/ucx_ciszo/hiveserde_in_place_rxmso 18:01 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_ciszo', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_ciszo.hiveserde_in_place_rxmso', metastore_id=None, name='hiveserde_in_place_rxmso', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 18:01 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_sknuk: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_sknuk 18:01 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_sknuk', metastore_id=None, name='ucx_sknuk', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 18:02 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.l43z/config.yml) doesn't exist. 18:02 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration 18:02 INFO [databricks.labs.ucx.installer.hms_lineage] HMS Lineage feature creates one system table named system.hms_to_uc_migration.table_access and helps in your migration process from HMS to UC by allowing you to programmatically query HMS lineage data. 18:02 INFO [databricks.labs.ucx.install] Fetching installations... 18:03 WARNING [databricks.labs.ucx.install] Existing installation at /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.l43z is corrupted. Skipping... 18:03 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy. 18:03 DEBUG [tests.integration.conftest] Waiting for clusters to start... 18:03 DEBUG [tests.integration.conftest] Waiting for clusters to start... 18:03 INFO [databricks.labs.ucx.install] Installing UCX v0.27.2+6020240702180303 18:03 INFO [databricks.labs.ucx.install] Creating ucx schemas... 18:03 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-hiveserde-tables-in-place-experimental 18:03 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups 18:03 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=experimental-workflow-linter 18:03 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-experimental 18:03 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=scan-tables-in-mounts-experimental 18:03 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental 18:03 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables 18:03 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions 18:03 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment 18:03 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-data-reconciliation 18:03 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups 18:03 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-tables-ctas 18:03 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=failing 18:03 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.l43z/README for the next steps. 18:03 DEBUG [databricks.labs.ucx.installer.workflows] starting migrate-external-hiveserde-tables-in-place-experimental job: https://DATABRICKS_HOST#job/936377062826816 18:01 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/YvGn 18:01 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.hiveserde_in_place_rxmso: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_rxmso 18:01 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.hiveserde_in_place_rxmso', metastore_id=None, name='hiveserde_in_place_rxmso', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 18:01 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_storage_dir fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_rxmso 18:01 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_rxmso.parquet_serde_rxmso: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_rxmso/parquet_serde_rxmso 18:01 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_rxmso.parquet_serde_rxmso', metastore_id=None, name='parquet_serde_rxmso', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='hiveserde_in_place_rxmso', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_rxmso/parquet_serde_rxmso', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 18:01 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_rxmso.orc_serde_rxmso: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_rxmso/orc_serde_rxmso 18:01 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_rxmso.orc_serde_rxmso', metastore_id=None, name='orc_serde_rxmso', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='hiveserde_in_place_rxmso', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_rxmso/orc_serde_rxmso', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 18:01 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_rxmso.avro_serde_rxmso: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_rxmso/avro_serde_rxmso 18:01 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_rxmso.avro_serde_rxmso', metastore_id=None, name='avro_serde_rxmso', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='hiveserde_in_place_rxmso', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_rxmso/avro_serde_rxmso', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 18:01 DEBUG [databricks.labs.ucx.mixins.fixtures] added catalog fixture: CatalogInfo(browse_only=False, catalog_type=, comment='', connection_name=None, created_at=1719943290870, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_ciszo', isolation_mode=, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_ciszo', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1719943290870, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d') 18:01 INFO [databricks.labs.ucx.mixins.fixtures] Schema ucx_ciszo.hiveserde_in_place_rxmso: https://DATABRICKS_HOST/explore/data/ucx_ciszo/hiveserde_in_place_rxmso 18:01 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_ciszo', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_ciszo.hiveserde_in_place_rxmso', metastore_id=None, name='hiveserde_in_place_rxmso', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 18:01 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_sknuk: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_sknuk 18:01 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_sknuk', metastore_id=None, name='ucx_sknuk', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 18:02 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.l43z/config.yml) doesn't exist. 18:02 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration 18:02 INFO [databricks.labs.ucx.installer.hms_lineage] HMS Lineage feature creates one system table named system.hms_to_uc_migration.table_access and helps in your migration process from HMS to UC by allowing you to programmatically query HMS lineage data. 18:02 INFO [databricks.labs.ucx.install] Fetching installations... 18:03 WARNING [databricks.labs.ucx.install] Existing installation at /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.l43z is corrupted. Skipping... 18:03 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy. 18:03 DEBUG [tests.integration.conftest] Waiting for clusters to start... 18:03 DEBUG [tests.integration.conftest] Waiting for clusters to start... 18:03 INFO [databricks.labs.ucx.install] Installing UCX v0.27.2+6020240702180303 18:03 INFO [databricks.labs.ucx.install] Creating ucx schemas... 18:03 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-hiveserde-tables-in-place-experimental 18:03 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups 18:03 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=experimental-workflow-linter 18:03 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-experimental 18:03 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=scan-tables-in-mounts-experimental 18:03 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental 18:03 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables 18:03 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions 18:03 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment 18:03 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-data-reconciliation 18:03 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups 18:03 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-tables-ctas 18:03 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=failing 18:03 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.l43z/README for the next steps. 18:03 DEBUG [databricks.labs.ucx.installer.workflows] starting migrate-external-hiveserde-tables-in-place-experimental job: https://DATABRICKS_HOST#job/936377062826816 18:11 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 make_storage_dir fixtures 18:11 DEBUG [databricks.labs.ucx.mixins.fixtures] removing make_storage_dir fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_rxmso 18:11 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 make_dbfs_data_copy fixtures 18:11 DEBUG [databricks.labs.ucx.mixins.fixtures] removing make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/YvGn 18:11 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 cluster fixtures 18:11 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 catalog fixtures 18:11 DEBUG [databricks.labs.ucx.mixins.fixtures] removing catalog fixture: CatalogInfo(browse_only=False, catalog_type=, comment='', connection_name=None, created_at=1719943290870, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_ciszo', isolation_mode=, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_ciszo', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1719943290870, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d') 18:11 INFO [databricks.labs.ucx.install] Deleting UCX v0.27.2+6020240702180303 from https://DATABRICKS_HOST 18:11 INFO [databricks.labs.ucx.install] Deleting inventory database ucx_sknuk 18:11 INFO [databricks.labs.ucx.install] Deleting jobs 18:11 INFO [databricks.labs.ucx.install] Deleting migrate-external-hiveserde-tables-in-place-experimental job_id=936377062826816. 18:11 INFO [databricks.labs.ucx.install] Deleting remove-workspace-local-backup-groups job_id=896243713770722. 18:11 INFO [databricks.labs.ucx.install] Deleting experimental-workflow-linter job_id=332859419960259. 18:11 INFO [databricks.labs.ucx.install] Deleting migrate-groups-experimental job_id=73011778950569. 18:11 INFO [databricks.labs.ucx.install] Deleting scan-tables-in-mounts-experimental job_id=1010257182526392. 18:11 INFO [databricks.labs.ucx.install] Deleting migrate-tables-in-mounts-experimental job_id=242457612040712. 18:11 INFO [databricks.labs.ucx.install] Deleting migrate-tables job_id=358527910493504. 18:11 INFO [databricks.labs.ucx.install] Deleting validate-groups-permissions job_id=59441545773670. 18:11 INFO [databricks.labs.ucx.install] Deleting assessment job_id=652574452839421. 18:11 INFO [databricks.labs.ucx.install] Deleting migrate-data-reconciliation job_id=1060506481917017. 18:11 INFO [databricks.labs.ucx.install] Deleting migrate-groups job_id=1035974751424557. 18:11 INFO [databricks.labs.ucx.install] Deleting migrate-external-tables-ctas job_id=538449821533752. 18:11 INFO [databricks.labs.ucx.install] Deleting failing job_id=695396212480885. 18:11 INFO [databricks.labs.ucx.install] Deleting cluster policy 18:11 INFO [databricks.labs.ucx.install] Deleting secret scope 18:11 INFO [databricks.labs.ucx.install] UnInstalling UCX complete 18:11 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 workspace user fixtures 18:11 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 account group fixtures 18:11 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 workspace group fixtures 18:11 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 table fixtures 18:11 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 3 table fixtures 18:11 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_rxmso.parquet_serde_rxmso', metastore_id=None, name='parquet_serde_rxmso', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='hiveserde_in_place_rxmso', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_rxmso/parquet_serde_rxmso', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 18:11 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_rxmso.orc_serde_rxmso', metastore_id=None, name='orc_serde_rxmso', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='hiveserde_in_place_rxmso', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_rxmso/orc_serde_rxmso', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 18:11 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_rxmso.avro_serde_rxmso', metastore_id=None, name='avro_serde_rxmso', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='hiveserde_in_place_rxmso', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_rxmso/avro_serde_rxmso', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 18:11 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 3 schema fixtures 18:11 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.hiveserde_in_place_rxmso', metastore_id=None, name='hiveserde_in_place_rxmso', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 18:11 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_ciszo', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_ciszo.hiveserde_in_place_rxmso', metastore_id=None, name='hiveserde_in_place_rxmso', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 18:11 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_sknuk', metastore_id=None, name='ucx_sknuk', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) [gw1] linux -- Python 3.10.14 /home/runner/work/ucx/ucx/.venv/bin/python ```
❌ test_table_migration_job_cluster_override[regular]: AssertionError: assert False (6m8.853s) ``` AssertionError: assert False + where False = >('migrate-tables') + where > = .validate_step + where = .deployed_workflows 18:19 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/JV3K 18:19 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.migrate_cikci: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_cikci 18:19 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.migrate_cikci', metastore_id=None, name='migrate_cikci', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 18:19 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_cikci.ucx_tac6y: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_cikci/ucx_tac6y 18:19 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_cikci.ucx_tac6y', metastore_id=None, name='ucx_tac6y', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='migrate_cikci', sql_path=None, storage_credential_name=None, storage_location='dbfs:/user/hive/warehouse/migrate_cikci/ucx_tac6y', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 18:19 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_cikci.ucx_tegts: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_cikci/ucx_tegts 18:19 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_cikci.ucx_tegts', metastore_id=None, name='ucx_tegts', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='migrate_cikci', sql_path=None, storage_credential_name=None, storage_location='dbfs:/tmp/ucx_test_RLdc', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 18:19 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_cikci.ucx_tbiyh: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_cikci/ucx_tbiyh 18:19 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_cikci.ucx_tbiyh', metastore_id=None, name='ucx_tbiyh', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='migrate_cikci', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/b/JV3K', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 18:19 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_cikci.ucx_tqvyi: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_cikci/ucx_tqvyi 18:19 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_cikci.ucx_tqvyi', metastore_id=None, name='ucx_tqvyi', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='migrate_cikci', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_cikci.ucx_tac6y', view_dependencies=None) 18:19 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_cikci.ucx_tssvh: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_cikci/ucx_tssvh 18:19 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_cikci.ucx_tssvh', metastore_id=None, name='ucx_tssvh', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='migrate_cikci', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_cikci.ucx_tqvyi', view_dependencies=None) 18:19 DEBUG [databricks.labs.ucx.mixins.fixtures] added catalog fixture: CatalogInfo(browse_only=False, catalog_type=, comment='', connection_name=None, created_at=1719944390785, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cuiyo', isolation_mode=, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_cuiyo', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1719944390785, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d') 18:19 INFO [databricks.labs.ucx.mixins.fixtures] Schema ucx_cuiyo.migrate_cikci: https://DATABRICKS_HOST/explore/data/ucx_cuiyo/migrate_cikci 18:19 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_cuiyo', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cuiyo.migrate_cikci', metastore_id=None, name='migrate_cikci', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 18:19 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_sih7u: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_sih7u 18:19 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_sih7u', metastore_id=None, name='ucx_sih7u', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) [gw9] linux -- Python 3.10.14 /home/runner/work/ucx/ucx/.venv/bin/python 18:19 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/JV3K 18:19 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.migrate_cikci: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_cikci 18:19 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.migrate_cikci', metastore_id=None, name='migrate_cikci', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 18:19 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_cikci.ucx_tac6y: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_cikci/ucx_tac6y 18:19 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_cikci.ucx_tac6y', metastore_id=None, name='ucx_tac6y', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='migrate_cikci', sql_path=None, storage_credential_name=None, storage_location='dbfs:/user/hive/warehouse/migrate_cikci/ucx_tac6y', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 18:19 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_cikci.ucx_tegts: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_cikci/ucx_tegts 18:19 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_cikci.ucx_tegts', metastore_id=None, name='ucx_tegts', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='migrate_cikci', sql_path=None, storage_credential_name=None, storage_location='dbfs:/tmp/ucx_test_RLdc', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 18:19 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_cikci.ucx_tbiyh: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_cikci/ucx_tbiyh 18:19 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_cikci.ucx_tbiyh', metastore_id=None, name='ucx_tbiyh', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='migrate_cikci', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/b/JV3K', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 18:19 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_cikci.ucx_tqvyi: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_cikci/ucx_tqvyi 18:19 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_cikci.ucx_tqvyi', metastore_id=None, name='ucx_tqvyi', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='migrate_cikci', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_cikci.ucx_tac6y', view_dependencies=None) 18:19 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_cikci.ucx_tssvh: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_cikci/ucx_tssvh 18:19 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_cikci.ucx_tssvh', metastore_id=None, name='ucx_tssvh', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='migrate_cikci', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_cikci.ucx_tqvyi', view_dependencies=None) 18:19 DEBUG [databricks.labs.ucx.mixins.fixtures] added catalog fixture: CatalogInfo(browse_only=False, catalog_type=, comment='', connection_name=None, created_at=1719944390785, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cuiyo', isolation_mode=, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_cuiyo', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1719944390785, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d') 18:19 INFO [databricks.labs.ucx.mixins.fixtures] Schema ucx_cuiyo.migrate_cikci: https://DATABRICKS_HOST/explore/data/ucx_cuiyo/migrate_cikci 18:19 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_cuiyo', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cuiyo.migrate_cikci', metastore_id=None, name='migrate_cikci', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 18:19 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_sih7u: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_sih7u 18:19 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_sih7u', metastore_id=None, name='ucx_sih7u', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 18:20 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.bmmE/config.yml) doesn't exist. 18:20 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration 18:20 INFO [databricks.labs.ucx.installer.hms_lineage] HMS Lineage feature creates one system table named system.hms_to_uc_migration.table_access and helps in your migration process from HMS to UC by allowing you to programmatically query HMS lineage data. 18:20 INFO [databricks.labs.ucx.install] Fetching installations... 18:20 WARNING [databricks.labs.ucx.install] Existing installation at /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.bmmE is corrupted. Skipping... 18:20 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy. 18:20 DEBUG [tests.integration.conftest] Waiting for clusters to start... 18:20 DEBUG [tests.integration.conftest] Waiting for clusters to start... 18:20 INFO [databricks.labs.ucx.install] Installing UCX v0.27.2+6020240702182025 18:20 INFO [databricks.labs.ucx.install] Creating ucx schemas... 18:20 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=failing 18:20 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-experimental 18:20 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-data-reconciliation 18:20 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-hiveserde-tables-in-place-experimental 18:20 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups 18:20 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables 18:20 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=experimental-workflow-linter 18:20 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions 18:20 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment 18:20 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-tables-ctas 18:20 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental 18:20 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=scan-tables-in-mounts-experimental 18:20 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups 18:20 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.bmmE/README for the next steps. 18:20 DEBUG [databricks.labs.ucx.installer.workflows] starting migrate-tables job: https://DATABRICKS_HOST#job/869591948613105 18:25 DEBUG [databricks.labs.ucx.installer.workflows] Validating migrate-tables workflow: https://DATABRICKS_HOST#job/869591948613105 18:19 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/JV3K 18:19 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.migrate_cikci: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_cikci 18:19 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.migrate_cikci', metastore_id=None, name='migrate_cikci', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 18:19 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_cikci.ucx_tac6y: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_cikci/ucx_tac6y 18:19 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_cikci.ucx_tac6y', metastore_id=None, name='ucx_tac6y', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='migrate_cikci', sql_path=None, storage_credential_name=None, storage_location='dbfs:/user/hive/warehouse/migrate_cikci/ucx_tac6y', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 18:19 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_cikci.ucx_tegts: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_cikci/ucx_tegts 18:19 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_cikci.ucx_tegts', metastore_id=None, name='ucx_tegts', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='migrate_cikci', sql_path=None, storage_credential_name=None, storage_location='dbfs:/tmp/ucx_test_RLdc', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 18:19 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_cikci.ucx_tbiyh: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_cikci/ucx_tbiyh 18:19 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_cikci.ucx_tbiyh', metastore_id=None, name='ucx_tbiyh', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='migrate_cikci', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/b/JV3K', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 18:19 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_cikci.ucx_tqvyi: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_cikci/ucx_tqvyi 18:19 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_cikci.ucx_tqvyi', metastore_id=None, name='ucx_tqvyi', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='migrate_cikci', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_cikci.ucx_tac6y', view_dependencies=None) 18:19 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_cikci.ucx_tssvh: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_cikci/ucx_tssvh 18:19 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_cikci.ucx_tssvh', metastore_id=None, name='ucx_tssvh', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='migrate_cikci', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_cikci.ucx_tqvyi', view_dependencies=None) 18:19 DEBUG [databricks.labs.ucx.mixins.fixtures] added catalog fixture: CatalogInfo(browse_only=False, catalog_type=, comment='', connection_name=None, created_at=1719944390785, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cuiyo', isolation_mode=, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_cuiyo', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1719944390785, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d') 18:19 INFO [databricks.labs.ucx.mixins.fixtures] Schema ucx_cuiyo.migrate_cikci: https://DATABRICKS_HOST/explore/data/ucx_cuiyo/migrate_cikci 18:19 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_cuiyo', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cuiyo.migrate_cikci', metastore_id=None, name='migrate_cikci', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 18:19 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_sih7u: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_sih7u 18:19 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_sih7u', metastore_id=None, name='ucx_sih7u', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 18:20 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.bmmE/config.yml) doesn't exist. 18:20 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration 18:20 INFO [databricks.labs.ucx.installer.hms_lineage] HMS Lineage feature creates one system table named system.hms_to_uc_migration.table_access and helps in your migration process from HMS to UC by allowing you to programmatically query HMS lineage data. 18:20 INFO [databricks.labs.ucx.install] Fetching installations... 18:20 WARNING [databricks.labs.ucx.install] Existing installation at /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.bmmE is corrupted. Skipping... 18:20 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy. 18:20 DEBUG [tests.integration.conftest] Waiting for clusters to start... 18:20 DEBUG [tests.integration.conftest] Waiting for clusters to start... 18:20 INFO [databricks.labs.ucx.install] Installing UCX v0.27.2+6020240702182025 18:20 INFO [databricks.labs.ucx.install] Creating ucx schemas... 18:20 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=failing 18:20 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-experimental 18:20 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-data-reconciliation 18:20 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-hiveserde-tables-in-place-experimental 18:20 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups 18:20 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables 18:20 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=experimental-workflow-linter 18:20 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions 18:20 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment 18:20 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-tables-ctas 18:20 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental 18:20 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=scan-tables-in-mounts-experimental 18:20 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups 18:20 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.bmmE/README for the next steps. 18:20 DEBUG [databricks.labs.ucx.installer.workflows] starting migrate-tables job: https://DATABRICKS_HOST#job/869591948613105 18:25 DEBUG [databricks.labs.ucx.installer.workflows] Validating migrate-tables workflow: https://DATABRICKS_HOST#job/869591948613105 18:25 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 make_storage_dir fixtures 18:25 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 make_dbfs_data_copy fixtures 18:25 DEBUG [databricks.labs.ucx.mixins.fixtures] removing make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/JV3K 18:25 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 cluster fixtures 18:25 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 catalog fixtures 18:25 DEBUG [databricks.labs.ucx.mixins.fixtures] removing catalog fixture: CatalogInfo(browse_only=False, catalog_type=, comment='', connection_name=None, created_at=1719944390785, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cuiyo', isolation_mode=, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_cuiyo', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1719944390785, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d') 18:25 INFO [databricks.labs.ucx.install] Deleting UCX v0.27.2+6020240702182025 from https://DATABRICKS_HOST 18:25 INFO [databricks.labs.ucx.install] Deleting inventory database ucx_sih7u 18:25 INFO [databricks.labs.ucx.install] Deleting jobs 18:25 INFO [databricks.labs.ucx.install] Deleting failing job_id=485216354710322. 18:25 INFO [databricks.labs.ucx.install] Deleting migrate-groups-experimental job_id=836398955774295. 18:25 INFO [databricks.labs.ucx.install] Deleting migrate-data-reconciliation job_id=309143775870978. 18:25 INFO [databricks.labs.ucx.install] Deleting migrate-external-hiveserde-tables-in-place-experimental job_id=765475466471170. 18:25 INFO [databricks.labs.ucx.install] Deleting migrate-groups job_id=531113669351343. 18:25 INFO [databricks.labs.ucx.install] Deleting migrate-tables job_id=869591948613105. 18:25 INFO [databricks.labs.ucx.install] Deleting experimental-workflow-linter job_id=279448986385349. 18:25 INFO [databricks.labs.ucx.install] Deleting validate-groups-permissions job_id=41112943037754. 18:25 INFO [databricks.labs.ucx.install] Deleting assessment job_id=858432348075096. 18:25 INFO [databricks.labs.ucx.install] Deleting migrate-external-tables-ctas job_id=213590436193312. 18:25 INFO [databricks.labs.ucx.install] Deleting migrate-tables-in-mounts-experimental job_id=494914195688628. 18:25 INFO [databricks.labs.ucx.install] Deleting scan-tables-in-mounts-experimental job_id=311628817593829. 18:25 INFO [databricks.labs.ucx.install] Deleting remove-workspace-local-backup-groups job_id=655333916242194. 18:25 INFO [databricks.labs.ucx.install] Deleting cluster policy 18:25 INFO [databricks.labs.ucx.install] Deleting secret scope 18:25 INFO [databricks.labs.ucx.install] UnInstalling UCX complete 18:25 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 workspace user fixtures 18:25 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 account group fixtures 18:25 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 workspace group fixtures 18:25 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 table fixtures 18:25 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 5 table fixtures 18:25 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_cikci.ucx_tac6y', metastore_id=None, name='ucx_tac6y', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='migrate_cikci', sql_path=None, storage_credential_name=None, storage_location='dbfs:/user/hive/warehouse/migrate_cikci/ucx_tac6y', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 18:25 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_cikci.ucx_tegts', metastore_id=None, name='ucx_tegts', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='migrate_cikci', sql_path=None, storage_credential_name=None, storage_location='dbfs:/tmp/ucx_test_RLdc', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 18:25 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_cikci.ucx_tbiyh', metastore_id=None, name='ucx_tbiyh', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='migrate_cikci', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/b/JV3K', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 18:25 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_cikci.ucx_tqvyi', metastore_id=None, name='ucx_tqvyi', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='migrate_cikci', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_cikci.ucx_tac6y', view_dependencies=None) 18:25 DEBUG [databricks.labs.ucx.mixins.fixtures] ignoring error while table TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_cikci.ucx_tqvyi', metastore_id=None, name='ucx_tqvyi', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='migrate_cikci', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_cikci.ucx_tac6y', view_dependencies=None) teardown: [WRONG_COMMAND_FOR_OBJECT_TYPE] The operation DROP TABLE requires a EXTERNAL or MANAGED. But hive_metastore.migrate_cikci.ucx_tqvyi is a VIEW. Use DROP VIEW instead. SQLSTATE: 42809 18:25 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_cikci.ucx_tssvh', metastore_id=None, name='ucx_tssvh', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='migrate_cikci', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_cikci.ucx_tqvyi', view_dependencies=None) 18:25 DEBUG [databricks.labs.ucx.mixins.fixtures] ignoring error while table TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_cikci.ucx_tssvh', metastore_id=None, name='ucx_tssvh', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='migrate_cikci', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_cikci.ucx_tqvyi', view_dependencies=None) teardown: [WRONG_COMMAND_FOR_OBJECT_TYPE] The operation DROP TABLE requires a EXTERNAL or MANAGED. But hive_metastore.migrate_cikci.ucx_tssvh is a VIEW. Use DROP VIEW instead. SQLSTATE: 42809 18:25 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 3 schema fixtures 18:25 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.migrate_cikci', metastore_id=None, name='migrate_cikci', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 18:25 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_cuiyo', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cuiyo.migrate_cikci', metastore_id=None, name='migrate_cikci', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 18:25 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_sih7u', metastore_id=None, name='ucx_sih7u', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) [gw9] linux -- Python 3.10.14 /home/runner/work/ucx/ucx/.venv/bin/python ```

Running from acceptance #4210