search

databrickslabs / ucx

Your best companion for upgrading to Unity Catalog. UCX will guide you, the Databricks customer, through the process of upgrading your account, groups, workspaces, jobs etc. to Unity Catalog.
Other
196 stars 70 forks source link

whitelist kaleido #2066

Closed ericvergnaud closed 1 week ago

ericvergnaud commented 1 week ago

Changes

whitelist kaleido

Linked issues

Progresses #1901

Functionality

None

Tests

github-actions[bot] commented 1 week ago

❌ 93/98 passed, 5 failed, 18 skipped, 5h54m36s total

❌ test_group: TimeoutError: Timed out after 0:00:30 (33.462s) ``` TimeoutError: Timed out after 0:00:30 [gw7] linux -- Python 3.10.14 /home/runner/work/ucx/ucx/.venv/bin/python 11:45 DEBUG [databricks.labs.ucx.mixins.fixtures] added workspace user fixture: User(active=True, display_name='sdk-ggo2m2ds@example.com', emails=[ComplexValue(display=None, primary=True, ref=None, type='work', value='sdk-ggo2m2ds@example.com')], entitlements=[], external_id=None, groups=[], id='6344317941009701', name=Name(family_name=None, given_name='sdk-ggo2m2ds@example.com'), roles=[], schemas=[, ], user_name='sdk-ggo2m2ds@example.com') 11:45 DEBUG [databricks.labs.ucx.mixins.fixtures] added workspace user fixture: User(active=True, display_name='sdk-ggo2m2ds@example.com', emails=[ComplexValue(display=None, primary=True, ref=None, type='work', value='sdk-ggo2m2ds@example.com')], entitlements=[], external_id=None, groups=[], id='6344317941009701', name=Name(family_name=None, given_name='sdk-ggo2m2ds@example.com'), roles=[], schemas=[, ], user_name='sdk-ggo2m2ds@example.com') 11:46 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 workspace user fixtures 11:46 DEBUG [databricks.labs.ucx.mixins.fixtures] removing workspace user fixture: User(active=True, display_name='sdk-ggo2m2ds@example.com', emails=[ComplexValue(display=None, primary=True, ref=None, type='work', value='sdk-ggo2m2ds@example.com')], entitlements=[], external_id=None, groups=[], id='6344317941009701', name=Name(family_name=None, given_name='sdk-ggo2m2ds@example.com'), roles=[], schemas=[, ], user_name='sdk-ggo2m2ds@example.com') 11:46 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 workspace group fixtures [gw7] linux -- Python 3.10.14 /home/runner/work/ucx/ucx/.venv/bin/python ```
❌ test_run_external_locations_using_access_connector: databricks.sdk.errors.platform.ResourceDoesNotExist: The specified pipeline 081a359d-59e4-4aa2-9665-1882eef44b05 was not found. (1m30.18s) ``` databricks.sdk.errors.platform.ResourceDoesNotExist: The specified pipeline 081a359d-59e4-4aa2-9665-1882eef44b05 was not found. [gw8] linux -- Python 3.10.14 /home/runner/work/ucx/ucx/.venv/bin/python 11:47 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/hMwo 11:47 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_sctwv: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_sctwv 11:47 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_sctwv', metastore_id=None, name='ucx_sctwv', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 11:48 WARNING [databricks.labs.ucx.CLOUD_ENV.credentials] There is an existing external location overlaps with the prefix that is mapped to the service principal and used for validating the migrated storage credential. Skip the validation 11:48 INFO [databricks.labs.ucx.CLOUD_ENV.credentials] Found 1 distinct service principals already used in UC storage credentials 11:48 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.ucx_sctwv.CLOUD_ENV_service_principals] fetching CLOUD_ENV_service_principals inventory 11:48 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.ucx_sctwv.CLOUD_ENV_service_principals] crawling new batch for CLOUD_ENV_service_principals 11:48 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 11:48 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 11:48 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 11:48 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 11:48 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 11:48 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 11:48 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 11:48 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 11:48 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 11:48 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 11:48 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 11:48 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 11:48 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 11:48 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 11:48 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 11:48 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 11:48 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 11:48 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 11:48 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 11:48 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 11:48 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 11:48 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 11:48 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 11:48 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 11:48 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 11:48 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 11:48 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 11:48 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 11:48 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 11:48 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 11:48 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 11:48 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 11:48 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 11:48 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 11:48 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 11:48 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 11:48 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 11:48 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 11:48 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 11:48 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 11:48 WARNING [databricks.labs.ucx.assessment.crawlers] The cluster policy was deleted: 0009EE2EF8857376 11:48 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 11:48 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 11:48 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 11:48 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 11:48 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 11:48 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 11:48 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 11:48 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 11:48 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 11:48 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 11:48 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 11:47 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/hMwo 11:47 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_sctwv: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_sctwv 11:47 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_sctwv', metastore_id=None, name='ucx_sctwv', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 11:48 WARNING [databricks.labs.ucx.CLOUD_ENV.credentials] There is an existing external location overlaps with the prefix that is mapped to the service principal and used for validating the migrated storage credential. Skip the validation 11:48 INFO [databricks.labs.ucx.CLOUD_ENV.credentials] Found 1 distinct service principals already used in UC storage credentials 11:48 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.ucx_sctwv.CLOUD_ENV_service_principals] fetching CLOUD_ENV_service_principals inventory 11:48 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.ucx_sctwv.CLOUD_ENV_service_principals] crawling new batch for CLOUD_ENV_service_principals 11:48 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 11:48 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 11:48 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 11:48 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 11:48 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 11:48 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 11:48 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 11:48 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 11:48 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 11:48 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 11:48 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 11:48 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 11:48 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 11:48 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 11:48 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 11:48 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 11:48 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 11:48 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 11:48 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 11:48 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 11:48 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 11:48 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 11:48 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 11:48 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 11:48 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 11:48 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 11:48 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 11:48 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 11:48 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 11:48 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 11:48 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 11:48 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 11:48 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 11:48 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 11:48 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 11:48 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 11:48 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 11:48 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 11:48 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 11:48 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 11:48 WARNING [databricks.labs.ucx.assessment.crawlers] The cluster policy was deleted: 0009EE2EF8857376 11:48 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 11:48 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 11:48 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 11:48 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 11:48 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 11:48 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 11:48 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 11:48 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 11:48 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 11:48 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 11:48 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 11:48 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 make_dbfs_data_copy fixtures 11:48 DEBUG [databricks.labs.ucx.mixins.fixtures] removing make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/hMwo 11:48 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 cluster fixtures 11:48 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 schema fixtures 11:48 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_sctwv', metastore_id=None, name='ucx_sctwv', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) [gw8] linux -- Python 3.10.14 /home/runner/work/ucx/ucx/.venv/bin/python ```
❌ test_table_migration_job_refreshes_migration_status[hiveserde-migrate-external-tables-ctas]: TimeoutError: Timed out after 0:05:00 (12m53.426s) ``` TimeoutError: Timed out after 0:05:00 11:54 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/cOON 11:54 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.hiveserde_in_place_e4ery: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_e4ery 11:54 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.hiveserde_in_place_e4ery', metastore_id=None, name='hiveserde_in_place_e4ery', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 11:54 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_storage_dir fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_e4ery 11:54 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_e4ery.parquet_serde_e4ery: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_e4ery/parquet_serde_e4ery 11:54 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_e4ery.parquet_serde_e4ery', metastore_id=None, name='parquet_serde_e4ery', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070313'}, row_filter=None, schema_name='hiveserde_in_place_e4ery', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_e4ery/parquet_serde_e4ery', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 11:55 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_e4ery.orc_serde_e4ery: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_e4ery/orc_serde_e4ery 11:55 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_e4ery.orc_serde_e4ery', metastore_id=None, name='orc_serde_e4ery', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070313'}, row_filter=None, schema_name='hiveserde_in_place_e4ery', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_e4ery/orc_serde_e4ery', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 11:55 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_e4ery.avro_serde_e4ery: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_e4ery/avro_serde_e4ery 11:55 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_e4ery.avro_serde_e4ery', metastore_id=None, name='avro_serde_e4ery', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070313'}, row_filter=None, schema_name='hiveserde_in_place_e4ery', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_e4ery/avro_serde_e4ery', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 11:55 DEBUG [databricks.labs.ucx.mixins.fixtures] added catalog fixture: CatalogInfo(browse_only=False, catalog_type=, comment='', connection_name=None, created_at=1720007729713, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cxpis', isolation_mode=, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_cxpis', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1720007729713, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d') 11:55 INFO [databricks.labs.ucx.mixins.fixtures] Schema ucx_cxpis.hiveserde_in_place_e4ery: https://DATABRICKS_HOST/explore/data/ucx_cxpis/hiveserde_in_place_e4ery 11:55 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_cxpis', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cxpis.hiveserde_in_place_e4ery', metastore_id=None, name='hiveserde_in_place_e4ery', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 11:55 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_s5tqb: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_s5tqb 11:55 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_s5tqb', metastore_id=None, name='ucx_s5tqb', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) [gw4] linux -- Python 3.10.14 /home/runner/work/ucx/ucx/.venv/bin/python 11:54 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/cOON 11:54 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.hiveserde_in_place_e4ery: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_e4ery 11:54 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.hiveserde_in_place_e4ery', metastore_id=None, name='hiveserde_in_place_e4ery', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 11:54 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_storage_dir fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_e4ery 11:54 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_e4ery.parquet_serde_e4ery: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_e4ery/parquet_serde_e4ery 11:54 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_e4ery.parquet_serde_e4ery', metastore_id=None, name='parquet_serde_e4ery', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070313'}, row_filter=None, schema_name='hiveserde_in_place_e4ery', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_e4ery/parquet_serde_e4ery', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 11:55 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_e4ery.orc_serde_e4ery: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_e4ery/orc_serde_e4ery 11:55 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_e4ery.orc_serde_e4ery', metastore_id=None, name='orc_serde_e4ery', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070313'}, row_filter=None, schema_name='hiveserde_in_place_e4ery', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_e4ery/orc_serde_e4ery', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 11:55 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_e4ery.avro_serde_e4ery: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_e4ery/avro_serde_e4ery 11:55 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_e4ery.avro_serde_e4ery', metastore_id=None, name='avro_serde_e4ery', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070313'}, row_filter=None, schema_name='hiveserde_in_place_e4ery', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_e4ery/avro_serde_e4ery', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 11:55 DEBUG [databricks.labs.ucx.mixins.fixtures] added catalog fixture: CatalogInfo(browse_only=False, catalog_type=, comment='', connection_name=None, created_at=1720007729713, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cxpis', isolation_mode=, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_cxpis', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1720007729713, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d') 11:55 INFO [databricks.labs.ucx.mixins.fixtures] Schema ucx_cxpis.hiveserde_in_place_e4ery: https://DATABRICKS_HOST/explore/data/ucx_cxpis/hiveserde_in_place_e4ery 11:55 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_cxpis', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cxpis.hiveserde_in_place_e4ery', metastore_id=None, name='hiveserde_in_place_e4ery', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 11:55 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_s5tqb: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_s5tqb 11:55 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_s5tqb', metastore_id=None, name='ucx_s5tqb', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 11:58 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.aPxN/config.yml) doesn't exist. 11:58 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration 11:58 INFO [databricks.labs.ucx.installer.hms_lineage] HMS Lineage feature creates one system table named system.hms_to_uc_migration.table_access and helps in your migration process from HMS to UC by allowing you to programmatically query HMS lineage data. 11:58 INFO [databricks.labs.ucx.install] Fetching installations... 11:58 WARNING [databricks.labs.ucx.install] Existing installation at /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.aPxN is corrupted. Skipping... 11:58 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy. 11:58 DEBUG [tests.integration.conftest] Waiting for clusters to start... 11:58 DEBUG [tests.integration.conftest] Waiting for clusters to start... 11:58 INFO [databricks.labs.ucx.install] Installing UCX v0.27.2+7020240703115820 11:58 INFO [databricks.labs.ucx.install] Creating ucx schemas... 12:00 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-data-reconciliation 12:00 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=experimental-workflow-linter 12:00 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-hiveserde-tables-in-place-experimental 12:00 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups 12:00 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-tables-ctas 12:00 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions 12:00 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-experimental 12:00 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment 12:00 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables 12:00 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=scan-tables-in-mounts-experimental 12:00 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental 12:00 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups 12:00 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=failing 12:00 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.aPxN/README for the next steps. 12:00 DEBUG [databricks.labs.ucx.installer.workflows] starting migrate-external-tables-ctas job: https://DATABRICKS_HOST#job/76112732976656 11:54 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/cOON 11:54 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.hiveserde_in_place_e4ery: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_e4ery 11:54 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.hiveserde_in_place_e4ery', metastore_id=None, name='hiveserde_in_place_e4ery', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 11:54 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_storage_dir fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_e4ery 11:54 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_e4ery.parquet_serde_e4ery: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_e4ery/parquet_serde_e4ery 11:54 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_e4ery.parquet_serde_e4ery', metastore_id=None, name='parquet_serde_e4ery', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070313'}, row_filter=None, schema_name='hiveserde_in_place_e4ery', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_e4ery/parquet_serde_e4ery', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 11:55 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_e4ery.orc_serde_e4ery: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_e4ery/orc_serde_e4ery 11:55 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_e4ery.orc_serde_e4ery', metastore_id=None, name='orc_serde_e4ery', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070313'}, row_filter=None, schema_name='hiveserde_in_place_e4ery', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_e4ery/orc_serde_e4ery', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 11:55 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_e4ery.avro_serde_e4ery: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_e4ery/avro_serde_e4ery 11:55 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_e4ery.avro_serde_e4ery', metastore_id=None, name='avro_serde_e4ery', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070313'}, row_filter=None, schema_name='hiveserde_in_place_e4ery', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_e4ery/avro_serde_e4ery', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 11:55 DEBUG [databricks.labs.ucx.mixins.fixtures] added catalog fixture: CatalogInfo(browse_only=False, catalog_type=, comment='', connection_name=None, created_at=1720007729713, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cxpis', isolation_mode=, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_cxpis', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1720007729713, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d') 11:55 INFO [databricks.labs.ucx.mixins.fixtures] Schema ucx_cxpis.hiveserde_in_place_e4ery: https://DATABRICKS_HOST/explore/data/ucx_cxpis/hiveserde_in_place_e4ery 11:55 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_cxpis', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cxpis.hiveserde_in_place_e4ery', metastore_id=None, name='hiveserde_in_place_e4ery', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 11:55 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_s5tqb: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_s5tqb 11:55 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_s5tqb', metastore_id=None, name='ucx_s5tqb', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 11:58 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.aPxN/config.yml) doesn't exist. 11:58 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration 11:58 INFO [databricks.labs.ucx.installer.hms_lineage] HMS Lineage feature creates one system table named system.hms_to_uc_migration.table_access and helps in your migration process from HMS to UC by allowing you to programmatically query HMS lineage data. 11:58 INFO [databricks.labs.ucx.install] Fetching installations... 11:58 WARNING [databricks.labs.ucx.install] Existing installation at /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.aPxN is corrupted. Skipping... 11:58 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy. 11:58 DEBUG [tests.integration.conftest] Waiting for clusters to start... 11:58 DEBUG [tests.integration.conftest] Waiting for clusters to start... 11:58 INFO [databricks.labs.ucx.install] Installing UCX v0.27.2+7020240703115820 11:58 INFO [databricks.labs.ucx.install] Creating ucx schemas... 12:00 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-data-reconciliation 12:00 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=experimental-workflow-linter 12:00 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-hiveserde-tables-in-place-experimental 12:00 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups 12:00 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-tables-ctas 12:00 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions 12:00 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-experimental 12:00 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment 12:00 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables 12:00 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=scan-tables-in-mounts-experimental 12:00 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental 12:00 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups 12:00 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=failing 12:00 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.aPxN/README for the next steps. 12:00 DEBUG [databricks.labs.ucx.installer.workflows] starting migrate-external-tables-ctas job: https://DATABRICKS_HOST#job/76112732976656 12:05 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 make_storage_dir fixtures 12:05 DEBUG [databricks.labs.ucx.mixins.fixtures] removing make_storage_dir fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_e4ery 12:05 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 make_dbfs_data_copy fixtures 12:05 DEBUG [databricks.labs.ucx.mixins.fixtures] removing make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/cOON 12:05 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 cluster fixtures 12:05 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 catalog fixtures 12:05 DEBUG [databricks.labs.ucx.mixins.fixtures] removing catalog fixture: CatalogInfo(browse_only=False, catalog_type=, comment='', connection_name=None, created_at=1720007729713, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cxpis', isolation_mode=, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_cxpis', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1720007729713, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d') 12:05 INFO [databricks.labs.ucx.install] Deleting UCX v0.27.2+7020240703115820 from https://DATABRICKS_HOST 12:05 INFO [databricks.labs.ucx.install] Deleting inventory database ucx_s5tqb 12:06 INFO [databricks.labs.ucx.install] Deleting jobs 12:06 INFO [databricks.labs.ucx.install] Deleting migrate-data-reconciliation job_id=499817837196600. 12:06 INFO [databricks.labs.ucx.install] Deleting experimental-workflow-linter job_id=646878859659157. 12:06 INFO [databricks.labs.ucx.install] Deleting migrate-external-hiveserde-tables-in-place-experimental job_id=749870669313181. 12:06 INFO [databricks.labs.ucx.install] Deleting migrate-groups job_id=1018334872506320. 12:06 INFO [databricks.labs.ucx.install] Deleting migrate-external-tables-ctas job_id=76112732976656. 12:06 INFO [databricks.labs.ucx.install] Deleting validate-groups-permissions job_id=52311418719515. 12:06 INFO [databricks.labs.ucx.install] Deleting migrate-groups-experimental job_id=298523344697960. 12:06 INFO [databricks.labs.ucx.install] Deleting assessment job_id=1886325692301. 12:06 INFO [databricks.labs.ucx.install] Deleting migrate-tables job_id=629610867125992. 12:06 INFO [databricks.labs.ucx.install] Deleting scan-tables-in-mounts-experimental job_id=330500178144336. 12:06 INFO [databricks.labs.ucx.install] Deleting migrate-tables-in-mounts-experimental job_id=1102924693286379. 12:06 INFO [databricks.labs.ucx.install] Deleting remove-workspace-local-backup-groups job_id=1088250477564336. 12:06 INFO [databricks.labs.ucx.install] Deleting failing job_id=255911693096051. 12:06 INFO [databricks.labs.ucx.install] Deleting cluster policy 12:06 INFO [databricks.labs.ucx.install] Deleting secret scope 12:06 INFO [databricks.labs.ucx.install] UnInstalling UCX complete 12:06 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 workspace user fixtures 12:06 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 account group fixtures 12:06 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 workspace group fixtures 12:06 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 table fixtures 12:06 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 3 table fixtures 12:06 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_e4ery.parquet_serde_e4ery', metastore_id=None, name='parquet_serde_e4ery', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070313'}, row_filter=None, schema_name='hiveserde_in_place_e4ery', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_e4ery/parquet_serde_e4ery', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 12:06 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_e4ery.orc_serde_e4ery', metastore_id=None, name='orc_serde_e4ery', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070313'}, row_filter=None, schema_name='hiveserde_in_place_e4ery', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_e4ery/orc_serde_e4ery', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 12:06 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_e4ery.avro_serde_e4ery', metastore_id=None, name='avro_serde_e4ery', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070313'}, row_filter=None, schema_name='hiveserde_in_place_e4ery', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_e4ery/avro_serde_e4ery', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 12:06 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 3 schema fixtures 12:06 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.hiveserde_in_place_e4ery', metastore_id=None, name='hiveserde_in_place_e4ery', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 12:06 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_cxpis', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cxpis.hiveserde_in_place_e4ery', metastore_id=None, name='hiveserde_in_place_e4ery', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 12:07 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_s5tqb', metastore_id=None, name='ucx_s5tqb', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) [gw4] linux -- Python 3.10.14 /home/runner/work/ucx/ucx/.venv/bin/python ```
❌ test_running_real_workflow_linter_job: TimeoutError: Timed out after 0:02:00 (9m14.009s) ``` TimeoutError: Timed out after 0:02:00 [gw3] linux -- Python 3.10.14 /home/runner/work/ucx/ucx/.venv/bin/python 12:06 DEBUG [databricks.labs.ucx.mixins.fixtures] added directory fixture: /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/sdk-gzQc 12:06 DEBUG [databricks.labs.ucx.mixins.fixtures] added notebook fixture: /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/sdk-gzQc/notebook.ipynb 12:06 INFO [databricks.labs.ucx.mixins.fixtures] Job: https://DATABRICKS_HOST#job/85504123707915 12:06 DEBUG [databricks.labs.ucx.mixins.fixtures] added job fixture: CreateResponse(job_id=85504123707915) 12:06 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_sjlby: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_sjlby 12:06 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_sjlby', metastore_id=None, name='ucx_sjlby', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 12:06 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.mkK5/config.yml) doesn't exist. 12:06 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration 12:06 INFO [databricks.labs.ucx.installer.hms_lineage] HMS Lineage feature creates one system table named system.hms_to_uc_migration.table_access and helps in your migration process from HMS to UC by allowing you to programmatically query HMS lineage data. 12:06 INFO [databricks.labs.ucx.install] Fetching installations... 12:07 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy. 12:07 DEBUG [tests.integration.conftest] Waiting for clusters to start... 12:08 DEBUG [tests.integration.conftest] Waiting for clusters to start... 12:08 INFO [databricks.labs.ucx.install] Installing UCX v0.27.2+7020240703120851 12:08 INFO [databricks.labs.ucx.install] Creating ucx schemas... 12:10 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-tables-ctas 12:10 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=scan-tables-in-mounts-experimental 12:10 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-data-reconciliation 12:10 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=failing 12:10 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-experimental 12:10 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment 12:10 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups 12:10 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups 12:10 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental 12:10 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables 12:10 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=experimental-workflow-linter 12:10 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions 12:10 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-hiveserde-tables-in-place-experimental 12:10 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.mkK5/README for the next steps. 12:10 DEBUG [databricks.labs.ucx.installer.workflows] starting experimental-workflow-linter job: https://DATABRICKS_HOST#job/680372467889391 12:14 DEBUG [databricks.labs.ucx.installer.workflows] Validating experimental-workflow-linter workflow: https://DATABRICKS_HOST#job/680372467889391 12:06 DEBUG [databricks.labs.ucx.mixins.fixtures] added directory fixture: /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/sdk-gzQc 12:06 DEBUG [databricks.labs.ucx.mixins.fixtures] added notebook fixture: /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/sdk-gzQc/notebook.ipynb 12:06 INFO [databricks.labs.ucx.mixins.fixtures] Job: https://DATABRICKS_HOST#job/85504123707915 12:06 DEBUG [databricks.labs.ucx.mixins.fixtures] added job fixture: CreateResponse(job_id=85504123707915) 12:06 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_sjlby: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_sjlby 12:06 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_sjlby', metastore_id=None, name='ucx_sjlby', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 12:06 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.mkK5/config.yml) doesn't exist. 12:06 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration 12:06 INFO [databricks.labs.ucx.installer.hms_lineage] HMS Lineage feature creates one system table named system.hms_to_uc_migration.table_access and helps in your migration process from HMS to UC by allowing you to programmatically query HMS lineage data. 12:06 INFO [databricks.labs.ucx.install] Fetching installations... 12:07 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy. 12:07 DEBUG [tests.integration.conftest] Waiting for clusters to start... 12:08 DEBUG [tests.integration.conftest] Waiting for clusters to start... 12:08 INFO [databricks.labs.ucx.install] Installing UCX v0.27.2+7020240703120851 12:08 INFO [databricks.labs.ucx.install] Creating ucx schemas... 12:10 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-tables-ctas 12:10 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=scan-tables-in-mounts-experimental 12:10 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-data-reconciliation 12:10 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=failing 12:10 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-experimental 12:10 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment 12:10 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups 12:10 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups 12:10 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental 12:10 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables 12:10 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=experimental-workflow-linter 12:10 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions 12:10 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-hiveserde-tables-in-place-experimental 12:10 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.mkK5/README for the next steps. 12:10 DEBUG [databricks.labs.ucx.installer.workflows] starting experimental-workflow-linter job: https://DATABRICKS_HOST#job/680372467889391 12:14 DEBUG [databricks.labs.ucx.installer.workflows] Validating experimental-workflow-linter workflow: https://DATABRICKS_HOST#job/680372467889391 12:15 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 job fixtures 12:15 DEBUG [databricks.labs.ucx.mixins.fixtures] removing job fixture: CreateResponse(job_id=85504123707915) 12:15 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 directory fixtures 12:15 DEBUG [databricks.labs.ucx.mixins.fixtures] removing directory fixture: /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/sdk-gzQc 12:15 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 notebook fixtures 12:15 DEBUG [databricks.labs.ucx.mixins.fixtures] removing notebook fixture: /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/sdk-gzQc/notebook.ipynb 12:15 DEBUG [databricks.labs.ucx.mixins.fixtures] ignoring error while notebook /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/sdk-gzQc/notebook.ipynb teardown: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/sdk-gzQc/notebook.ipynb) doesn't exist. 12:15 INFO [databricks.labs.ucx.install] Deleting UCX v0.27.2+7020240703120851 from https://DATABRICKS_HOST 12:15 INFO [databricks.labs.ucx.install] Deleting inventory database ucx_sjlby 12:15 INFO [databricks.labs.ucx.install] Deleting jobs 12:15 INFO [databricks.labs.ucx.install] Deleting migrate-external-tables-ctas job_id=953726823271170. 12:15 INFO [databricks.labs.ucx.install] Deleting scan-tables-in-mounts-experimental job_id=828625124638020. 12:15 INFO [databricks.labs.ucx.install] Deleting migrate-data-reconciliation job_id=1014379554886187. 12:15 INFO [databricks.labs.ucx.install] Deleting failing job_id=601863893285518. 12:15 INFO [databricks.labs.ucx.install] Deleting migrate-groups-experimental job_id=524182414136805. 12:15 INFO [databricks.labs.ucx.install] Deleting assessment job_id=906299206100753. 12:15 INFO [databricks.labs.ucx.install] Deleting remove-workspace-local-backup-groups job_id=399351505020603. 12:15 INFO [databricks.labs.ucx.install] Deleting migrate-groups job_id=244524325097771. 12:15 INFO [databricks.labs.ucx.install] Deleting migrate-tables-in-mounts-experimental job_id=592515562289495. 12:15 INFO [databricks.labs.ucx.install] Deleting migrate-tables job_id=1024832619305850. 12:15 INFO [databricks.labs.ucx.install] Deleting experimental-workflow-linter job_id=680372467889391. 12:15 INFO [databricks.labs.ucx.install] Deleting validate-groups-permissions job_id=123649521185609. 12:15 INFO [databricks.labs.ucx.install] Deleting migrate-external-hiveserde-tables-in-place-experimental job_id=973875785052051. 12:15 INFO [databricks.labs.ucx.install] Deleting cluster policy 12:15 INFO [databricks.labs.ucx.install] Deleting secret scope 12:15 INFO [databricks.labs.ucx.install] UnInstalling UCX complete 12:15 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 workspace user fixtures 12:15 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 account group fixtures 12:15 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 workspace group fixtures 12:15 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 table fixtures 12:15 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 table fixtures 12:15 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 schema fixtures 12:15 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_sjlby', metastore_id=None, name='ucx_sjlby', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) [gw3] linux -- Python 3.10.14 /home/runner/work/ucx/ucx/.venv/bin/python ```
❌ test_table_migration_job_refreshes_migration_status[regular-migrate-tables]: TimeoutError: Timed out after 0:05:00 (13m1.889s) ``` TimeoutError: Timed out after 0:05:00 12:05 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/PLf9 12:05 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.migrate_eizxh: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_eizxh 12:05 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.migrate_eizxh', metastore_id=None, name='migrate_eizxh', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 12:05 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_eizxh.ucx_t63ao: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_eizxh/ucx_t63ao 12:05 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_eizxh.ucx_t63ao', metastore_id=None, name='ucx_t63ao', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070313'}, row_filter=None, schema_name='migrate_eizxh', sql_path=None, storage_credential_name=None, storage_location='dbfs:/user/hive/warehouse/migrate_eizxh/ucx_t63ao', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 12:05 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_eizxh.ucx_tdslx: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_eizxh/ucx_tdslx 12:05 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_eizxh.ucx_tdslx', metastore_id=None, name='ucx_tdslx', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070313'}, row_filter=None, schema_name='migrate_eizxh', sql_path=None, storage_credential_name=None, storage_location='dbfs:/tmp/ucx_test_OLQw', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 12:06 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_eizxh.ucx_td3v1: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_eizxh/ucx_td3v1 12:06 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_eizxh.ucx_td3v1', metastore_id=None, name='ucx_td3v1', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070313'}, row_filter=None, schema_name='migrate_eizxh', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/b/PLf9', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 12:06 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_eizxh.ucx_trqtj: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_eizxh/ucx_trqtj 12:06 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_eizxh.ucx_trqtj', metastore_id=None, name='ucx_trqtj', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070313'}, row_filter=None, schema_name='migrate_eizxh', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_eizxh.ucx_t63ao', view_dependencies=None) 12:06 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_eizxh.ucx_tmycg: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_eizxh/ucx_tmycg 12:06 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_eizxh.ucx_tmycg', metastore_id=None, name='ucx_tmycg', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070313'}, row_filter=None, schema_name='migrate_eizxh', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_eizxh.ucx_trqtj', view_dependencies=None) 12:07 DEBUG [databricks.labs.ucx.mixins.fixtures] added catalog fixture: CatalogInfo(browse_only=False, catalog_type=, comment='', connection_name=None, created_at=1720008424119, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cwlaq', isolation_mode=, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_cwlaq', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1720008424119, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d') 12:07 INFO [databricks.labs.ucx.mixins.fixtures] Schema ucx_cwlaq.migrate_eizxh: https://DATABRICKS_HOST/explore/data/ucx_cwlaq/migrate_eizxh 12:07 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_cwlaq', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cwlaq.migrate_eizxh', metastore_id=None, name='migrate_eizxh', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 12:07 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_sdalv: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_sdalv 12:07 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_sdalv', metastore_id=None, name='ucx_sdalv', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) [gw2] linux -- Python 3.10.14 /home/runner/work/ucx/ucx/.venv/bin/python 12:05 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/PLf9 12:05 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.migrate_eizxh: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_eizxh 12:05 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.migrate_eizxh', metastore_id=None, name='migrate_eizxh', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 12:05 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_eizxh.ucx_t63ao: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_eizxh/ucx_t63ao 12:05 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_eizxh.ucx_t63ao', metastore_id=None, name='ucx_t63ao', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070313'}, row_filter=None, schema_name='migrate_eizxh', sql_path=None, storage_credential_name=None, storage_location='dbfs:/user/hive/warehouse/migrate_eizxh/ucx_t63ao', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 12:05 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_eizxh.ucx_tdslx: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_eizxh/ucx_tdslx 12:05 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_eizxh.ucx_tdslx', metastore_id=None, name='ucx_tdslx', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070313'}, row_filter=None, schema_name='migrate_eizxh', sql_path=None, storage_credential_name=None, storage_location='dbfs:/tmp/ucx_test_OLQw', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 12:06 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_eizxh.ucx_td3v1: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_eizxh/ucx_td3v1 12:06 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_eizxh.ucx_td3v1', metastore_id=None, name='ucx_td3v1', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070313'}, row_filter=None, schema_name='migrate_eizxh', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/b/PLf9', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 12:06 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_eizxh.ucx_trqtj: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_eizxh/ucx_trqtj 12:06 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_eizxh.ucx_trqtj', metastore_id=None, name='ucx_trqtj', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070313'}, row_filter=None, schema_name='migrate_eizxh', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_eizxh.ucx_t63ao', view_dependencies=None) 12:06 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_eizxh.ucx_tmycg: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_eizxh/ucx_tmycg 12:06 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_eizxh.ucx_tmycg', metastore_id=None, name='ucx_tmycg', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070313'}, row_filter=None, schema_name='migrate_eizxh', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_eizxh.ucx_trqtj', view_dependencies=None) 12:07 DEBUG [databricks.labs.ucx.mixins.fixtures] added catalog fixture: CatalogInfo(browse_only=False, catalog_type=, comment='', connection_name=None, created_at=1720008424119, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cwlaq', isolation_mode=, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_cwlaq', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1720008424119, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d') 12:07 INFO [databricks.labs.ucx.mixins.fixtures] Schema ucx_cwlaq.migrate_eizxh: https://DATABRICKS_HOST/explore/data/ucx_cwlaq/migrate_eizxh 12:07 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_cwlaq', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cwlaq.migrate_eizxh', metastore_id=None, name='migrate_eizxh', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 12:07 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_sdalv: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_sdalv 12:07 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_sdalv', metastore_id=None, name='ucx_sdalv', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 12:08 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.vcxs/config.yml) doesn't exist. 12:08 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration 12:08 INFO [databricks.labs.ucx.installer.hms_lineage] HMS Lineage feature creates one system table named system.hms_to_uc_migration.table_access and helps in your migration process from HMS to UC by allowing you to programmatically query HMS lineage data. 12:08 INFO [databricks.labs.ucx.install] Fetching installations... 12:08 WARNING [databricks.labs.ucx.install] Existing installation at /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.vcxs is corrupted. Skipping... 12:08 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy. 12:08 DEBUG [tests.integration.conftest] Waiting for clusters to start... 12:08 DEBUG [tests.integration.conftest] Waiting for clusters to start... 12:08 INFO [databricks.labs.ucx.install] Installing UCX v0.27.2+7020240703120852 12:08 INFO [databricks.labs.ucx.install] Creating ucx schemas... 12:10 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental 12:10 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=failing 12:10 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=experimental-workflow-linter 12:10 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions 12:10 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-data-reconciliation 12:10 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups 12:10 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-hiveserde-tables-in-place-experimental 12:10 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment 12:10 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-experimental 12:10 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables 12:10 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups 12:10 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=scan-tables-in-mounts-experimental 12:10 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-tables-ctas 12:10 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.vcxs/README for the next steps. 12:10 DEBUG [databricks.labs.ucx.installer.workflows] starting migrate-tables job: https://DATABRICKS_HOST#job/814146224920281 12:05 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/PLf9 12:05 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.migrate_eizxh: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_eizxh 12:05 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.migrate_eizxh', metastore_id=None, name='migrate_eizxh', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 12:05 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_eizxh.ucx_t63ao: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_eizxh/ucx_t63ao 12:05 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_eizxh.ucx_t63ao', metastore_id=None, name='ucx_t63ao', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070313'}, row_filter=None, schema_name='migrate_eizxh', sql_path=None, storage_credential_name=None, storage_location='dbfs:/user/hive/warehouse/migrate_eizxh/ucx_t63ao', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 12:05 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_eizxh.ucx_tdslx: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_eizxh/ucx_tdslx 12:05 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_eizxh.ucx_tdslx', metastore_id=None, name='ucx_tdslx', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070313'}, row_filter=None, schema_name='migrate_eizxh', sql_path=None, storage_credential_name=None, storage_location='dbfs:/tmp/ucx_test_OLQw', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 12:06 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_eizxh.ucx_td3v1: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_eizxh/ucx_td3v1 12:06 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_eizxh.ucx_td3v1', metastore_id=None, name='ucx_td3v1', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070313'}, row_filter=None, schema_name='migrate_eizxh', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/b/PLf9', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 12:06 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_eizxh.ucx_trqtj: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_eizxh/ucx_trqtj 12:06 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_eizxh.ucx_trqtj', metastore_id=None, name='ucx_trqtj', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070313'}, row_filter=None, schema_name='migrate_eizxh', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_eizxh.ucx_t63ao', view_dependencies=None) 12:06 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_eizxh.ucx_tmycg: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_eizxh/ucx_tmycg 12:06 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_eizxh.ucx_tmycg', metastore_id=None, name='ucx_tmycg', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070313'}, row_filter=None, schema_name='migrate_eizxh', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_eizxh.ucx_trqtj', view_dependencies=None) 12:07 DEBUG [databricks.labs.ucx.mixins.fixtures] added catalog fixture: CatalogInfo(browse_only=False, catalog_type=, comment='', connection_name=None, created_at=1720008424119, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cwlaq', isolation_mode=, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_cwlaq', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1720008424119, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d') 12:07 INFO [databricks.labs.ucx.mixins.fixtures] Schema ucx_cwlaq.migrate_eizxh: https://DATABRICKS_HOST/explore/data/ucx_cwlaq/migrate_eizxh 12:07 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_cwlaq', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cwlaq.migrate_eizxh', metastore_id=None, name='migrate_eizxh', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 12:07 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_sdalv: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_sdalv 12:07 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_sdalv', metastore_id=None, name='ucx_sdalv', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 12:08 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.vcxs/config.yml) doesn't exist. 12:08 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration 12:08 INFO [databricks.labs.ucx.installer.hms_lineage] HMS Lineage feature creates one system table named system.hms_to_uc_migration.table_access and helps in your migration process from HMS to UC by allowing you to programmatically query HMS lineage data. 12:08 INFO [databricks.labs.ucx.install] Fetching installations... 12:08 WARNING [databricks.labs.ucx.install] Existing installation at /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.vcxs is corrupted. Skipping... 12:08 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy. 12:08 DEBUG [tests.integration.conftest] Waiting for clusters to start... 12:08 DEBUG [tests.integration.conftest] Waiting for clusters to start... 12:08 INFO [databricks.labs.ucx.install] Installing UCX v0.27.2+7020240703120852 12:08 INFO [databricks.labs.ucx.install] Creating ucx schemas... 12:10 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental 12:10 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=failing 12:10 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=experimental-workflow-linter 12:10 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions 12:10 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-data-reconciliation 12:10 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups 12:10 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-hiveserde-tables-in-place-experimental 12:10 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment 12:10 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-experimental 12:10 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables 12:10 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups 12:10 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=scan-tables-in-mounts-experimental 12:10 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-tables-ctas 12:10 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.vcxs/README for the next steps. 12:10 DEBUG [databricks.labs.ucx.installer.workflows] starting migrate-tables job: https://DATABRICKS_HOST#job/814146224920281 12:15 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 make_storage_dir fixtures 12:15 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 make_dbfs_data_copy fixtures 12:15 DEBUG [databricks.labs.ucx.mixins.fixtures] removing make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/PLf9 12:15 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 cluster fixtures 12:15 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 catalog fixtures 12:15 DEBUG [databricks.labs.ucx.mixins.fixtures] removing catalog fixture: CatalogInfo(browse_only=False, catalog_type=, comment='', connection_name=None, created_at=1720008424119, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cwlaq', isolation_mode=, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_cwlaq', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1720008424119, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d') 12:15 INFO [databricks.labs.ucx.install] Deleting UCX v0.27.2+7020240703120852 from https://DATABRICKS_HOST 12:15 INFO [databricks.labs.ucx.install] Deleting inventory database ucx_sdalv 12:15 INFO [databricks.labs.ucx.install] Deleting jobs 12:15 INFO [databricks.labs.ucx.install] Deleting migrate-tables-in-mounts-experimental job_id=71178285117893. 12:15 INFO [databricks.labs.ucx.install] Deleting failing job_id=222039804785238. 12:15 INFO [databricks.labs.ucx.install] Deleting experimental-workflow-linter job_id=717152305307216. 12:15 INFO [databricks.labs.ucx.install] Deleting validate-groups-permissions job_id=951899207026210. 12:15 INFO [databricks.labs.ucx.install] Deleting migrate-data-reconciliation job_id=208512849397927. 12:15 INFO [databricks.labs.ucx.install] Deleting remove-workspace-local-backup-groups job_id=743175546684108. 12:15 INFO [databricks.labs.ucx.install] Deleting migrate-external-hiveserde-tables-in-place-experimental job_id=765472589549712. 12:15 INFO [databricks.labs.ucx.install] Deleting assessment job_id=574785592423998. 12:15 INFO [databricks.labs.ucx.install] Deleting migrate-groups-experimental job_id=276623665744540. 12:15 INFO [databricks.labs.ucx.install] Deleting migrate-tables job_id=814146224920281. 12:15 INFO [databricks.labs.ucx.install] Deleting migrate-groups job_id=354476253270405. 12:16 INFO [databricks.labs.ucx.install] Deleting scan-tables-in-mounts-experimental job_id=521090362255115. 12:16 INFO [databricks.labs.ucx.install] Deleting migrate-external-tables-ctas job_id=961266195346761. 12:16 INFO [databricks.labs.ucx.install] Deleting cluster policy 12:16 INFO [databricks.labs.ucx.install] Deleting secret scope 12:16 INFO [databricks.labs.ucx.install] UnInstalling UCX complete 12:16 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 workspace user fixtures 12:16 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 account group fixtures 12:16 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 workspace group fixtures 12:16 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 table fixtures 12:16 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 5 table fixtures 12:16 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_eizxh.ucx_t63ao', metastore_id=None, name='ucx_t63ao', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070313'}, row_filter=None, schema_name='migrate_eizxh', sql_path=None, storage_credential_name=None, storage_location='dbfs:/user/hive/warehouse/migrate_eizxh/ucx_t63ao', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 12:16 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_eizxh.ucx_tdslx', metastore_id=None, name='ucx_tdslx', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070313'}, row_filter=None, schema_name='migrate_eizxh', sql_path=None, storage_credential_name=None, storage_location='dbfs:/tmp/ucx_test_OLQw', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 12:16 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_eizxh.ucx_td3v1', metastore_id=None, name='ucx_td3v1', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070313'}, row_filter=None, schema_name='migrate_eizxh', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/b/PLf9', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 12:16 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_eizxh.ucx_trqtj', metastore_id=None, name='ucx_trqtj', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070313'}, row_filter=None, schema_name='migrate_eizxh', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_eizxh.ucx_t63ao', view_dependencies=None) 12:17 DEBUG [databricks.labs.ucx.mixins.fixtures] ignoring error while table TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_eizxh.ucx_trqtj', metastore_id=None, name='ucx_trqtj', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070313'}, row_filter=None, schema_name='migrate_eizxh', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_eizxh.ucx_t63ao', view_dependencies=None) teardown: [WRONG_COMMAND_FOR_OBJECT_TYPE] The operation DROP TABLE requires a EXTERNAL or MANAGED. But hive_metastore.migrate_eizxh.ucx_trqtj is a VIEW. Use DROP VIEW instead. SQLSTATE: 42809 12:17 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_eizxh.ucx_tmycg', metastore_id=None, name='ucx_tmycg', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070313'}, row_filter=None, schema_name='migrate_eizxh', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_eizxh.ucx_trqtj', view_dependencies=None) 12:17 DEBUG [databricks.labs.ucx.mixins.fixtures] ignoring error while table TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_eizxh.ucx_tmycg', metastore_id=None, name='ucx_tmycg', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070313'}, row_filter=None, schema_name='migrate_eizxh', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_eizxh.ucx_trqtj', view_dependencies=None) teardown: [WRONG_COMMAND_FOR_OBJECT_TYPE] The operation DROP TABLE requires a EXTERNAL or MANAGED. But hive_metastore.migrate_eizxh.ucx_tmycg is a VIEW. Use DROP VIEW instead. SQLSTATE: 42809 12:17 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 3 schema fixtures 12:17 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.migrate_eizxh', metastore_id=None, name='migrate_eizxh', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 12:17 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_cwlaq', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cwlaq.migrate_eizxh', metastore_id=None, name='migrate_eizxh', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 12:17 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_sdalv', metastore_id=None, name='ucx_sdalv', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) [gw2] linux -- Python 3.10.14 /home/runner/work/ucx/ucx/.venv/bin/python ```

Running from acceptance #4289