search

databrickslabs / ucx

Your best companion for upgrading to Unity Catalog. UCX will guide you, the Databricks customer, through the process of upgrading your account, groups, workspaces, jobs etc. to Unity Catalog.
Other
193 stars 69 forks source link

whitelist databricks-feature-engineering #1976

Closed ericvergnaud closed 1 day ago

ericvergnaud commented 2 days ago

Changes

whitelist databricks-feature-engineering

Linked issues

Progresses #1901

Functionality

None

Tests

github-actions[bot] commented 2 days ago

❌ 87/91 passed, 4 failed, 21 skipped, 5h23m9s total

❌ test_create_catalog_schema_with_principal_acl_CLOUD_ENV: TimeoutError: Timed out after 0:02:00 (5m46.451s) ``` TimeoutError: Timed out after 0:02:00 16:47 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/Z14w 16:48 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_s90eq: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_s90eq 16:48 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_s90eq', metastore_id=None, name='ucx_s90eq', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 16:48 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.ucx_s90eq.ucx_tydsj: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_s90eq/ucx_tydsj 16:48 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.ucx_s90eq.ucx_tydsj', metastore_id=None, name='ucx_tydsj', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070218'}, row_filter=None, schema_name='ucx_s90eq', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/b/Z14w', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 16:48 DEBUG [databricks.labs.ucx.mixins.fixtures] added catalog fixture: CatalogInfo(browse_only=False, catalog_type=, comment='', connection_name=None, created_at=1719938933692, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_c16lb', isolation_mode=, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_c16lb', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1719938933692, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d') 16:49 INFO [databricks.labs.ucx.mixins.fixtures] Schema ucx_c16lb.ucx_s90eq: https://DATABRICKS_HOST/explore/data/ucx_c16lb/ucx_s90eq 16:49 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_c16lb', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_c16lb.ucx_s90eq', metastore_id=None, name='ucx_s90eq', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) [gw8] linux -- Python 3.10.14 /home/runner/work/ucx/ucx/.venv/bin/python 16:47 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/Z14w 16:48 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_s90eq: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_s90eq 16:48 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_s90eq', metastore_id=None, name='ucx_s90eq', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 16:48 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.ucx_s90eq.ucx_tydsj: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_s90eq/ucx_tydsj 16:48 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.ucx_s90eq.ucx_tydsj', metastore_id=None, name='ucx_tydsj', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070218'}, row_filter=None, schema_name='ucx_s90eq', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/b/Z14w', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 16:48 DEBUG [databricks.labs.ucx.mixins.fixtures] added catalog fixture: CatalogInfo(browse_only=False, catalog_type=, comment='', connection_name=None, created_at=1719938933692, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_c16lb', isolation_mode=, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_c16lb', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1719938933692, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d') 16:49 INFO [databricks.labs.ucx.mixins.fixtures] Schema ucx_c16lb.ucx_s90eq: https://DATABRICKS_HOST/explore/data/ucx_c16lb/ucx_s90eq 16:49 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_c16lb', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_c16lb.ucx_s90eq', metastore_id=None, name='ucx_s90eq', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 16:49 DEBUG [databricks.labs.ucx.mixins.fixtures] added cluster fixture: 16:49 DEBUG [databricks.labs.ucx.mixins.fixtures] added workspace user fixture: User(active=True, display_name='sdk-xbhd7owq@example.com', emails=[ComplexValue(display=None, primary=True, ref=None, type='work', value='sdk-xbhd7owq@example.com')], entitlements=[], external_id=None, groups=[], id='1962998821917202', name=Name(family_name=None, given_name='sdk-xbhd7owq@example.com'), roles=[], schemas=[, ], user_name='sdk-xbhd7owq@example.com') 16:49 DEBUG [databricks.labs.ucx.mixins.fixtures] added cluster permissions fixture: 0702-164921-x8h66ybl [group_name admins CAN_MANAGE] -> [user_name sdk-xbhd7owq@example.com CAN_ATTACH_TO] 16:49 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_sfhmy: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_sfhmy 16:49 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_sfhmy', metastore_id=None, name='ucx_sfhmy', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 16:49 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 16:49 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 16:49 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 16:49 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 16:49 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 16:49 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 16:49 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 16:49 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 16:49 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 16:49 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 16:49 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 16:49 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 16:49 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 16:49 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 16:49 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 16:49 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 16:49 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 16:49 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 16:49 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 16:49 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 16:49 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 16:49 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 16:49 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 16:49 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 16:49 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 16:49 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 16:49 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 16:49 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 16:49 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 16:49 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 16:49 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 16:49 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 16:49 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 16:49 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 16:49 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 16:49 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 16:49 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 16:49 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 16:49 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 16:49 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 16:49 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 16:49 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 16:49 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 16:49 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 16:49 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 16:49 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 16:49 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 16:49 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 16:49 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 16:49 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 16:49 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 16:49 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 16:49 INFO [databricks.labs.ucx.hive_metastore.catalog_schema] Listing existing UC catalogs and schemas 16:50 DEBUG [databricks.labs.ucx.mixins.fixtures] added cluster fixture: 16:50 DEBUG [databricks.labs.ucx.mixins.fixtures] added workspace user fixture: User(active=True, display_name='sdk-smh0c9ts@example.com', emails=[ComplexValue(display=None, primary=True, ref=None, type='work', value='sdk-smh0c9ts@example.com')], entitlements=[], external_id=None, groups=[], id='8888737530109253', name=Name(family_name=None, given_name='sdk-smh0c9ts@example.com'), roles=[], schemas=[, ], user_name='sdk-smh0c9ts@example.com') 16:50 DEBUG [databricks.labs.ucx.mixins.fixtures] added cluster permissions fixture: 0702-165019-pmmrn61t [group_name admins CAN_MANAGE] -> [user_name sdk-smh0c9ts@example.com CAN_ATTACH_TO] 16:50 INFO [databricks.labs.ucx.hive_metastore.catalog_schema] Listing existing UC catalogs and schemas 16:50 DEBUG [databricks.labs.ucx.mixins.fixtures] added cluster fixture: 16:50 DEBUG [databricks.labs.ucx.mixins.fixtures] added workspace user fixture: User(active=True, display_name='sdk-fyznqg18@example.com', emails=[ComplexValue(display=None, primary=True, ref=None, type='work', value='sdk-fyznqg18@example.com')], entitlements=[], external_id=None, groups=[], id='7888406658453865', name=Name(family_name=None, given_name='sdk-fyznqg18@example.com'), roles=[], schemas=[, ], user_name='sdk-fyznqg18@example.com') 16:51 DEBUG [databricks.labs.ucx.mixins.fixtures] added cluster permissions fixture: 0702-165051-7v3hdvio [group_name admins CAN_MANAGE] -> [user_name sdk-fyznqg18@example.com CAN_ATTACH_TO] 16:51 INFO [databricks.labs.ucx.hive_metastore.catalog_schema] Listing existing UC catalogs and schemas 16:47 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/Z14w 16:48 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_s90eq: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_s90eq 16:48 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_s90eq', metastore_id=None, name='ucx_s90eq', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 16:48 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.ucx_s90eq.ucx_tydsj: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_s90eq/ucx_tydsj 16:48 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.ucx_s90eq.ucx_tydsj', metastore_id=None, name='ucx_tydsj', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070218'}, row_filter=None, schema_name='ucx_s90eq', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/b/Z14w', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 16:48 DEBUG [databricks.labs.ucx.mixins.fixtures] added catalog fixture: CatalogInfo(browse_only=False, catalog_type=, comment='', connection_name=None, created_at=1719938933692, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_c16lb', isolation_mode=, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_c16lb', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1719938933692, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d') 16:49 INFO [databricks.labs.ucx.mixins.fixtures] Schema ucx_c16lb.ucx_s90eq: https://DATABRICKS_HOST/explore/data/ucx_c16lb/ucx_s90eq 16:49 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_c16lb', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_c16lb.ucx_s90eq', metastore_id=None, name='ucx_s90eq', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 16:49 DEBUG [databricks.labs.ucx.mixins.fixtures] added cluster fixture: 16:49 DEBUG [databricks.labs.ucx.mixins.fixtures] added workspace user fixture: User(active=True, display_name='sdk-xbhd7owq@example.com', emails=[ComplexValue(display=None, primary=True, ref=None, type='work', value='sdk-xbhd7owq@example.com')], entitlements=[], external_id=None, groups=[], id='1962998821917202', name=Name(family_name=None, given_name='sdk-xbhd7owq@example.com'), roles=[], schemas=[, ], user_name='sdk-xbhd7owq@example.com') 16:49 DEBUG [databricks.labs.ucx.mixins.fixtures] added cluster permissions fixture: 0702-164921-x8h66ybl [group_name admins CAN_MANAGE] -> [user_name sdk-xbhd7owq@example.com CAN_ATTACH_TO] 16:49 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_sfhmy: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_sfhmy 16:49 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_sfhmy', metastore_id=None, name='ucx_sfhmy', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 16:49 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 16:49 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 16:49 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 16:49 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 16:49 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 16:49 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 16:49 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 16:49 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 16:49 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 16:49 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 16:49 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 16:49 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 16:49 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 16:49 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 16:49 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 16:49 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 16:49 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 16:49 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 16:49 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 16:49 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 16:49 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 16:49 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 16:49 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 16:49 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 16:49 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 16:49 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 16:49 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 16:49 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 16:49 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 16:49 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 16:49 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 16:49 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 16:49 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 16:49 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 16:49 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 16:49 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 16:49 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 16:49 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 16:49 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 16:49 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 16:49 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 16:49 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 16:49 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 16:49 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 16:49 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 16:49 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 16:49 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 16:49 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 16:49 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 16:49 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 16:49 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 16:49 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 16:49 INFO [databricks.labs.ucx.hive_metastore.catalog_schema] Listing existing UC catalogs and schemas 16:50 DEBUG [databricks.labs.ucx.mixins.fixtures] added cluster fixture: 16:50 DEBUG [databricks.labs.ucx.mixins.fixtures] added workspace user fixture: User(active=True, display_name='sdk-smh0c9ts@example.com', emails=[ComplexValue(display=None, primary=True, ref=None, type='work', value='sdk-smh0c9ts@example.com')], entitlements=[], external_id=None, groups=[], id='8888737530109253', name=Name(family_name=None, given_name='sdk-smh0c9ts@example.com'), roles=[], schemas=[, ], user_name='sdk-smh0c9ts@example.com') 16:50 DEBUG [databricks.labs.ucx.mixins.fixtures] added cluster permissions fixture: 0702-165019-pmmrn61t [group_name admins CAN_MANAGE] -> [user_name sdk-smh0c9ts@example.com CAN_ATTACH_TO] 16:50 INFO [databricks.labs.ucx.hive_metastore.catalog_schema] Listing existing UC catalogs and schemas 16:50 DEBUG [databricks.labs.ucx.mixins.fixtures] added cluster fixture: 16:50 DEBUG [databricks.labs.ucx.mixins.fixtures] added workspace user fixture: User(active=True, display_name='sdk-fyznqg18@example.com', emails=[ComplexValue(display=None, primary=True, ref=None, type='work', value='sdk-fyznqg18@example.com')], entitlements=[], external_id=None, groups=[], id='7888406658453865', name=Name(family_name=None, given_name='sdk-fyznqg18@example.com'), roles=[], schemas=[, ], user_name='sdk-fyznqg18@example.com') 16:51 DEBUG [databricks.labs.ucx.mixins.fixtures] added cluster permissions fixture: 0702-165051-7v3hdvio [group_name admins CAN_MANAGE] -> [user_name sdk-fyznqg18@example.com CAN_ATTACH_TO] 16:51 INFO [databricks.labs.ucx.hive_metastore.catalog_schema] Listing existing UC catalogs and schemas 16:51 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 3 cluster permissions fixtures 16:51 DEBUG [databricks.labs.ucx.mixins.fixtures] removing cluster permissions fixture: 0702-164921-x8h66ybl [group_name admins CAN_MANAGE] -> [user_name sdk-xbhd7owq@example.com CAN_ATTACH_TO] 16:51 DEBUG [databricks.labs.ucx.mixins.fixtures] removing cluster permissions fixture: 0702-165019-pmmrn61t [group_name admins CAN_MANAGE] -> [user_name sdk-smh0c9ts@example.com CAN_ATTACH_TO] 16:51 DEBUG [databricks.labs.ucx.mixins.fixtures] removing cluster permissions fixture: 0702-165051-7v3hdvio [group_name admins CAN_MANAGE] -> [user_name sdk-fyznqg18@example.com CAN_ATTACH_TO] 16:52 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 catalog fixtures 16:52 DEBUG [databricks.labs.ucx.mixins.fixtures] removing catalog fixture: CatalogInfo(browse_only=False, catalog_type=, comment='', connection_name=None, created_at=1719938933692, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_c16lb', isolation_mode=, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_c16lb', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1719938933692, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d') 16:52 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 make_dbfs_data_copy fixtures 16:52 DEBUG [databricks.labs.ucx.mixins.fixtures] removing make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/Z14w 16:52 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 3 cluster fixtures 16:52 DEBUG [databricks.labs.ucx.mixins.fixtures] removing cluster fixture: 16:52 DEBUG [databricks.labs.ucx.mixins.fixtures] removing cluster fixture: 16:52 DEBUG [databricks.labs.ucx.mixins.fixtures] removing cluster fixture: 16:52 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 workspace group fixtures 16:52 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 table fixtures 16:52 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 table fixtures 16:52 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.ucx_s90eq.ucx_tydsj', metastore_id=None, name='ucx_tydsj', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070218'}, row_filter=None, schema_name='ucx_s90eq', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/b/Z14w', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 16:52 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 3 schema fixtures 16:52 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_s90eq', metastore_id=None, name='ucx_s90eq', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 16:52 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_c16lb', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_c16lb.ucx_s90eq', metastore_id=None, name='ucx_s90eq', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 16:53 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_sfhmy', metastore_id=None, name='ucx_sfhmy', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 16:53 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 3 workspace user fixtures 16:53 DEBUG [databricks.labs.ucx.mixins.fixtures] removing workspace user fixture: User(active=True, display_name='sdk-xbhd7owq@example.com', emails=[ComplexValue(display=None, primary=True, ref=None, type='work', value='sdk-xbhd7owq@example.com')], entitlements=[], external_id=None, groups=[], id='1962998821917202', name=Name(family_name=None, given_name='sdk-xbhd7owq@example.com'), roles=[], schemas=[, ], user_name='sdk-xbhd7owq@example.com') 16:53 DEBUG [databricks.labs.ucx.mixins.fixtures] removing workspace user fixture: User(active=True, display_name='sdk-smh0c9ts@example.com', emails=[ComplexValue(display=None, primary=True, ref=None, type='work', value='sdk-smh0c9ts@example.com')], entitlements=[], external_id=None, groups=[], id='8888737530109253', name=Name(family_name=None, given_name='sdk-smh0c9ts@example.com'), roles=[], schemas=[, ], user_name='sdk-smh0c9ts@example.com') 16:53 DEBUG [databricks.labs.ucx.mixins.fixtures] removing workspace user fixture: User(active=True, display_name='sdk-fyznqg18@example.com', emails=[ComplexValue(display=None, primary=True, ref=None, type='work', value='sdk-fyznqg18@example.com')], entitlements=[], external_id=None, groups=[], id='7888406658453865', name=Name(family_name=None, given_name='sdk-fyznqg18@example.com'), roles=[], schemas=[, ], user_name='sdk-fyznqg18@example.com') [gw8] linux -- Python 3.10.14 /home/runner/work/ucx/ucx/.venv/bin/python ```
❌ test_table_migration_job_refreshes_migration_status[hiveserde-migrate-external-tables-ctas]: TimeoutError: Timed out after 0:05:00 (19m35.491s) ``` TimeoutError: Timed out after 0:05:00 16:54 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/8Tyy 16:55 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.hiveserde_in_place_bu5hf: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_bu5hf 16:55 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.hiveserde_in_place_bu5hf', metastore_id=None, name='hiveserde_in_place_bu5hf', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 16:55 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_storage_dir fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_bu5hf 16:55 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_bu5hf.parquet_serde_bu5hf: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_bu5hf/parquet_serde_bu5hf 16:55 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_bu5hf.parquet_serde_bu5hf', metastore_id=None, name='parquet_serde_bu5hf', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070218'}, row_filter=None, schema_name='hiveserde_in_place_bu5hf', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_bu5hf/parquet_serde_bu5hf', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 16:56 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_bu5hf.orc_serde_bu5hf: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_bu5hf/orc_serde_bu5hf 16:56 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_bu5hf.orc_serde_bu5hf', metastore_id=None, name='orc_serde_bu5hf', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070218'}, row_filter=None, schema_name='hiveserde_in_place_bu5hf', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_bu5hf/orc_serde_bu5hf', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 16:56 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_bu5hf.avro_serde_bu5hf: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_bu5hf/avro_serde_bu5hf 16:56 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_bu5hf.avro_serde_bu5hf', metastore_id=None, name='avro_serde_bu5hf', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070218'}, row_filter=None, schema_name='hiveserde_in_place_bu5hf', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_bu5hf/avro_serde_bu5hf', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 16:56 DEBUG [databricks.labs.ucx.mixins.fixtures] added catalog fixture: CatalogInfo(browse_only=False, catalog_type=, comment='', connection_name=None, created_at=1719939409344, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_chh16', isolation_mode=, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_chh16', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1719939409344, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d') 16:57 INFO [databricks.labs.ucx.mixins.fixtures] Schema ucx_chh16.hiveserde_in_place_bu5hf: https://DATABRICKS_HOST/explore/data/ucx_chh16/hiveserde_in_place_bu5hf 16:57 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_chh16', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_chh16.hiveserde_in_place_bu5hf', metastore_id=None, name='hiveserde_in_place_bu5hf', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 16:57 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_src3c: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_src3c 16:57 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_src3c', metastore_id=None, name='ucx_src3c', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) [gw6] linux -- Python 3.10.14 /home/runner/work/ucx/ucx/.venv/bin/python 16:54 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/8Tyy 16:55 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.hiveserde_in_place_bu5hf: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_bu5hf 16:55 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.hiveserde_in_place_bu5hf', metastore_id=None, name='hiveserde_in_place_bu5hf', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 16:55 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_storage_dir fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_bu5hf 16:55 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_bu5hf.parquet_serde_bu5hf: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_bu5hf/parquet_serde_bu5hf 16:55 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_bu5hf.parquet_serde_bu5hf', metastore_id=None, name='parquet_serde_bu5hf', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070218'}, row_filter=None, schema_name='hiveserde_in_place_bu5hf', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_bu5hf/parquet_serde_bu5hf', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 16:56 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_bu5hf.orc_serde_bu5hf: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_bu5hf/orc_serde_bu5hf 16:56 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_bu5hf.orc_serde_bu5hf', metastore_id=None, name='orc_serde_bu5hf', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070218'}, row_filter=None, schema_name='hiveserde_in_place_bu5hf', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_bu5hf/orc_serde_bu5hf', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 16:56 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_bu5hf.avro_serde_bu5hf: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_bu5hf/avro_serde_bu5hf 16:56 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_bu5hf.avro_serde_bu5hf', metastore_id=None, name='avro_serde_bu5hf', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070218'}, row_filter=None, schema_name='hiveserde_in_place_bu5hf', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_bu5hf/avro_serde_bu5hf', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 16:56 DEBUG [databricks.labs.ucx.mixins.fixtures] added catalog fixture: CatalogInfo(browse_only=False, catalog_type=, comment='', connection_name=None, created_at=1719939409344, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_chh16', isolation_mode=, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_chh16', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1719939409344, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d') 16:57 INFO [databricks.labs.ucx.mixins.fixtures] Schema ucx_chh16.hiveserde_in_place_bu5hf: https://DATABRICKS_HOST/explore/data/ucx_chh16/hiveserde_in_place_bu5hf 16:57 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_chh16', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_chh16.hiveserde_in_place_bu5hf', metastore_id=None, name='hiveserde_in_place_bu5hf', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 16:57 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_src3c: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_src3c 16:57 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_src3c', metastore_id=None, name='ucx_src3c', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 17:01 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.Twdb/config.yml) doesn't exist. 17:01 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration 17:01 INFO [databricks.labs.ucx.installer.hms_lineage] HMS Lineage feature creates one system table named system.hms_to_uc_migration.table_access and helps in your migration process from HMS to UC by allowing you to programmatically query HMS lineage data. 17:01 INFO [databricks.labs.ucx.install] Fetching installations... 17:01 WARNING [databricks.labs.ucx.install] Existing installation at /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.Twdb is corrupted. Skipping... 17:01 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy. 17:01 DEBUG [tests.integration.conftest] Waiting for clusters to start... 17:01 DEBUG [tests.integration.conftest] Waiting for clusters to start... 17:02 INFO [databricks.labs.ucx.install] Installing UCX v0.27.2+6020240702170252 17:02 INFO [databricks.labs.ucx.install] Creating ucx schemas... 17:06 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=scan-tables-in-mounts-experimental 17:06 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-hiveserde-tables-in-place-experimental 17:07 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=experimental-workflow-linter 17:07 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables 17:07 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-experimental 17:07 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-data-reconciliation 17:07 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=failing 17:07 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups 17:07 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment 17:07 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups 17:07 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental 17:07 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-tables-ctas 17:07 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions 17:07 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.Twdb/README for the next steps. 17:07 DEBUG [databricks.labs.ucx.installer.workflows] starting migrate-external-tables-ctas job: https://DATABRICKS_HOST#job/199414338629540 16:54 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/8Tyy 16:55 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.hiveserde_in_place_bu5hf: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_bu5hf 16:55 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.hiveserde_in_place_bu5hf', metastore_id=None, name='hiveserde_in_place_bu5hf', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 16:55 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_storage_dir fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_bu5hf 16:55 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_bu5hf.parquet_serde_bu5hf: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_bu5hf/parquet_serde_bu5hf 16:55 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_bu5hf.parquet_serde_bu5hf', metastore_id=None, name='parquet_serde_bu5hf', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070218'}, row_filter=None, schema_name='hiveserde_in_place_bu5hf', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_bu5hf/parquet_serde_bu5hf', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 16:56 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_bu5hf.orc_serde_bu5hf: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_bu5hf/orc_serde_bu5hf 16:56 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_bu5hf.orc_serde_bu5hf', metastore_id=None, name='orc_serde_bu5hf', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070218'}, row_filter=None, schema_name='hiveserde_in_place_bu5hf', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_bu5hf/orc_serde_bu5hf', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 16:56 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_bu5hf.avro_serde_bu5hf: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_bu5hf/avro_serde_bu5hf 16:56 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_bu5hf.avro_serde_bu5hf', metastore_id=None, name='avro_serde_bu5hf', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070218'}, row_filter=None, schema_name='hiveserde_in_place_bu5hf', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_bu5hf/avro_serde_bu5hf', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 16:56 DEBUG [databricks.labs.ucx.mixins.fixtures] added catalog fixture: CatalogInfo(browse_only=False, catalog_type=, comment='', connection_name=None, created_at=1719939409344, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_chh16', isolation_mode=, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_chh16', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1719939409344, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d') 16:57 INFO [databricks.labs.ucx.mixins.fixtures] Schema ucx_chh16.hiveserde_in_place_bu5hf: https://DATABRICKS_HOST/explore/data/ucx_chh16/hiveserde_in_place_bu5hf 16:57 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_chh16', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_chh16.hiveserde_in_place_bu5hf', metastore_id=None, name='hiveserde_in_place_bu5hf', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 16:57 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_src3c: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_src3c 16:57 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_src3c', metastore_id=None, name='ucx_src3c', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 17:01 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.Twdb/config.yml) doesn't exist. 17:01 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration 17:01 INFO [databricks.labs.ucx.installer.hms_lineage] HMS Lineage feature creates one system table named system.hms_to_uc_migration.table_access and helps in your migration process from HMS to UC by allowing you to programmatically query HMS lineage data. 17:01 INFO [databricks.labs.ucx.install] Fetching installations... 17:01 WARNING [databricks.labs.ucx.install] Existing installation at /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.Twdb is corrupted. Skipping... 17:01 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy. 17:01 DEBUG [tests.integration.conftest] Waiting for clusters to start... 17:01 DEBUG [tests.integration.conftest] Waiting for clusters to start... 17:02 INFO [databricks.labs.ucx.install] Installing UCX v0.27.2+6020240702170252 17:02 INFO [databricks.labs.ucx.install] Creating ucx schemas... 17:06 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=scan-tables-in-mounts-experimental 17:06 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-hiveserde-tables-in-place-experimental 17:07 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=experimental-workflow-linter 17:07 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables 17:07 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-experimental 17:07 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-data-reconciliation 17:07 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=failing 17:07 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups 17:07 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment 17:07 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups 17:07 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental 17:07 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-tables-ctas 17:07 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions 17:07 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.Twdb/README for the next steps. 17:07 DEBUG [databricks.labs.ucx.installer.workflows] starting migrate-external-tables-ctas job: https://DATABRICKS_HOST#job/199414338629540 17:11 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 make_storage_dir fixtures 17:11 DEBUG [databricks.labs.ucx.mixins.fixtures] removing make_storage_dir fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_bu5hf 17:11 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 make_dbfs_data_copy fixtures 17:11 DEBUG [databricks.labs.ucx.mixins.fixtures] removing make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/8Tyy 17:12 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 cluster fixtures 17:12 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 catalog fixtures 17:12 DEBUG [databricks.labs.ucx.mixins.fixtures] removing catalog fixture: CatalogInfo(browse_only=False, catalog_type=, comment='', connection_name=None, created_at=1719939409344, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_chh16', isolation_mode=, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_chh16', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1719939409344, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d') 17:12 INFO [databricks.labs.ucx.install] Deleting UCX v0.27.2+6020240702170252 from https://DATABRICKS_HOST 17:12 INFO [databricks.labs.ucx.install] Deleting inventory database ucx_src3c 17:12 INFO [databricks.labs.ucx.install] Deleting jobs 17:12 INFO [databricks.labs.ucx.install] Deleting scan-tables-in-mounts-experimental job_id=98464748957468. 17:12 INFO [databricks.labs.ucx.install] Deleting migrate-external-hiveserde-tables-in-place-experimental job_id=697908783517541. 17:12 INFO [databricks.labs.ucx.install] Deleting experimental-workflow-linter job_id=699829952266564. 17:12 INFO [databricks.labs.ucx.install] Deleting migrate-tables job_id=1019664110950990. 17:12 INFO [databricks.labs.ucx.install] Deleting migrate-groups-experimental job_id=213699773501601. 17:12 INFO [databricks.labs.ucx.install] Deleting migrate-data-reconciliation job_id=160523731286056. 17:12 INFO [databricks.labs.ucx.install] Deleting failing job_id=244865312564026. 17:12 INFO [databricks.labs.ucx.install] Deleting remove-workspace-local-backup-groups job_id=230331030866844. 17:12 INFO [databricks.labs.ucx.install] Deleting assessment job_id=853610090304833. 17:12 INFO [databricks.labs.ucx.install] Deleting migrate-groups job_id=510320754075154. 17:12 INFO [databricks.labs.ucx.install] Deleting migrate-tables-in-mounts-experimental job_id=952510595764189. 17:12 INFO [databricks.labs.ucx.install] Deleting migrate-external-tables-ctas job_id=199414338629540. 17:12 INFO [databricks.labs.ucx.install] Deleting validate-groups-permissions job_id=217443090601129. 17:12 INFO [databricks.labs.ucx.install] Deleting cluster policy 17:12 INFO [databricks.labs.ucx.install] Deleting secret scope 17:12 INFO [databricks.labs.ucx.install] UnInstalling UCX complete 17:12 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 workspace user fixtures 17:12 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 account group fixtures 17:12 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 workspace group fixtures 17:12 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 table fixtures 17:12 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 3 table fixtures 17:12 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_bu5hf.parquet_serde_bu5hf', metastore_id=None, name='parquet_serde_bu5hf', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070218'}, row_filter=None, schema_name='hiveserde_in_place_bu5hf', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_bu5hf/parquet_serde_bu5hf', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 17:12 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_bu5hf.orc_serde_bu5hf', metastore_id=None, name='orc_serde_bu5hf', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070218'}, row_filter=None, schema_name='hiveserde_in_place_bu5hf', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_bu5hf/orc_serde_bu5hf', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 17:13 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_bu5hf.avro_serde_bu5hf', metastore_id=None, name='avro_serde_bu5hf', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070218'}, row_filter=None, schema_name='hiveserde_in_place_bu5hf', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_bu5hf/avro_serde_bu5hf', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 17:13 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 3 schema fixtures 17:13 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.hiveserde_in_place_bu5hf', metastore_id=None, name='hiveserde_in_place_bu5hf', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 17:13 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_chh16', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_chh16.hiveserde_in_place_bu5hf', metastore_id=None, name='hiveserde_in_place_bu5hf', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 17:14 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_src3c', metastore_id=None, name='ucx_src3c', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) [gw6] linux -- Python 3.10.14 /home/runner/work/ucx/ucx/.venv/bin/python ```
❌ test_running_real_workflow_linter_job: TimeoutError: Timed out after 0:02:00 (7m53.998s) ``` TimeoutError: Timed out after 0:02:00 [gw8] linux -- Python 3.10.14 /home/runner/work/ucx/ucx/.venv/bin/python 17:15 DEBUG [databricks.labs.ucx.mixins.fixtures] added directory fixture: /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/sdk-TZkQ 17:15 DEBUG [databricks.labs.ucx.mixins.fixtures] added notebook fixture: /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/sdk-TZkQ/notebook.ipynb 17:15 INFO [databricks.labs.ucx.mixins.fixtures] Job: https://DATABRICKS_HOST#job/760292123173154 17:15 DEBUG [databricks.labs.ucx.mixins.fixtures] added job fixture: CreateResponse(job_id=760292123173154) 17:16 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_s3anz: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_s3anz 17:16 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_s3anz', metastore_id=None, name='ucx_s3anz', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 17:16 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.wbk2/config.yml) doesn't exist. 17:16 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration 17:16 INFO [databricks.labs.ucx.installer.hms_lineage] HMS Lineage feature creates one system table named system.hms_to_uc_migration.table_access and helps in your migration process from HMS to UC by allowing you to programmatically query HMS lineage data. 17:16 INFO [databricks.labs.ucx.install] Fetching installations... 17:16 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy. 17:16 DEBUG [tests.integration.conftest] Waiting for clusters to start... 17:16 DEBUG [tests.integration.conftest] Waiting for clusters to start... 17:16 INFO [databricks.labs.ucx.install] Installing UCX v0.27.2+6020240702171627 17:16 INFO [databricks.labs.ucx.install] Creating ucx schemas... 17:18 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables 17:18 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-tables-ctas 17:18 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=scan-tables-in-mounts-experimental 17:18 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions 17:18 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=experimental-workflow-linter 17:18 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-data-reconciliation 17:18 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment 17:18 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups 17:18 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups 17:18 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=failing 17:18 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-experimental 17:18 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental 17:18 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-hiveserde-tables-in-place-experimental 17:18 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.wbk2/README for the next steps. 17:18 DEBUG [databricks.labs.ucx.installer.workflows] starting experimental-workflow-linter job: https://DATABRICKS_HOST#job/977100948186972 17:23 DEBUG [databricks.labs.ucx.installer.workflows] Validating experimental-workflow-linter workflow: https://DATABRICKS_HOST#job/977100948186972 17:15 DEBUG [databricks.labs.ucx.mixins.fixtures] added directory fixture: /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/sdk-TZkQ 17:15 DEBUG [databricks.labs.ucx.mixins.fixtures] added notebook fixture: /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/sdk-TZkQ/notebook.ipynb 17:15 INFO [databricks.labs.ucx.mixins.fixtures] Job: https://DATABRICKS_HOST#job/760292123173154 17:15 DEBUG [databricks.labs.ucx.mixins.fixtures] added job fixture: CreateResponse(job_id=760292123173154) 17:16 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_s3anz: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_s3anz 17:16 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_s3anz', metastore_id=None, name='ucx_s3anz', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 17:16 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.wbk2/config.yml) doesn't exist. 17:16 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration 17:16 INFO [databricks.labs.ucx.installer.hms_lineage] HMS Lineage feature creates one system table named system.hms_to_uc_migration.table_access and helps in your migration process from HMS to UC by allowing you to programmatically query HMS lineage data. 17:16 INFO [databricks.labs.ucx.install] Fetching installations... 17:16 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy. 17:16 DEBUG [tests.integration.conftest] Waiting for clusters to start... 17:16 DEBUG [tests.integration.conftest] Waiting for clusters to start... 17:16 INFO [databricks.labs.ucx.install] Installing UCX v0.27.2+6020240702171627 17:16 INFO [databricks.labs.ucx.install] Creating ucx schemas... 17:18 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables 17:18 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-tables-ctas 17:18 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=scan-tables-in-mounts-experimental 17:18 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions 17:18 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=experimental-workflow-linter 17:18 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-data-reconciliation 17:18 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment 17:18 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups 17:18 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups 17:18 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=failing 17:18 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-experimental 17:18 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental 17:18 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-hiveserde-tables-in-place-experimental 17:18 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.wbk2/README for the next steps. 17:18 DEBUG [databricks.labs.ucx.installer.workflows] starting experimental-workflow-linter job: https://DATABRICKS_HOST#job/977100948186972 17:23 DEBUG [databricks.labs.ucx.installer.workflows] Validating experimental-workflow-linter workflow: https://DATABRICKS_HOST#job/977100948186972 17:23 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 job fixtures 17:23 DEBUG [databricks.labs.ucx.mixins.fixtures] removing job fixture: CreateResponse(job_id=760292123173154) 17:23 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 directory fixtures 17:23 DEBUG [databricks.labs.ucx.mixins.fixtures] removing directory fixture: /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/sdk-TZkQ 17:23 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 notebook fixtures 17:23 DEBUG [databricks.labs.ucx.mixins.fixtures] removing notebook fixture: /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/sdk-TZkQ/notebook.ipynb 17:23 DEBUG [databricks.labs.ucx.mixins.fixtures] ignoring error while notebook /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/sdk-TZkQ/notebook.ipynb teardown: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/sdk-TZkQ/notebook.ipynb) doesn't exist. 17:23 INFO [databricks.labs.ucx.install] Deleting UCX v0.27.2+6020240702171627 from https://DATABRICKS_HOST 17:23 INFO [databricks.labs.ucx.install] Deleting inventory database ucx_s3anz 17:23 INFO [databricks.labs.ucx.install] Deleting jobs 17:23 INFO [databricks.labs.ucx.install] Deleting migrate-tables job_id=723231228145789. 17:23 INFO [databricks.labs.ucx.install] Deleting migrate-external-tables-ctas job_id=347909732218227. 17:23 INFO [databricks.labs.ucx.install] Deleting scan-tables-in-mounts-experimental job_id=845361999255028. 17:23 INFO [databricks.labs.ucx.install] Deleting validate-groups-permissions job_id=572077493140177. 17:23 INFO [databricks.labs.ucx.install] Deleting experimental-workflow-linter job_id=977100948186972. 17:23 INFO [databricks.labs.ucx.install] Deleting migrate-data-reconciliation job_id=878211440419680. 17:23 INFO [databricks.labs.ucx.install] Deleting assessment job_id=8483047074347. 17:23 INFO [databricks.labs.ucx.install] Deleting remove-workspace-local-backup-groups job_id=818702192280558. 17:23 INFO [databricks.labs.ucx.install] Deleting migrate-groups job_id=1059290601950185. 17:23 INFO [databricks.labs.ucx.install] Deleting failing job_id=852213740503740. 17:23 INFO [databricks.labs.ucx.install] Deleting migrate-groups-experimental job_id=767664490868990. 17:23 INFO [databricks.labs.ucx.install] Deleting migrate-tables-in-mounts-experimental job_id=497441898961117. 17:23 INFO [databricks.labs.ucx.install] Deleting migrate-external-hiveserde-tables-in-place-experimental job_id=584800356606911. 17:23 INFO [databricks.labs.ucx.install] Deleting cluster policy 17:23 INFO [databricks.labs.ucx.install] Deleting secret scope 17:23 INFO [databricks.labs.ucx.install] UnInstalling UCX complete 17:23 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 workspace user fixtures 17:23 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 account group fixtures 17:23 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 workspace group fixtures 17:23 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 table fixtures 17:23 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 table fixtures 17:23 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 schema fixtures 17:23 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_s3anz', metastore_id=None, name='ucx_s3anz', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) [gw8] linux -- Python 3.10.14 /home/runner/work/ucx/ucx/.venv/bin/python ```
❌ test_table_migration_job_refreshes_migration_status[regular-migrate-tables]: TimeoutError: Timed out after 0:05:00 (12m36.021s) ``` TimeoutError: Timed out after 0:05:00 17:14 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/DSaW 17:14 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.migrate_75zn9: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_75zn9 17:14 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.migrate_75zn9', metastore_id=None, name='migrate_75zn9', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 17:15 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_75zn9.ucx_tnzeu: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_75zn9/ucx_tnzeu 17:15 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_75zn9.ucx_tnzeu', metastore_id=None, name='ucx_tnzeu', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070218'}, row_filter=None, schema_name='migrate_75zn9', sql_path=None, storage_credential_name=None, storage_location='dbfs:/user/hive/warehouse/migrate_75zn9/ucx_tnzeu', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 17:15 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_75zn9.ucx_togpb: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_75zn9/ucx_togpb 17:15 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_75zn9.ucx_togpb', metastore_id=None, name='ucx_togpb', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070218'}, row_filter=None, schema_name='migrate_75zn9', sql_path=None, storage_credential_name=None, storage_location='dbfs:/tmp/ucx_test_U8sA', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 17:15 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_75zn9.ucx_tq0di: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_75zn9/ucx_tq0di 17:15 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_75zn9.ucx_tq0di', metastore_id=None, name='ucx_tq0di', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070218'}, row_filter=None, schema_name='migrate_75zn9', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/b/DSaW', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 17:16 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_75zn9.ucx_tafdm: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_75zn9/ucx_tafdm 17:16 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_75zn9.ucx_tafdm', metastore_id=None, name='ucx_tafdm', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070218'}, row_filter=None, schema_name='migrate_75zn9', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_75zn9.ucx_tnzeu', view_dependencies=None) 17:16 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_75zn9.ucx_tkjfm: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_75zn9/ucx_tkjfm 17:16 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_75zn9.ucx_tkjfm', metastore_id=None, name='ucx_tkjfm', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070218'}, row_filter=None, schema_name='migrate_75zn9', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_75zn9.ucx_tafdm', view_dependencies=None) 17:17 DEBUG [databricks.labs.ucx.mixins.fixtures] added catalog fixture: CatalogInfo(browse_only=False, catalog_type=, comment='', connection_name=None, created_at=1719940623841, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cdrns', isolation_mode=, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_cdrns', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1719940623841, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d') 17:17 INFO [databricks.labs.ucx.mixins.fixtures] Schema ucx_cdrns.migrate_75zn9: https://DATABRICKS_HOST/explore/data/ucx_cdrns/migrate_75zn9 17:17 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_cdrns', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cdrns.migrate_75zn9', metastore_id=None, name='migrate_75zn9', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 17:17 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_svvej: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_svvej 17:17 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_svvej', metastore_id=None, name='ucx_svvej', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) [gw7] linux -- Python 3.10.14 /home/runner/work/ucx/ucx/.venv/bin/python 17:14 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/DSaW 17:14 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.migrate_75zn9: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_75zn9 17:14 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.migrate_75zn9', metastore_id=None, name='migrate_75zn9', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 17:15 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_75zn9.ucx_tnzeu: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_75zn9/ucx_tnzeu 17:15 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_75zn9.ucx_tnzeu', metastore_id=None, name='ucx_tnzeu', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070218'}, row_filter=None, schema_name='migrate_75zn9', sql_path=None, storage_credential_name=None, storage_location='dbfs:/user/hive/warehouse/migrate_75zn9/ucx_tnzeu', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 17:15 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_75zn9.ucx_togpb: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_75zn9/ucx_togpb 17:15 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_75zn9.ucx_togpb', metastore_id=None, name='ucx_togpb', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070218'}, row_filter=None, schema_name='migrate_75zn9', sql_path=None, storage_credential_name=None, storage_location='dbfs:/tmp/ucx_test_U8sA', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 17:15 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_75zn9.ucx_tq0di: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_75zn9/ucx_tq0di 17:15 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_75zn9.ucx_tq0di', metastore_id=None, name='ucx_tq0di', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070218'}, row_filter=None, schema_name='migrate_75zn9', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/b/DSaW', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 17:16 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_75zn9.ucx_tafdm: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_75zn9/ucx_tafdm 17:16 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_75zn9.ucx_tafdm', metastore_id=None, name='ucx_tafdm', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070218'}, row_filter=None, schema_name='migrate_75zn9', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_75zn9.ucx_tnzeu', view_dependencies=None) 17:16 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_75zn9.ucx_tkjfm: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_75zn9/ucx_tkjfm 17:16 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_75zn9.ucx_tkjfm', metastore_id=None, name='ucx_tkjfm', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070218'}, row_filter=None, schema_name='migrate_75zn9', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_75zn9.ucx_tafdm', view_dependencies=None) 17:17 DEBUG [databricks.labs.ucx.mixins.fixtures] added catalog fixture: CatalogInfo(browse_only=False, catalog_type=, comment='', connection_name=None, created_at=1719940623841, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cdrns', isolation_mode=, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_cdrns', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1719940623841, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d') 17:17 INFO [databricks.labs.ucx.mixins.fixtures] Schema ucx_cdrns.migrate_75zn9: https://DATABRICKS_HOST/explore/data/ucx_cdrns/migrate_75zn9 17:17 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_cdrns', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cdrns.migrate_75zn9', metastore_id=None, name='migrate_75zn9', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 17:17 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_svvej: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_svvej 17:17 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_svvej', metastore_id=None, name='ucx_svvej', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 17:18 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.rRsr/config.yml) doesn't exist. 17:18 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration 17:19 INFO [databricks.labs.ucx.installer.hms_lineage] HMS Lineage feature creates one system table named system.hms_to_uc_migration.table_access and helps in your migration process from HMS to UC by allowing you to programmatically query HMS lineage data. 17:19 INFO [databricks.labs.ucx.install] Fetching installations... 17:19 WARNING [databricks.labs.ucx.install] Existing installation at /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.rRsr is corrupted. Skipping... 17:19 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy. 17:19 DEBUG [tests.integration.conftest] Waiting for clusters to start... 17:19 DEBUG [tests.integration.conftest] Waiting for clusters to start... 17:19 INFO [databricks.labs.ucx.install] Installing UCX v0.27.2+6020240702171924 17:19 INFO [databricks.labs.ucx.install] Creating ucx schemas... 17:20 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=failing 17:20 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups 17:20 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=scan-tables-in-mounts-experimental 17:20 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-data-reconciliation 17:20 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-experimental 17:20 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-tables-ctas 17:20 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental 17:20 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment 17:20 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups 17:20 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=experimental-workflow-linter 17:20 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables 17:20 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions 17:20 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-hiveserde-tables-in-place-experimental 17:20 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.rRsr/README for the next steps. 17:20 DEBUG [databricks.labs.ucx.installer.workflows] starting migrate-tables job: https://DATABRICKS_HOST#job/213377662691383 17:14 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/DSaW 17:14 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.migrate_75zn9: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_75zn9 17:14 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.migrate_75zn9', metastore_id=None, name='migrate_75zn9', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 17:15 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_75zn9.ucx_tnzeu: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_75zn9/ucx_tnzeu 17:15 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_75zn9.ucx_tnzeu', metastore_id=None, name='ucx_tnzeu', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070218'}, row_filter=None, schema_name='migrate_75zn9', sql_path=None, storage_credential_name=None, storage_location='dbfs:/user/hive/warehouse/migrate_75zn9/ucx_tnzeu', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 17:15 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_75zn9.ucx_togpb: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_75zn9/ucx_togpb 17:15 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_75zn9.ucx_togpb', metastore_id=None, name='ucx_togpb', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070218'}, row_filter=None, schema_name='migrate_75zn9', sql_path=None, storage_credential_name=None, storage_location='dbfs:/tmp/ucx_test_U8sA', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 17:15 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_75zn9.ucx_tq0di: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_75zn9/ucx_tq0di 17:15 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_75zn9.ucx_tq0di', metastore_id=None, name='ucx_tq0di', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070218'}, row_filter=None, schema_name='migrate_75zn9', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/b/DSaW', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 17:16 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_75zn9.ucx_tafdm: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_75zn9/ucx_tafdm 17:16 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_75zn9.ucx_tafdm', metastore_id=None, name='ucx_tafdm', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070218'}, row_filter=None, schema_name='migrate_75zn9', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_75zn9.ucx_tnzeu', view_dependencies=None) 17:16 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_75zn9.ucx_tkjfm: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_75zn9/ucx_tkjfm 17:16 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_75zn9.ucx_tkjfm', metastore_id=None, name='ucx_tkjfm', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070218'}, row_filter=None, schema_name='migrate_75zn9', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_75zn9.ucx_tafdm', view_dependencies=None) 17:17 DEBUG [databricks.labs.ucx.mixins.fixtures] added catalog fixture: CatalogInfo(browse_only=False, catalog_type=, comment='', connection_name=None, created_at=1719940623841, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cdrns', isolation_mode=, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_cdrns', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1719940623841, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d') 17:17 INFO [databricks.labs.ucx.mixins.fixtures] Schema ucx_cdrns.migrate_75zn9: https://DATABRICKS_HOST/explore/data/ucx_cdrns/migrate_75zn9 17:17 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_cdrns', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cdrns.migrate_75zn9', metastore_id=None, name='migrate_75zn9', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 17:17 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_svvej: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_svvej 17:17 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_svvej', metastore_id=None, name='ucx_svvej', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 17:18 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.rRsr/config.yml) doesn't exist. 17:18 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration 17:19 INFO [databricks.labs.ucx.installer.hms_lineage] HMS Lineage feature creates one system table named system.hms_to_uc_migration.table_access and helps in your migration process from HMS to UC by allowing you to programmatically query HMS lineage data. 17:19 INFO [databricks.labs.ucx.install] Fetching installations... 17:19 WARNING [databricks.labs.ucx.install] Existing installation at /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.rRsr is corrupted. Skipping... 17:19 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy. 17:19 DEBUG [tests.integration.conftest] Waiting for clusters to start... 17:19 DEBUG [tests.integration.conftest] Waiting for clusters to start... 17:19 INFO [databricks.labs.ucx.install] Installing UCX v0.27.2+6020240702171924 17:19 INFO [databricks.labs.ucx.install] Creating ucx schemas... 17:20 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=failing 17:20 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups 17:20 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=scan-tables-in-mounts-experimental 17:20 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-data-reconciliation 17:20 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-experimental 17:20 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-tables-ctas 17:20 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental 17:20 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment 17:20 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups 17:20 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=experimental-workflow-linter 17:20 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables 17:20 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions 17:20 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-hiveserde-tables-in-place-experimental 17:20 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.rRsr/README for the next steps. 17:20 DEBUG [databricks.labs.ucx.installer.workflows] starting migrate-tables job: https://DATABRICKS_HOST#job/213377662691383 17:26 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 make_storage_dir fixtures 17:26 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 make_dbfs_data_copy fixtures 17:26 DEBUG [databricks.labs.ucx.mixins.fixtures] removing make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/DSaW 17:26 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 cluster fixtures 17:26 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 catalog fixtures 17:26 DEBUG [databricks.labs.ucx.mixins.fixtures] removing catalog fixture: CatalogInfo(browse_only=False, catalog_type=, comment='', connection_name=None, created_at=1719940623841, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cdrns', isolation_mode=, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_cdrns', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1719940623841, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d') 17:26 INFO [databricks.labs.ucx.install] Deleting UCX v0.27.2+6020240702171924 from https://DATABRICKS_HOST 17:26 INFO [databricks.labs.ucx.install] Deleting inventory database ucx_svvej 17:26 INFO [databricks.labs.ucx.install] Deleting jobs 17:26 INFO [databricks.labs.ucx.install] Deleting failing job_id=378876543871563. 17:26 INFO [databricks.labs.ucx.install] Deleting migrate-groups job_id=495580131839094. 17:26 INFO [databricks.labs.ucx.install] Deleting scan-tables-in-mounts-experimental job_id=901471361410736. 17:26 INFO [databricks.labs.ucx.install] Deleting migrate-data-reconciliation job_id=686915452022449. 17:26 INFO [databricks.labs.ucx.install] Deleting migrate-groups-experimental job_id=678407480146698. 17:26 INFO [databricks.labs.ucx.install] Deleting migrate-external-tables-ctas job_id=888168290551622. 17:26 INFO [databricks.labs.ucx.install] Deleting migrate-tables-in-mounts-experimental job_id=514017254878961. 17:26 INFO [databricks.labs.ucx.install] Deleting assessment job_id=678036895199687. 17:26 INFO [databricks.labs.ucx.install] Deleting remove-workspace-local-backup-groups job_id=436468672430603. 17:26 INFO [databricks.labs.ucx.install] Deleting experimental-workflow-linter job_id=959957621721702. 17:26 INFO [databricks.labs.ucx.install] Deleting migrate-tables job_id=213377662691383. 17:26 INFO [databricks.labs.ucx.install] Deleting validate-groups-permissions job_id=1032649123038499. 17:26 INFO [databricks.labs.ucx.install] Deleting migrate-external-hiveserde-tables-in-place-experimental job_id=41227504722938. 17:26 INFO [databricks.labs.ucx.install] Deleting cluster policy 17:26 INFO [databricks.labs.ucx.install] Deleting secret scope 17:26 INFO [databricks.labs.ucx.install] UnInstalling UCX complete 17:26 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 workspace user fixtures 17:26 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 account group fixtures 17:26 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 workspace group fixtures 17:26 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 table fixtures 17:26 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 5 table fixtures 17:26 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_75zn9.ucx_tnzeu', metastore_id=None, name='ucx_tnzeu', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070218'}, row_filter=None, schema_name='migrate_75zn9', sql_path=None, storage_credential_name=None, storage_location='dbfs:/user/hive/warehouse/migrate_75zn9/ucx_tnzeu', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 17:26 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_75zn9.ucx_togpb', metastore_id=None, name='ucx_togpb', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070218'}, row_filter=None, schema_name='migrate_75zn9', sql_path=None, storage_credential_name=None, storage_location='dbfs:/tmp/ucx_test_U8sA', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 17:27 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_75zn9.ucx_tq0di', metastore_id=None, name='ucx_tq0di', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070218'}, row_filter=None, schema_name='migrate_75zn9', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/b/DSaW', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 17:27 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_75zn9.ucx_tafdm', metastore_id=None, name='ucx_tafdm', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070218'}, row_filter=None, schema_name='migrate_75zn9', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_75zn9.ucx_tnzeu', view_dependencies=None) 17:27 DEBUG [databricks.labs.ucx.mixins.fixtures] ignoring error while table TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_75zn9.ucx_tafdm', metastore_id=None, name='ucx_tafdm', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070218'}, row_filter=None, schema_name='migrate_75zn9', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_75zn9.ucx_tnzeu', view_dependencies=None) teardown: [WRONG_COMMAND_FOR_OBJECT_TYPE] The operation DROP TABLE requires a EXTERNAL or MANAGED. But hive_metastore.migrate_75zn9.ucx_tafdm is a VIEW. Use DROP VIEW instead. SQLSTATE: 42809 17:27 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_75zn9.ucx_tkjfm', metastore_id=None, name='ucx_tkjfm', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070218'}, row_filter=None, schema_name='migrate_75zn9', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_75zn9.ucx_tafdm', view_dependencies=None) 17:27 DEBUG [databricks.labs.ucx.mixins.fixtures] ignoring error while table TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_75zn9.ucx_tkjfm', metastore_id=None, name='ucx_tkjfm', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070218'}, row_filter=None, schema_name='migrate_75zn9', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_75zn9.ucx_tafdm', view_dependencies=None) teardown: [WRONG_COMMAND_FOR_OBJECT_TYPE] The operation DROP TABLE requires a EXTERNAL or MANAGED. But hive_metastore.migrate_75zn9.ucx_tkjfm is a VIEW. Use DROP VIEW instead. SQLSTATE: 42809 17:27 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 3 schema fixtures 17:27 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.migrate_75zn9', metastore_id=None, name='migrate_75zn9', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 17:27 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_cdrns', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cdrns.migrate_75zn9', metastore_id=None, name='migrate_75zn9', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 17:27 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_svvej', metastore_id=None, name='ucx_svvej', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) [gw7] linux -- Python 3.10.14 /home/runner/work/ucx/ucx/.venv/bin/python ```

Running from acceptance #4202

ericvergnaud commented 2 days ago

@nfx noise removed

ericvergnaud commented 1 day ago

already merged somehow...