search

databrickslabs / ucx

Your best companion for upgrading to Unity Catalog. UCX will guide you, the Databricks customer, through the process of upgrading your account, groups, workspaces, jobs etc. to Unity Catalog.
Other
194 stars 69 forks source link

whitelist prophet #2032

Closed ericvergnaud closed 2 days ago

ericvergnaud commented 4 days ago

Changes

whitelist prophet

Linked issues

Progresses #1901

Functionality

None

Tests

github-actions[bot] commented 4 days ago

❌ 91/95 passed, 4 failed, 21 skipped, 4h57m15s total

❌ test_create_catalog_schema_with_principal_acl_CLOUD_ENV: AssertionError: assert PrivilegeAssignment(principal='sdk-rwaazvru@example.com', privileges=[]) in [PrivilegeAssignment(principal='sdk-g0i4wcku@example.com', privileges=[]), Privile...HEMA'>]), PrivilegeAssignment(principal='sdk-kj5bej3n@example.com', privileges=[])] (9m33.354s) ``` AssertionError: assert PrivilegeAssignment(principal='sdk-rwaazvru@example.com', privileges=[]) in [PrivilegeAssignment(principal='sdk-g0i4wcku@example.com', privileges=[]), Privile...HEMA'>]), PrivilegeAssignment(principal='sdk-kj5bej3n@example.com', privileges=[])] + where [PrivilegeAssignment(principal='sdk-g0i4wcku@example.com', privileges=[]), Privile...HEMA'>]), PrivilegeAssignment(principal='sdk-kj5bej3n@example.com', privileges=[])] = PermissionsList(privilege_assignments=[PrivilegeAssignment(principal='sdk-g0i4wcku@example.com', privileges=[]), PrivilegeAssignment(principal='sdk-kj5bej3n@example.com', privileges=[])]).privilege_assignments 10:13 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/7Dt0 10:13 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_sr1ln: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_sr1ln 10:13 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_sr1ln', metastore_id=None, name='ucx_sr1ln', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 10:13 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.ucx_sr1ln.ucx_twm4s: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_sr1ln/ucx_twm4s 10:13 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.ucx_sr1ln.ucx_twm4s', metastore_id=None, name='ucx_twm4s', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070311'}, row_filter=None, schema_name='ucx_sr1ln', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/b/7Dt0', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 10:14 DEBUG [databricks.labs.ucx.mixins.fixtures] added catalog fixture: CatalogInfo(browse_only=False, catalog_type=, comment='', connection_name=None, created_at=1720001646982, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_c8trc', isolation_mode=, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_c8trc', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1720001646982, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d') 10:14 INFO [databricks.labs.ucx.mixins.fixtures] Schema ucx_c8trc.ucx_sr1ln: https://DATABRICKS_HOST/explore/data/ucx_c8trc/ucx_sr1ln 10:14 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_c8trc', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_c8trc.ucx_sr1ln', metastore_id=None, name='ucx_sr1ln', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) [gw9] linux -- Python 3.10.14 /home/runner/work/ucx/ucx/.venv/bin/python 10:13 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/7Dt0 10:13 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_sr1ln: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_sr1ln 10:13 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_sr1ln', metastore_id=None, name='ucx_sr1ln', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 10:13 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.ucx_sr1ln.ucx_twm4s: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_sr1ln/ucx_twm4s 10:13 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.ucx_sr1ln.ucx_twm4s', metastore_id=None, name='ucx_twm4s', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070311'}, row_filter=None, schema_name='ucx_sr1ln', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/b/7Dt0', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 10:14 DEBUG [databricks.labs.ucx.mixins.fixtures] added catalog fixture: CatalogInfo(browse_only=False, catalog_type=, comment='', connection_name=None, created_at=1720001646982, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_c8trc', isolation_mode=, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_c8trc', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1720001646982, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d') 10:14 INFO [databricks.labs.ucx.mixins.fixtures] Schema ucx_c8trc.ucx_sr1ln: https://DATABRICKS_HOST/explore/data/ucx_c8trc/ucx_sr1ln 10:14 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_c8trc', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_c8trc.ucx_sr1ln', metastore_id=None, name='ucx_sr1ln', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 10:14 DEBUG [databricks.labs.ucx.mixins.fixtures] added cluster fixture: 10:14 DEBUG [databricks.labs.ucx.mixins.fixtures] added workspace user fixture: User(active=True, display_name='sdk-kj5bej3n@example.com', emails=[ComplexValue(display=None, primary=True, ref=None, type='work', value='sdk-kj5bej3n@example.com')], entitlements=[], external_id=None, groups=[], id='4904734513274162', name=Name(family_name=None, given_name='sdk-kj5bej3n@example.com'), roles=[], schemas=[, ], user_name='sdk-kj5bej3n@example.com') 10:15 DEBUG [databricks.labs.ucx.mixins.fixtures] added cluster permissions fixture: 0703-101447-9hjyqzs7 [group_name admins CAN_MANAGE] -> [user_name sdk-kj5bej3n@example.com CAN_ATTACH_TO] 10:15 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_sgixg: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_sgixg 10:15 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_sgixg', metastore_id=None, name='ucx_sgixg', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 10:15 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 10:15 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 10:15 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 10:15 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 10:15 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 10:15 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 10:15 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 10:15 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 10:15 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 10:15 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 10:15 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 10:15 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 10:15 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 10:15 INFO [databricks.labs.ucx.hive_metastore.catalog_schema] Listing existing UC catalogs and schemas 10:15 DEBUG [databricks.labs.ucx.mixins.fixtures] added cluster fixture: 10:15 DEBUG [databricks.labs.ucx.mixins.fixtures] added workspace user fixture: User(active=True, display_name='sdk-frxbvk99@example.com', emails=[ComplexValue(display=None, primary=True, ref=None, type='work', value='sdk-frxbvk99@example.com')], entitlements=[], external_id=None, groups=[], id='5192879253816315', name=Name(family_name=None, given_name='sdk-frxbvk99@example.com'), roles=[], schemas=[, ], user_name='sdk-frxbvk99@example.com') 10:15 DEBUG [databricks.labs.ucx.mixins.fixtures] added cluster permissions fixture: 0703-101522-e1ikqyp7 [group_name admins CAN_MANAGE] -> [user_name sdk-frxbvk99@example.com CAN_ATTACH_TO] 10:15 INFO [databricks.labs.ucx.hive_metastore.catalog_schema] Listing existing UC catalogs and schemas 10:15 DEBUG [databricks.labs.ucx.mixins.fixtures] added cluster fixture: 10:15 DEBUG [databricks.labs.ucx.mixins.fixtures] added workspace user fixture: User(active=True, display_name='sdk-6ebpomzj@example.com', emails=[ComplexValue(display=None, primary=True, ref=None, type='work', value='sdk-6ebpomzj@example.com')], entitlements=[], external_id=None, groups=[], id='6415533911769308', name=Name(family_name=None, given_name='sdk-6ebpomzj@example.com'), roles=[], schemas=[, ], user_name='sdk-6ebpomzj@example.com') 10:15 DEBUG [databricks.labs.ucx.mixins.fixtures] added cluster permissions fixture: 0703-101544-gyd9y9kc [group_name admins CAN_MANAGE] -> [user_name sdk-6ebpomzj@example.com CAN_ATTACH_TO] 10:15 INFO [databricks.labs.ucx.hive_metastore.catalog_schema] Listing existing UC catalogs and schemas 10:16 DEBUG [databricks.labs.ucx.mixins.fixtures] added cluster fixture: 10:16 DEBUG [databricks.labs.ucx.mixins.fixtures] added workspace user fixture: User(active=True, display_name='sdk-rwaazvru@example.com', emails=[ComplexValue(display=None, primary=True, ref=None, type='work', value='sdk-rwaazvru@example.com')], entitlements=[], external_id=None, groups=[], id='4018278560344619', name=Name(family_name=None, given_name='sdk-rwaazvru@example.com'), roles=[], schemas=[, ], user_name='sdk-rwaazvru@example.com') 10:16 DEBUG [databricks.labs.ucx.mixins.fixtures] added cluster permissions fixture: 0703-101607-ueh79hn5 [group_name admins CAN_MANAGE] -> [user_name sdk-rwaazvru@example.com CAN_ATTACH_TO] 10:16 INFO [databricks.labs.ucx.hive_metastore.catalog_schema] Listing existing UC catalogs and schemas 10:16 INFO [databricks.labs.ucx.hive_metastore.catalog_schema] Preparing a list of UC catalogs and schema to be created 10:16 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.ucx_sgixg.tables] fetching tables inventory 10:16 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.ucx_sgixg.tables] crawling new batch for tables 10:16 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sr1ln] listing tables 10:16 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sr1ln.ucx_twm4s] fetching table metadata 10:17 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.ucx_sgixg.tables] found 1 new records for tables 10:18 DEBUG [databricks.labs.ucx.hive_metastore.catalog_schema] Migrating acls on ('DATABASE', 'ucx_c8trc.ucx_sr1ln') using SQL query: GRANT USE SCHEMA ON DATABASE ucx_c8trc.ucx_sr1ln TO `5a11359f-ba1f-483f-8e00-0fe55ec003ed` 10:19 DEBUG [databricks.labs.ucx.hive_metastore.catalog_schema] Migrating acls on ('DATABASE', 'ucx_c8trc.ucx_sr1ln') using SQL query: GRANT USE SCHEMA ON DATABASE ucx_c8trc.ucx_sr1ln TO `sdk-kj5bej3n@example.com` 10:19 DEBUG [databricks.labs.ucx.hive_metastore.catalog_schema] Migrating acls on ('DATABASE', 'ucx_c8trc.ucx_sr1ln') using SQL query: GRANT USE SCHEMA ON DATABASE ucx_c8trc.ucx_sr1ln TO `0a330eb5-dd51-4d97-b6e4-c474356b1d5d` 10:19 DEBUG [databricks.labs.ucx.hive_metastore.catalog_schema] Migrating acls on ('DATABASE', 'ucx_c8trc.ucx_sr1ln') using SQL query: GRANT USE SCHEMA ON DATABASE ucx_c8trc.ucx_sr1ln TO `serge.smertin@databricks.com` 10:20 DEBUG [databricks.labs.ucx.hive_metastore.catalog_schema] Migrating acls on ('DATABASE', 'ucx_c8trc.ucx_sr1ln') using SQL query: GRANT USE SCHEMA ON DATABASE ucx_c8trc.ucx_sr1ln TO `sdk-g0i4wcku@example.com` 10:20 DEBUG [databricks.labs.ucx.hive_metastore.catalog_schema] Migrating acls on ('DATABASE', 'ucx_c8trc.ucx_sr1ln') using SQL query: GRANT USE SCHEMA ON DATABASE ucx_c8trc.ucx_sr1ln TO `sdk-jcdsocp8@example.com` 10:20 DEBUG [databricks.labs.ucx.hive_metastore.catalog_schema] Migrating acls on ('CATALOG', 'ucx_c8trc') using SQL query: GRANT USE CATALOG ON CATALOG ucx_c8trc TO `0a330eb5-dd51-4d97-b6e4-c474356b1d5d` 10:20 DEBUG [databricks.labs.ucx.hive_metastore.catalog_schema] Migrating acls on ('CATALOG', 'ucx_c8trc') using SQL query: GRANT USE CATALOG ON CATALOG ucx_c8trc TO `serge.smertin@databricks.com` 10:20 DEBUG [databricks.labs.ucx.hive_metastore.catalog_schema] Migrating acls on ('CATALOG', 'ucx_c8trc') using SQL query: GRANT USE CATALOG ON CATALOG ucx_c8trc TO `sdk-kj5bej3n@example.com` 10:20 DEBUG [databricks.labs.ucx.hive_metastore.catalog_schema] Migrating acls on ('CATALOG', 'ucx_c8trc') using SQL query: GRANT USE CATALOG ON CATALOG ucx_c8trc TO `sdk-g0i4wcku@example.com` 10:21 DEBUG [databricks.labs.ucx.hive_metastore.catalog_schema] Migrating acls on ('CATALOG', 'ucx_c8trc') using SQL query: GRANT USE CATALOG ON CATALOG ucx_c8trc TO `5a11359f-ba1f-483f-8e00-0fe55ec003ed` 10:21 DEBUG [databricks.labs.ucx.hive_metastore.catalog_schema] Migrating acls on ('CATALOG', 'ucx_c8trc') using SQL query: GRANT USE CATALOG ON CATALOG ucx_c8trc TO `sdk-jcdsocp8@example.com` 10:13 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/7Dt0 10:13 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_sr1ln: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_sr1ln 10:13 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_sr1ln', metastore_id=None, name='ucx_sr1ln', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 10:13 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.ucx_sr1ln.ucx_twm4s: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_sr1ln/ucx_twm4s 10:13 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.ucx_sr1ln.ucx_twm4s', metastore_id=None, name='ucx_twm4s', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070311'}, row_filter=None, schema_name='ucx_sr1ln', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/b/7Dt0', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 10:14 DEBUG [databricks.labs.ucx.mixins.fixtures] added catalog fixture: CatalogInfo(browse_only=False, catalog_type=, comment='', connection_name=None, created_at=1720001646982, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_c8trc', isolation_mode=, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_c8trc', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1720001646982, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d') 10:14 INFO [databricks.labs.ucx.mixins.fixtures] Schema ucx_c8trc.ucx_sr1ln: https://DATABRICKS_HOST/explore/data/ucx_c8trc/ucx_sr1ln 10:14 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_c8trc', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_c8trc.ucx_sr1ln', metastore_id=None, name='ucx_sr1ln', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 10:14 DEBUG [databricks.labs.ucx.mixins.fixtures] added cluster fixture: 10:14 DEBUG [databricks.labs.ucx.mixins.fixtures] added workspace user fixture: User(active=True, display_name='sdk-kj5bej3n@example.com', emails=[ComplexValue(display=None, primary=True, ref=None, type='work', value='sdk-kj5bej3n@example.com')], entitlements=[], external_id=None, groups=[], id='4904734513274162', name=Name(family_name=None, given_name='sdk-kj5bej3n@example.com'), roles=[], schemas=[, ], user_name='sdk-kj5bej3n@example.com') 10:15 DEBUG [databricks.labs.ucx.mixins.fixtures] added cluster permissions fixture: 0703-101447-9hjyqzs7 [group_name admins CAN_MANAGE] -> [user_name sdk-kj5bej3n@example.com CAN_ATTACH_TO] 10:15 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_sgixg: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_sgixg 10:15 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_sgixg', metastore_id=None, name='ucx_sgixg', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 10:15 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 10:15 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 10:15 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 10:15 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 10:15 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 10:15 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 10:15 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 10:15 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 10:15 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 10:15 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 10:15 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 10:15 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 10:15 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext. 10:15 INFO [databricks.labs.ucx.hive_metastore.catalog_schema] Listing existing UC catalogs and schemas 10:15 DEBUG [databricks.labs.ucx.mixins.fixtures] added cluster fixture: 10:15 DEBUG [databricks.labs.ucx.mixins.fixtures] added workspace user fixture: User(active=True, display_name='sdk-frxbvk99@example.com', emails=[ComplexValue(display=None, primary=True, ref=None, type='work', value='sdk-frxbvk99@example.com')], entitlements=[], external_id=None, groups=[], id='5192879253816315', name=Name(family_name=None, given_name='sdk-frxbvk99@example.com'), roles=[], schemas=[, ], user_name='sdk-frxbvk99@example.com') 10:15 DEBUG [databricks.labs.ucx.mixins.fixtures] added cluster permissions fixture: 0703-101522-e1ikqyp7 [group_name admins CAN_MANAGE] -> [user_name sdk-frxbvk99@example.com CAN_ATTACH_TO] 10:15 INFO [databricks.labs.ucx.hive_metastore.catalog_schema] Listing existing UC catalogs and schemas 10:15 DEBUG [databricks.labs.ucx.mixins.fixtures] added cluster fixture: 10:15 DEBUG [databricks.labs.ucx.mixins.fixtures] added workspace user fixture: User(active=True, display_name='sdk-6ebpomzj@example.com', emails=[ComplexValue(display=None, primary=True, ref=None, type='work', value='sdk-6ebpomzj@example.com')], entitlements=[], external_id=None, groups=[], id='6415533911769308', name=Name(family_name=None, given_name='sdk-6ebpomzj@example.com'), roles=[], schemas=[, ], user_name='sdk-6ebpomzj@example.com') 10:15 DEBUG [databricks.labs.ucx.mixins.fixtures] added cluster permissions fixture: 0703-101544-gyd9y9kc [group_name admins CAN_MANAGE] -> [user_name sdk-6ebpomzj@example.com CAN_ATTACH_TO] 10:15 INFO [databricks.labs.ucx.hive_metastore.catalog_schema] Listing existing UC catalogs and schemas 10:16 DEBUG [databricks.labs.ucx.mixins.fixtures] added cluster fixture: 10:16 DEBUG [databricks.labs.ucx.mixins.fixtures] added workspace user fixture: User(active=True, display_name='sdk-rwaazvru@example.com', emails=[ComplexValue(display=None, primary=True, ref=None, type='work', value='sdk-rwaazvru@example.com')], entitlements=[], external_id=None, groups=[], id='4018278560344619', name=Name(family_name=None, given_name='sdk-rwaazvru@example.com'), roles=[], schemas=[, ], user_name='sdk-rwaazvru@example.com') 10:16 DEBUG [databricks.labs.ucx.mixins.fixtures] added cluster permissions fixture: 0703-101607-ueh79hn5 [group_name admins CAN_MANAGE] -> [user_name sdk-rwaazvru@example.com CAN_ATTACH_TO] 10:16 INFO [databricks.labs.ucx.hive_metastore.catalog_schema] Listing existing UC catalogs and schemas 10:16 INFO [databricks.labs.ucx.hive_metastore.catalog_schema] Preparing a list of UC catalogs and schema to be created 10:16 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.ucx_sgixg.tables] fetching tables inventory 10:16 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.ucx_sgixg.tables] crawling new batch for tables 10:16 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sr1ln] listing tables 10:16 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sr1ln.ucx_twm4s] fetching table metadata 10:17 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.ucx_sgixg.tables] found 1 new records for tables 10:18 DEBUG [databricks.labs.ucx.hive_metastore.catalog_schema] Migrating acls on ('DATABASE', 'ucx_c8trc.ucx_sr1ln') using SQL query: GRANT USE SCHEMA ON DATABASE ucx_c8trc.ucx_sr1ln TO `5a11359f-ba1f-483f-8e00-0fe55ec003ed` 10:19 DEBUG [databricks.labs.ucx.hive_metastore.catalog_schema] Migrating acls on ('DATABASE', 'ucx_c8trc.ucx_sr1ln') using SQL query: GRANT USE SCHEMA ON DATABASE ucx_c8trc.ucx_sr1ln TO `sdk-kj5bej3n@example.com` 10:19 DEBUG [databricks.labs.ucx.hive_metastore.catalog_schema] Migrating acls on ('DATABASE', 'ucx_c8trc.ucx_sr1ln') using SQL query: GRANT USE SCHEMA ON DATABASE ucx_c8trc.ucx_sr1ln TO `0a330eb5-dd51-4d97-b6e4-c474356b1d5d` 10:19 DEBUG [databricks.labs.ucx.hive_metastore.catalog_schema] Migrating acls on ('DATABASE', 'ucx_c8trc.ucx_sr1ln') using SQL query: GRANT USE SCHEMA ON DATABASE ucx_c8trc.ucx_sr1ln TO `serge.smertin@databricks.com` 10:20 DEBUG [databricks.labs.ucx.hive_metastore.catalog_schema] Migrating acls on ('DATABASE', 'ucx_c8trc.ucx_sr1ln') using SQL query: GRANT USE SCHEMA ON DATABASE ucx_c8trc.ucx_sr1ln TO `sdk-g0i4wcku@example.com` 10:20 DEBUG [databricks.labs.ucx.hive_metastore.catalog_schema] Migrating acls on ('DATABASE', 'ucx_c8trc.ucx_sr1ln') using SQL query: GRANT USE SCHEMA ON DATABASE ucx_c8trc.ucx_sr1ln TO `sdk-jcdsocp8@example.com` 10:20 DEBUG [databricks.labs.ucx.hive_metastore.catalog_schema] Migrating acls on ('CATALOG', 'ucx_c8trc') using SQL query: GRANT USE CATALOG ON CATALOG ucx_c8trc TO `0a330eb5-dd51-4d97-b6e4-c474356b1d5d` 10:20 DEBUG [databricks.labs.ucx.hive_metastore.catalog_schema] Migrating acls on ('CATALOG', 'ucx_c8trc') using SQL query: GRANT USE CATALOG ON CATALOG ucx_c8trc TO `serge.smertin@databricks.com` 10:20 DEBUG [databricks.labs.ucx.hive_metastore.catalog_schema] Migrating acls on ('CATALOG', 'ucx_c8trc') using SQL query: GRANT USE CATALOG ON CATALOG ucx_c8trc TO `sdk-kj5bej3n@example.com` 10:20 DEBUG [databricks.labs.ucx.hive_metastore.catalog_schema] Migrating acls on ('CATALOG', 'ucx_c8trc') using SQL query: GRANT USE CATALOG ON CATALOG ucx_c8trc TO `sdk-g0i4wcku@example.com` 10:21 DEBUG [databricks.labs.ucx.hive_metastore.catalog_schema] Migrating acls on ('CATALOG', 'ucx_c8trc') using SQL query: GRANT USE CATALOG ON CATALOG ucx_c8trc TO `5a11359f-ba1f-483f-8e00-0fe55ec003ed` 10:21 DEBUG [databricks.labs.ucx.hive_metastore.catalog_schema] Migrating acls on ('CATALOG', 'ucx_c8trc') using SQL query: GRANT USE CATALOG ON CATALOG ucx_c8trc TO `sdk-jcdsocp8@example.com` 10:21 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 4 cluster permissions fixtures 10:21 DEBUG [databricks.labs.ucx.mixins.fixtures] removing cluster permissions fixture: 0703-101447-9hjyqzs7 [group_name admins CAN_MANAGE] -> [user_name sdk-kj5bej3n@example.com CAN_ATTACH_TO] 10:21 DEBUG [databricks.labs.ucx.mixins.fixtures] removing cluster permissions fixture: 0703-101522-e1ikqyp7 [group_name admins CAN_MANAGE] -> [user_name sdk-frxbvk99@example.com CAN_ATTACH_TO] 10:21 DEBUG [databricks.labs.ucx.mixins.fixtures] removing cluster permissions fixture: 0703-101544-gyd9y9kc [group_name admins CAN_MANAGE] -> [user_name sdk-6ebpomzj@example.com CAN_ATTACH_TO] 10:21 DEBUG [databricks.labs.ucx.mixins.fixtures] removing cluster permissions fixture: 0703-101607-ueh79hn5 [group_name admins CAN_MANAGE] -> [user_name sdk-rwaazvru@example.com CAN_ATTACH_TO] 10:22 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 catalog fixtures 10:22 DEBUG [databricks.labs.ucx.mixins.fixtures] removing catalog fixture: CatalogInfo(browse_only=False, catalog_type=, comment='', connection_name=None, created_at=1720001646982, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_c8trc', isolation_mode=, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_c8trc', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1720001646982, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d') 10:22 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 make_dbfs_data_copy fixtures 10:22 DEBUG [databricks.labs.ucx.mixins.fixtures] removing make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/7Dt0 10:22 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 4 cluster fixtures 10:22 DEBUG [databricks.labs.ucx.mixins.fixtures] removing cluster fixture: 10:22 DEBUG [databricks.labs.ucx.mixins.fixtures] removing cluster fixture: 10:22 DEBUG [databricks.labs.ucx.mixins.fixtures] removing cluster fixture: 10:22 DEBUG [databricks.labs.ucx.mixins.fixtures] removing cluster fixture: 10:22 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 workspace group fixtures 10:22 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 table fixtures 10:22 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 table fixtures 10:22 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.ucx_sr1ln.ucx_twm4s', metastore_id=None, name='ucx_twm4s', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070311'}, row_filter=None, schema_name='ucx_sr1ln', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/b/7Dt0', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 10:22 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 3 schema fixtures 10:22 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_sr1ln', metastore_id=None, name='ucx_sr1ln', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 10:22 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_c8trc', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_c8trc.ucx_sr1ln', metastore_id=None, name='ucx_sr1ln', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 10:22 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_sgixg', metastore_id=None, name='ucx_sgixg', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 10:22 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 4 workspace user fixtures 10:22 DEBUG [databricks.labs.ucx.mixins.fixtures] removing workspace user fixture: User(active=True, display_name='sdk-kj5bej3n@example.com', emails=[ComplexValue(display=None, primary=True, ref=None, type='work', value='sdk-kj5bej3n@example.com')], entitlements=[], external_id=None, groups=[], id='4904734513274162', name=Name(family_name=None, given_name='sdk-kj5bej3n@example.com'), roles=[], schemas=[, ], user_name='sdk-kj5bej3n@example.com') 10:22 DEBUG [databricks.labs.ucx.mixins.fixtures] removing workspace user fixture: User(active=True, display_name='sdk-frxbvk99@example.com', emails=[ComplexValue(display=None, primary=True, ref=None, type='work', value='sdk-frxbvk99@example.com')], entitlements=[], external_id=None, groups=[], id='5192879253816315', name=Name(family_name=None, given_name='sdk-frxbvk99@example.com'), roles=[], schemas=[, ], user_name='sdk-frxbvk99@example.com') 10:22 DEBUG [databricks.labs.ucx.mixins.fixtures] removing workspace user fixture: User(active=True, display_name='sdk-6ebpomzj@example.com', emails=[ComplexValue(display=None, primary=True, ref=None, type='work', value='sdk-6ebpomzj@example.com')], entitlements=[], external_id=None, groups=[], id='6415533911769308', name=Name(family_name=None, given_name='sdk-6ebpomzj@example.com'), roles=[], schemas=[, ], user_name='sdk-6ebpomzj@example.com') 10:22 DEBUG [databricks.labs.ucx.mixins.fixtures] removing workspace user fixture: User(active=True, display_name='sdk-rwaazvru@example.com', emails=[ComplexValue(display=None, primary=True, ref=None, type='work', value='sdk-rwaazvru@example.com')], entitlements=[], external_id=None, groups=[], id='4018278560344619', name=Name(family_name=None, given_name='sdk-rwaazvru@example.com'), roles=[], schemas=[, ], user_name='sdk-rwaazvru@example.com') [gw9] linux -- Python 3.10.14 /home/runner/work/ucx/ucx/.venv/bin/python ```
❌ test_running_real_workflow_linter_job: TimeoutError: Timed out after 0:02:00 (10m55.226s) ``` TimeoutError: Timed out after 0:02:00 [gw0] linux -- Python 3.10.14 /home/runner/work/ucx/ucx/.venv/bin/python 10:33 DEBUG [databricks.labs.ucx.mixins.fixtures] added directory fixture: /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/sdk-mlle 10:33 DEBUG [databricks.labs.ucx.mixins.fixtures] added notebook fixture: /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/sdk-mlle/notebook.ipynb 10:33 INFO [databricks.labs.ucx.mixins.fixtures] Job: https://DATABRICKS_HOST#job/783740741491772 10:33 DEBUG [databricks.labs.ucx.mixins.fixtures] added job fixture: CreateResponse(job_id=783740741491772) 10:34 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_sx0tu: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_sx0tu 10:34 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_sx0tu', metastore_id=None, name='ucx_sx0tu', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 10:34 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.A75A/config.yml) doesn't exist. 10:34 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration 10:34 INFO [databricks.labs.ucx.installer.hms_lineage] HMS Lineage feature creates one system table named system.hms_to_uc_migration.table_access and helps in your migration process from HMS to UC by allowing you to programmatically query HMS lineage data. 10:34 INFO [databricks.labs.ucx.install] Fetching installations... 10:34 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy. 10:34 DEBUG [tests.integration.conftest] Waiting for clusters to start... 10:34 DEBUG [tests.integration.conftest] Waiting for clusters to start... 10:34 INFO [databricks.labs.ucx.install] Installing UCX v0.27.2+7020240703103437 10:34 INFO [databricks.labs.ucx.install] Creating ucx schemas... 10:37 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-hiveserde-tables-in-place-experimental 10:37 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups 10:37 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=scan-tables-in-mounts-experimental 10:37 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups 10:37 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-data-reconciliation 10:37 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=failing 10:37 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=experimental-workflow-linter 10:37 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions 10:37 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental 10:37 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-experimental 10:37 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables 10:37 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment 10:37 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-tables-ctas 10:37 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.A75A/README for the next steps. 10:37 DEBUG [databricks.labs.ucx.installer.workflows] starting experimental-workflow-linter job: https://DATABRICKS_HOST#job/151189960240657 10:41 DEBUG [databricks.labs.ucx.installer.workflows] Validating experimental-workflow-linter workflow: https://DATABRICKS_HOST#job/151189960240657 10:33 DEBUG [databricks.labs.ucx.mixins.fixtures] added directory fixture: /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/sdk-mlle 10:33 DEBUG [databricks.labs.ucx.mixins.fixtures] added notebook fixture: /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/sdk-mlle/notebook.ipynb 10:33 INFO [databricks.labs.ucx.mixins.fixtures] Job: https://DATABRICKS_HOST#job/783740741491772 10:33 DEBUG [databricks.labs.ucx.mixins.fixtures] added job fixture: CreateResponse(job_id=783740741491772) 10:34 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_sx0tu: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_sx0tu 10:34 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_sx0tu', metastore_id=None, name='ucx_sx0tu', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 10:34 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.A75A/config.yml) doesn't exist. 10:34 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration 10:34 INFO [databricks.labs.ucx.installer.hms_lineage] HMS Lineage feature creates one system table named system.hms_to_uc_migration.table_access and helps in your migration process from HMS to UC by allowing you to programmatically query HMS lineage data. 10:34 INFO [databricks.labs.ucx.install] Fetching installations... 10:34 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy. 10:34 DEBUG [tests.integration.conftest] Waiting for clusters to start... 10:34 DEBUG [tests.integration.conftest] Waiting for clusters to start... 10:34 INFO [databricks.labs.ucx.install] Installing UCX v0.27.2+7020240703103437 10:34 INFO [databricks.labs.ucx.install] Creating ucx schemas... 10:37 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-hiveserde-tables-in-place-experimental 10:37 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups 10:37 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=scan-tables-in-mounts-experimental 10:37 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups 10:37 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-data-reconciliation 10:37 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=failing 10:37 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=experimental-workflow-linter 10:37 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions 10:37 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental 10:37 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-experimental 10:37 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables 10:37 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment 10:37 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-tables-ctas 10:37 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.A75A/README for the next steps. 10:37 DEBUG [databricks.labs.ucx.installer.workflows] starting experimental-workflow-linter job: https://DATABRICKS_HOST#job/151189960240657 10:41 DEBUG [databricks.labs.ucx.installer.workflows] Validating experimental-workflow-linter workflow: https://DATABRICKS_HOST#job/151189960240657 10:41 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 job fixtures 10:41 DEBUG [databricks.labs.ucx.mixins.fixtures] removing job fixture: CreateResponse(job_id=783740741491772) 10:41 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 directory fixtures 10:41 DEBUG [databricks.labs.ucx.mixins.fixtures] removing directory fixture: /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/sdk-mlle 10:41 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 notebook fixtures 10:41 DEBUG [databricks.labs.ucx.mixins.fixtures] removing notebook fixture: /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/sdk-mlle/notebook.ipynb 10:41 DEBUG [databricks.labs.ucx.mixins.fixtures] ignoring error while notebook /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/sdk-mlle/notebook.ipynb teardown: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/sdk-mlle/notebook.ipynb) doesn't exist. 10:41 INFO [databricks.labs.ucx.install] Deleting UCX v0.27.2+7020240703103437 from https://DATABRICKS_HOST 10:41 INFO [databricks.labs.ucx.install] Deleting inventory database ucx_sx0tu 10:42 INFO [databricks.labs.ucx.install] Deleting jobs 10:42 INFO [databricks.labs.ucx.install] Deleting migrate-external-hiveserde-tables-in-place-experimental job_id=1122280434843297. 10:42 INFO [databricks.labs.ucx.install] Deleting migrate-groups job_id=374275984064352. 10:42 INFO [databricks.labs.ucx.install] Deleting scan-tables-in-mounts-experimental job_id=107296937081756. 10:42 INFO [databricks.labs.ucx.install] Deleting remove-workspace-local-backup-groups job_id=305884109932492. 10:42 INFO [databricks.labs.ucx.install] Deleting migrate-data-reconciliation job_id=1106978033565079. 10:42 INFO [databricks.labs.ucx.install] Deleting failing job_id=740765071659492. 10:42 INFO [databricks.labs.ucx.install] Deleting experimental-workflow-linter job_id=151189960240657. 10:42 INFO [databricks.labs.ucx.install] Deleting validate-groups-permissions job_id=368510185963795. 10:42 INFO [databricks.labs.ucx.install] Deleting migrate-tables-in-mounts-experimental job_id=716108869000403. 10:42 INFO [databricks.labs.ucx.install] Deleting migrate-groups-experimental job_id=792899628302334. 10:42 INFO [databricks.labs.ucx.install] Deleting migrate-tables job_id=504301347945275. 10:42 INFO [databricks.labs.ucx.install] Deleting assessment job_id=1093871305676123. 10:42 INFO [databricks.labs.ucx.install] Deleting migrate-external-tables-ctas job_id=963243963517652. 10:42 INFO [databricks.labs.ucx.install] Deleting cluster policy 10:42 INFO [databricks.labs.ucx.install] Deleting secret scope 10:42 INFO [databricks.labs.ucx.install] UnInstalling UCX complete 10:42 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 workspace user fixtures 10:42 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 account group fixtures 10:42 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 workspace group fixtures 10:42 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 table fixtures 10:42 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 table fixtures 10:42 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 schema fixtures 10:42 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_sx0tu', metastore_id=None, name='ucx_sx0tu', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) [gw0] linux -- Python 3.10.14 /home/runner/work/ucx/ucx/.venv/bin/python ```
❌ test_table_migration_job_refreshes_migration_status[hiveserde-migrate-external-tables-ctas]: TimeoutError: timed out after 0:20:00: (30m58.897s) ``` TimeoutError: timed out after 0:20:00: 10:18 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/hmI7 10:19 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.hiveserde_in_place_0ksca: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_0ksca 10:19 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.hiveserde_in_place_0ksca', metastore_id=None, name='hiveserde_in_place_0ksca', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 10:19 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_storage_dir fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_0ksca 10:19 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_0ksca.parquet_serde_0ksca: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_0ksca/parquet_serde_0ksca 10:19 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_0ksca.parquet_serde_0ksca', metastore_id=None, name='parquet_serde_0ksca', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070311'}, row_filter=None, schema_name='hiveserde_in_place_0ksca', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_0ksca/parquet_serde_0ksca', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 10:19 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_0ksca.orc_serde_0ksca: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_0ksca/orc_serde_0ksca 10:19 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_0ksca.orc_serde_0ksca', metastore_id=None, name='orc_serde_0ksca', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070311'}, row_filter=None, schema_name='hiveserde_in_place_0ksca', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_0ksca/orc_serde_0ksca', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 10:20 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_0ksca.avro_serde_0ksca: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_0ksca/avro_serde_0ksca 10:20 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_0ksca.avro_serde_0ksca', metastore_id=None, name='avro_serde_0ksca', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070311'}, row_filter=None, schema_name='hiveserde_in_place_0ksca', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_0ksca/avro_serde_0ksca', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 10:20 DEBUG [databricks.labs.ucx.mixins.fixtures] added catalog fixture: CatalogInfo(browse_only=False, catalog_type=, comment='', connection_name=None, created_at=1720002010116, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_c1uws', isolation_mode=, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_c1uws', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1720002010116, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d') 10:20 INFO [databricks.labs.ucx.mixins.fixtures] Schema ucx_c1uws.hiveserde_in_place_0ksca: https://DATABRICKS_HOST/explore/data/ucx_c1uws/hiveserde_in_place_0ksca 10:20 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_c1uws', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_c1uws.hiveserde_in_place_0ksca', metastore_id=None, name='hiveserde_in_place_0ksca', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 10:20 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_sribh: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_sribh 10:20 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_sribh', metastore_id=None, name='ucx_sribh', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) [gw7] linux -- Python 3.10.14 /home/runner/work/ucx/ucx/.venv/bin/python 10:18 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/hmI7 10:19 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.hiveserde_in_place_0ksca: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_0ksca 10:19 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.hiveserde_in_place_0ksca', metastore_id=None, name='hiveserde_in_place_0ksca', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 10:19 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_storage_dir fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_0ksca 10:19 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_0ksca.parquet_serde_0ksca: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_0ksca/parquet_serde_0ksca 10:19 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_0ksca.parquet_serde_0ksca', metastore_id=None, name='parquet_serde_0ksca', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070311'}, row_filter=None, schema_name='hiveserde_in_place_0ksca', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_0ksca/parquet_serde_0ksca', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 10:19 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_0ksca.orc_serde_0ksca: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_0ksca/orc_serde_0ksca 10:19 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_0ksca.orc_serde_0ksca', metastore_id=None, name='orc_serde_0ksca', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070311'}, row_filter=None, schema_name='hiveserde_in_place_0ksca', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_0ksca/orc_serde_0ksca', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 10:20 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_0ksca.avro_serde_0ksca: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_0ksca/avro_serde_0ksca 10:20 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_0ksca.avro_serde_0ksca', metastore_id=None, name='avro_serde_0ksca', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070311'}, row_filter=None, schema_name='hiveserde_in_place_0ksca', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_0ksca/avro_serde_0ksca', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 10:20 DEBUG [databricks.labs.ucx.mixins.fixtures] added catalog fixture: CatalogInfo(browse_only=False, catalog_type=, comment='', connection_name=None, created_at=1720002010116, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_c1uws', isolation_mode=, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_c1uws', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1720002010116, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d') 10:20 INFO [databricks.labs.ucx.mixins.fixtures] Schema ucx_c1uws.hiveserde_in_place_0ksca: https://DATABRICKS_HOST/explore/data/ucx_c1uws/hiveserde_in_place_0ksca 10:20 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_c1uws', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_c1uws.hiveserde_in_place_0ksca', metastore_id=None, name='hiveserde_in_place_0ksca', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 10:20 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_sribh: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_sribh 10:20 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_sribh', metastore_id=None, name='ucx_sribh', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 10:22 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.FyZC/config.yml) doesn't exist. 10:22 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration 10:22 INFO [databricks.labs.ucx.installer.hms_lineage] HMS Lineage feature creates one system table named system.hms_to_uc_migration.table_access and helps in your migration process from HMS to UC by allowing you to programmatically query HMS lineage data. 10:22 INFO [databricks.labs.ucx.install] Fetching installations... 10:22 WARNING [databricks.labs.ucx.install] Existing installation at /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.FyZC is corrupted. Skipping... 10:22 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy. 10:22 DEBUG [tests.integration.conftest] Waiting for clusters to start... 10:22 DEBUG [tests.integration.conftest] Waiting for clusters to start... 10:22 INFO [databricks.labs.ucx.install] Installing UCX v0.27.2+7020240703102241 10:22 INFO [databricks.labs.ucx.install] Creating ucx schemas... 10:25 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-data-reconciliation 10:25 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-hiveserde-tables-in-place-experimental 10:26 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-experimental 10:26 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-tables-ctas 10:26 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups 10:26 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables 10:26 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions 10:26 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups 10:26 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental 10:26 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=experimental-workflow-linter 10:26 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=failing 10:26 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment 10:26 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=scan-tables-in-mounts-experimental 10:26 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.FyZC/README for the next steps. 10:26 DEBUG [databricks.labs.ucx.installer.workflows] starting migrate-external-tables-ctas job: https://DATABRICKS_HOST#job/298762180646862 10:18 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/hmI7 10:19 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.hiveserde_in_place_0ksca: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_0ksca 10:19 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.hiveserde_in_place_0ksca', metastore_id=None, name='hiveserde_in_place_0ksca', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 10:19 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_storage_dir fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_0ksca 10:19 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_0ksca.parquet_serde_0ksca: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_0ksca/parquet_serde_0ksca 10:19 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_0ksca.parquet_serde_0ksca', metastore_id=None, name='parquet_serde_0ksca', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070311'}, row_filter=None, schema_name='hiveserde_in_place_0ksca', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_0ksca/parquet_serde_0ksca', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 10:19 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_0ksca.orc_serde_0ksca: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_0ksca/orc_serde_0ksca 10:19 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_0ksca.orc_serde_0ksca', metastore_id=None, name='orc_serde_0ksca', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070311'}, row_filter=None, schema_name='hiveserde_in_place_0ksca', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_0ksca/orc_serde_0ksca', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 10:20 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_0ksca.avro_serde_0ksca: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_0ksca/avro_serde_0ksca 10:20 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_0ksca.avro_serde_0ksca', metastore_id=None, name='avro_serde_0ksca', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070311'}, row_filter=None, schema_name='hiveserde_in_place_0ksca', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_0ksca/avro_serde_0ksca', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 10:20 DEBUG [databricks.labs.ucx.mixins.fixtures] added catalog fixture: CatalogInfo(browse_only=False, catalog_type=, comment='', connection_name=None, created_at=1720002010116, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_c1uws', isolation_mode=, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_c1uws', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1720002010116, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d') 10:20 INFO [databricks.labs.ucx.mixins.fixtures] Schema ucx_c1uws.hiveserde_in_place_0ksca: https://DATABRICKS_HOST/explore/data/ucx_c1uws/hiveserde_in_place_0ksca 10:20 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_c1uws', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_c1uws.hiveserde_in_place_0ksca', metastore_id=None, name='hiveserde_in_place_0ksca', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 10:20 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_sribh: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_sribh 10:20 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_sribh', metastore_id=None, name='ucx_sribh', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 10:22 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.FyZC/config.yml) doesn't exist. 10:22 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration 10:22 INFO [databricks.labs.ucx.installer.hms_lineage] HMS Lineage feature creates one system table named system.hms_to_uc_migration.table_access and helps in your migration process from HMS to UC by allowing you to programmatically query HMS lineage data. 10:22 INFO [databricks.labs.ucx.install] Fetching installations... 10:22 WARNING [databricks.labs.ucx.install] Existing installation at /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.FyZC is corrupted. Skipping... 10:22 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy. 10:22 DEBUG [tests.integration.conftest] Waiting for clusters to start... 10:22 DEBUG [tests.integration.conftest] Waiting for clusters to start... 10:22 INFO [databricks.labs.ucx.install] Installing UCX v0.27.2+7020240703102241 10:22 INFO [databricks.labs.ucx.install] Creating ucx schemas... 10:25 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-data-reconciliation 10:25 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-hiveserde-tables-in-place-experimental 10:26 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-experimental 10:26 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-tables-ctas 10:26 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups 10:26 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables 10:26 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions 10:26 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups 10:26 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental 10:26 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=experimental-workflow-linter 10:26 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=failing 10:26 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment 10:26 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=scan-tables-in-mounts-experimental 10:26 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.FyZC/README for the next steps. 10:26 DEBUG [databricks.labs.ucx.installer.workflows] starting migrate-external-tables-ctas job: https://DATABRICKS_HOST#job/298762180646862 10:46 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 make_storage_dir fixtures 10:46 DEBUG [databricks.labs.ucx.mixins.fixtures] removing make_storage_dir fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_0ksca 10:46 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 make_dbfs_data_copy fixtures 10:46 DEBUG [databricks.labs.ucx.mixins.fixtures] removing make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/hmI7 10:46 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 cluster fixtures 10:46 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 catalog fixtures 10:46 DEBUG [databricks.labs.ucx.mixins.fixtures] removing catalog fixture: CatalogInfo(browse_only=False, catalog_type=, comment='', connection_name=None, created_at=1720002010116, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_c1uws', isolation_mode=, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_c1uws', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1720002010116, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d') 10:46 INFO [databricks.labs.ucx.install] Deleting UCX v0.27.2+7020240703102241 from https://DATABRICKS_HOST 10:46 INFO [databricks.labs.ucx.install] Deleting inventory database ucx_sribh 10:46 INFO [databricks.labs.ucx.install] Deleting jobs 10:46 INFO [databricks.labs.ucx.install] Deleting migrate-data-reconciliation job_id=615327712840959. 10:46 INFO [databricks.labs.ucx.install] Deleting migrate-external-hiveserde-tables-in-place-experimental job_id=434866032309829. 10:46 INFO [databricks.labs.ucx.install] Deleting migrate-groups-experimental job_id=759406498739943. 10:46 INFO [databricks.labs.ucx.install] Deleting migrate-external-tables-ctas job_id=298762180646862. 10:46 INFO [databricks.labs.ucx.install] Deleting migrate-groups job_id=552943946678337. 10:46 INFO [databricks.labs.ucx.install] Deleting migrate-tables job_id=104068852787432. 10:46 INFO [databricks.labs.ucx.install] Deleting validate-groups-permissions job_id=602591221327419. 10:46 INFO [databricks.labs.ucx.install] Deleting remove-workspace-local-backup-groups job_id=718111740664588. 10:46 INFO [databricks.labs.ucx.install] Deleting migrate-tables-in-mounts-experimental job_id=927815857105121. 10:46 INFO [databricks.labs.ucx.install] Deleting experimental-workflow-linter job_id=637438495007399. 10:46 INFO [databricks.labs.ucx.install] Deleting failing job_id=553637315402536. 10:46 INFO [databricks.labs.ucx.install] Deleting assessment job_id=1077063013523495. 10:46 INFO [databricks.labs.ucx.install] Deleting scan-tables-in-mounts-experimental job_id=307797962829261. 10:46 INFO [databricks.labs.ucx.install] Deleting cluster policy 10:46 INFO [databricks.labs.ucx.install] Deleting secret scope 10:46 INFO [databricks.labs.ucx.install] UnInstalling UCX complete 10:46 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 workspace user fixtures 10:46 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 account group fixtures 10:46 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 workspace group fixtures 10:46 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 table fixtures 10:46 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 3 table fixtures 10:46 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_0ksca.parquet_serde_0ksca', metastore_id=None, name='parquet_serde_0ksca', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070311'}, row_filter=None, schema_name='hiveserde_in_place_0ksca', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_0ksca/parquet_serde_0ksca', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 10:47 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_0ksca.orc_serde_0ksca', metastore_id=None, name='orc_serde_0ksca', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070311'}, row_filter=None, schema_name='hiveserde_in_place_0ksca', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_0ksca/orc_serde_0ksca', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 10:47 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_0ksca.avro_serde_0ksca', metastore_id=None, name='avro_serde_0ksca', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070311'}, row_filter=None, schema_name='hiveserde_in_place_0ksca', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_0ksca/avro_serde_0ksca', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 10:48 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 3 schema fixtures 10:48 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.hiveserde_in_place_0ksca', metastore_id=None, name='hiveserde_in_place_0ksca', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 10:48 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_c1uws', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_c1uws.hiveserde_in_place_0ksca', metastore_id=None, name='hiveserde_in_place_0ksca', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 10:49 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_sribh', metastore_id=None, name='ucx_sribh', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) [gw7] linux -- Python 3.10.14 /home/runner/work/ucx/ucx/.venv/bin/python ```
❌ test_table_migration_job_refreshes_migration_status[regular-migrate-tables]: TimeoutError: Timed out after 0:05:00 (19m13.87s) ``` TimeoutError: Timed out after 0:05:00 10:32 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/4nY4 10:32 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.migrate_skbk2: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_skbk2 10:32 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.migrate_skbk2', metastore_id=None, name='migrate_skbk2', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 10:32 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_skbk2.ucx_tamab: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_skbk2/ucx_tamab 10:32 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_skbk2.ucx_tamab', metastore_id=None, name='ucx_tamab', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070311'}, row_filter=None, schema_name='migrate_skbk2', sql_path=None, storage_credential_name=None, storage_location='dbfs:/user/hive/warehouse/migrate_skbk2/ucx_tamab', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 10:33 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_skbk2.ucx_tlgyk: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_skbk2/ucx_tlgyk 10:33 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_skbk2.ucx_tlgyk', metastore_id=None, name='ucx_tlgyk', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070311'}, row_filter=None, schema_name='migrate_skbk2', sql_path=None, storage_credential_name=None, storage_location='dbfs:/tmp/ucx_test_D8sb', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 10:33 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_skbk2.ucx_tdsao: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_skbk2/ucx_tdsao 10:33 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_skbk2.ucx_tdsao', metastore_id=None, name='ucx_tdsao', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070311'}, row_filter=None, schema_name='migrate_skbk2', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/b/4nY4', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 10:34 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_skbk2.ucx_tah1k: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_skbk2/ucx_tah1k 10:34 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_skbk2.ucx_tah1k', metastore_id=None, name='ucx_tah1k', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070311'}, row_filter=None, schema_name='migrate_skbk2', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_skbk2.ucx_tamab', view_dependencies=None) 10:34 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_skbk2.ucx_tsicm: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_skbk2/ucx_tsicm 10:34 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_skbk2.ucx_tsicm', metastore_id=None, name='ucx_tsicm', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070311'}, row_filter=None, schema_name='migrate_skbk2', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_skbk2.ucx_tah1k', view_dependencies=None) 10:35 DEBUG [databricks.labs.ucx.mixins.fixtures] added catalog fixture: CatalogInfo(browse_only=False, catalog_type=, comment='', connection_name=None, created_at=1720002928886, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_ciklu', isolation_mode=, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_ciklu', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1720002928886, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d') 10:35 INFO [databricks.labs.ucx.mixins.fixtures] Schema ucx_ciklu.migrate_skbk2: https://DATABRICKS_HOST/explore/data/ucx_ciklu/migrate_skbk2 10:35 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_ciklu', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_ciklu.migrate_skbk2', metastore_id=None, name='migrate_skbk2', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 10:36 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_sysov: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_sysov 10:36 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_sysov', metastore_id=None, name='ucx_sysov', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) [gw6] linux -- Python 3.10.14 /home/runner/work/ucx/ucx/.venv/bin/python 10:32 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/4nY4 10:32 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.migrate_skbk2: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_skbk2 10:32 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.migrate_skbk2', metastore_id=None, name='migrate_skbk2', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 10:32 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_skbk2.ucx_tamab: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_skbk2/ucx_tamab 10:32 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_skbk2.ucx_tamab', metastore_id=None, name='ucx_tamab', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070311'}, row_filter=None, schema_name='migrate_skbk2', sql_path=None, storage_credential_name=None, storage_location='dbfs:/user/hive/warehouse/migrate_skbk2/ucx_tamab', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 10:33 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_skbk2.ucx_tlgyk: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_skbk2/ucx_tlgyk 10:33 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_skbk2.ucx_tlgyk', metastore_id=None, name='ucx_tlgyk', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070311'}, row_filter=None, schema_name='migrate_skbk2', sql_path=None, storage_credential_name=None, storage_location='dbfs:/tmp/ucx_test_D8sb', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 10:33 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_skbk2.ucx_tdsao: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_skbk2/ucx_tdsao 10:33 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_skbk2.ucx_tdsao', metastore_id=None, name='ucx_tdsao', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070311'}, row_filter=None, schema_name='migrate_skbk2', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/b/4nY4', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 10:34 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_skbk2.ucx_tah1k: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_skbk2/ucx_tah1k 10:34 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_skbk2.ucx_tah1k', metastore_id=None, name='ucx_tah1k', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070311'}, row_filter=None, schema_name='migrate_skbk2', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_skbk2.ucx_tamab', view_dependencies=None) 10:34 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_skbk2.ucx_tsicm: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_skbk2/ucx_tsicm 10:34 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_skbk2.ucx_tsicm', metastore_id=None, name='ucx_tsicm', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070311'}, row_filter=None, schema_name='migrate_skbk2', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_skbk2.ucx_tah1k', view_dependencies=None) 10:35 DEBUG [databricks.labs.ucx.mixins.fixtures] added catalog fixture: CatalogInfo(browse_only=False, catalog_type=, comment='', connection_name=None, created_at=1720002928886, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_ciklu', isolation_mode=, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_ciklu', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1720002928886, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d') 10:35 INFO [databricks.labs.ucx.mixins.fixtures] Schema ucx_ciklu.migrate_skbk2: https://DATABRICKS_HOST/explore/data/ucx_ciklu/migrate_skbk2 10:35 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_ciklu', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_ciklu.migrate_skbk2', metastore_id=None, name='migrate_skbk2', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 10:36 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_sysov: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_sysov 10:36 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_sysov', metastore_id=None, name='ucx_sysov', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 10:38 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.e8YY/config.yml) doesn't exist. 10:38 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration 10:38 INFO [databricks.labs.ucx.installer.hms_lineage] HMS Lineage feature creates one system table named system.hms_to_uc_migration.table_access and helps in your migration process from HMS to UC by allowing you to programmatically query HMS lineage data. 10:38 INFO [databricks.labs.ucx.install] Fetching installations... 10:38 WARNING [databricks.labs.ucx.install] Existing installation at /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.e8YY is corrupted. Skipping... 10:38 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy. 10:38 DEBUG [tests.integration.conftest] Waiting for clusters to start... 10:38 DEBUG [tests.integration.conftest] Waiting for clusters to start... 10:38 INFO [databricks.labs.ucx.install] Installing UCX v0.27.2+7020240703103844 10:38 INFO [databricks.labs.ucx.install] Creating ucx schemas... 10:42 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-tables-ctas 10:42 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=failing 10:42 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups 10:42 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=scan-tables-in-mounts-experimental 10:42 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental 10:42 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-hiveserde-tables-in-place-experimental 10:42 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment 10:42 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions 10:42 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups 10:42 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=experimental-workflow-linter 10:42 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-experimental 10:42 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-data-reconciliation 10:42 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables 10:42 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.e8YY/README for the next steps. 10:42 DEBUG [databricks.labs.ucx.installer.workflows] starting migrate-tables job: https://DATABRICKS_HOST#job/788254193723173 10:32 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/4nY4 10:32 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.migrate_skbk2: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_skbk2 10:32 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.migrate_skbk2', metastore_id=None, name='migrate_skbk2', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 10:32 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_skbk2.ucx_tamab: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_skbk2/ucx_tamab 10:32 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_skbk2.ucx_tamab', metastore_id=None, name='ucx_tamab', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070311'}, row_filter=None, schema_name='migrate_skbk2', sql_path=None, storage_credential_name=None, storage_location='dbfs:/user/hive/warehouse/migrate_skbk2/ucx_tamab', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 10:33 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_skbk2.ucx_tlgyk: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_skbk2/ucx_tlgyk 10:33 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_skbk2.ucx_tlgyk', metastore_id=None, name='ucx_tlgyk', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070311'}, row_filter=None, schema_name='migrate_skbk2', sql_path=None, storage_credential_name=None, storage_location='dbfs:/tmp/ucx_test_D8sb', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 10:33 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_skbk2.ucx_tdsao: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_skbk2/ucx_tdsao 10:33 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_skbk2.ucx_tdsao', metastore_id=None, name='ucx_tdsao', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070311'}, row_filter=None, schema_name='migrate_skbk2', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/b/4nY4', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 10:34 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_skbk2.ucx_tah1k: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_skbk2/ucx_tah1k 10:34 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_skbk2.ucx_tah1k', metastore_id=None, name='ucx_tah1k', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070311'}, row_filter=None, schema_name='migrate_skbk2', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_skbk2.ucx_tamab', view_dependencies=None) 10:34 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_skbk2.ucx_tsicm: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_skbk2/ucx_tsicm 10:34 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_skbk2.ucx_tsicm', metastore_id=None, name='ucx_tsicm', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070311'}, row_filter=None, schema_name='migrate_skbk2', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_skbk2.ucx_tah1k', view_dependencies=None) 10:35 DEBUG [databricks.labs.ucx.mixins.fixtures] added catalog fixture: CatalogInfo(browse_only=False, catalog_type=, comment='', connection_name=None, created_at=1720002928886, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_ciklu', isolation_mode=, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_ciklu', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1720002928886, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d') 10:35 INFO [databricks.labs.ucx.mixins.fixtures] Schema ucx_ciklu.migrate_skbk2: https://DATABRICKS_HOST/explore/data/ucx_ciklu/migrate_skbk2 10:35 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_ciklu', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_ciklu.migrate_skbk2', metastore_id=None, name='migrate_skbk2', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 10:36 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_sysov: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_sysov 10:36 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_sysov', metastore_id=None, name='ucx_sysov', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 10:38 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.e8YY/config.yml) doesn't exist. 10:38 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration 10:38 INFO [databricks.labs.ucx.installer.hms_lineage] HMS Lineage feature creates one system table named system.hms_to_uc_migration.table_access and helps in your migration process from HMS to UC by allowing you to programmatically query HMS lineage data. 10:38 INFO [databricks.labs.ucx.install] Fetching installations... 10:38 WARNING [databricks.labs.ucx.install] Existing installation at /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.e8YY is corrupted. Skipping... 10:38 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy. 10:38 DEBUG [tests.integration.conftest] Waiting for clusters to start... 10:38 DEBUG [tests.integration.conftest] Waiting for clusters to start... 10:38 INFO [databricks.labs.ucx.install] Installing UCX v0.27.2+7020240703103844 10:38 INFO [databricks.labs.ucx.install] Creating ucx schemas... 10:42 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-tables-ctas 10:42 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=failing 10:42 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups 10:42 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=scan-tables-in-mounts-experimental 10:42 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental 10:42 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-hiveserde-tables-in-place-experimental 10:42 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment 10:42 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions 10:42 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups 10:42 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=experimental-workflow-linter 10:42 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-experimental 10:42 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-data-reconciliation 10:42 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables 10:42 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.e8YY/README for the next steps. 10:42 DEBUG [databricks.labs.ucx.installer.workflows] starting migrate-tables job: https://DATABRICKS_HOST#job/788254193723173 10:47 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 make_storage_dir fixtures 10:47 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 make_dbfs_data_copy fixtures 10:47 DEBUG [databricks.labs.ucx.mixins.fixtures] removing make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/4nY4 10:47 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 cluster fixtures 10:47 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 catalog fixtures 10:47 DEBUG [databricks.labs.ucx.mixins.fixtures] removing catalog fixture: CatalogInfo(browse_only=False, catalog_type=, comment='', connection_name=None, created_at=1720002928886, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_ciklu', isolation_mode=, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_ciklu', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1720002928886, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d') 10:47 INFO [databricks.labs.ucx.install] Deleting UCX v0.27.2+7020240703103844 from https://DATABRICKS_HOST 10:47 INFO [databricks.labs.ucx.install] Deleting inventory database ucx_sysov 10:47 INFO [databricks.labs.ucx.install] Deleting jobs 10:47 INFO [databricks.labs.ucx.install] Deleting migrate-external-tables-ctas job_id=234098824797199. 10:47 INFO [databricks.labs.ucx.install] Deleting failing job_id=521544874493980. 10:47 INFO [databricks.labs.ucx.install] Deleting migrate-groups job_id=1108993676188030. 10:47 INFO [databricks.labs.ucx.install] Deleting scan-tables-in-mounts-experimental job_id=116633063990732. 10:47 INFO [databricks.labs.ucx.install] Deleting migrate-tables-in-mounts-experimental job_id=575919738001630. 10:47 INFO [databricks.labs.ucx.install] Deleting migrate-external-hiveserde-tables-in-place-experimental job_id=882776377884359. 10:47 INFO [databricks.labs.ucx.install] Deleting assessment job_id=875782110244714. 10:47 INFO [databricks.labs.ucx.install] Deleting validate-groups-permissions job_id=613506859980095. 10:47 INFO [databricks.labs.ucx.install] Deleting remove-workspace-local-backup-groups job_id=1054089206941091. 10:47 INFO [databricks.labs.ucx.install] Deleting experimental-workflow-linter job_id=727046154921114. 10:47 INFO [databricks.labs.ucx.install] Deleting migrate-groups-experimental job_id=567394419673251. 10:47 INFO [databricks.labs.ucx.install] Deleting migrate-data-reconciliation job_id=1073201196173110. 10:47 INFO [databricks.labs.ucx.install] Deleting migrate-tables job_id=788254193723173. 10:47 INFO [databricks.labs.ucx.install] Deleting cluster policy 10:47 INFO [databricks.labs.ucx.install] Deleting secret scope 10:47 INFO [databricks.labs.ucx.install] UnInstalling UCX complete 10:47 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 workspace user fixtures 10:47 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 account group fixtures 10:47 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 workspace group fixtures 10:47 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 table fixtures 10:47 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 5 table fixtures 10:47 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_skbk2.ucx_tamab', metastore_id=None, name='ucx_tamab', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070311'}, row_filter=None, schema_name='migrate_skbk2', sql_path=None, storage_credential_name=None, storage_location='dbfs:/user/hive/warehouse/migrate_skbk2/ucx_tamab', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 10:48 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_skbk2.ucx_tlgyk', metastore_id=None, name='ucx_tlgyk', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070311'}, row_filter=None, schema_name='migrate_skbk2', sql_path=None, storage_credential_name=None, storage_location='dbfs:/tmp/ucx_test_D8sb', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 10:48 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_skbk2.ucx_tdsao', metastore_id=None, name='ucx_tdsao', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070311'}, row_filter=None, schema_name='migrate_skbk2', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/b/4nY4', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 10:49 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_skbk2.ucx_tah1k', metastore_id=None, name='ucx_tah1k', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070311'}, row_filter=None, schema_name='migrate_skbk2', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_skbk2.ucx_tamab', view_dependencies=None) 10:49 DEBUG [databricks.labs.ucx.mixins.fixtures] ignoring error while table TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_skbk2.ucx_tah1k', metastore_id=None, name='ucx_tah1k', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070311'}, row_filter=None, schema_name='migrate_skbk2', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_skbk2.ucx_tamab', view_dependencies=None) teardown: [WRONG_COMMAND_FOR_OBJECT_TYPE] The operation DROP TABLE requires a EXTERNAL or MANAGED. But hive_metastore.migrate_skbk2.ucx_tah1k is a VIEW. Use DROP VIEW instead. SQLSTATE: 42809 10:49 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_skbk2.ucx_tsicm', metastore_id=None, name='ucx_tsicm', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070311'}, row_filter=None, schema_name='migrate_skbk2', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_skbk2.ucx_tah1k', view_dependencies=None) 10:50 DEBUG [databricks.labs.ucx.mixins.fixtures] ignoring error while table TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_skbk2.ucx_tsicm', metastore_id=None, name='ucx_tsicm', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070311'}, row_filter=None, schema_name='migrate_skbk2', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_skbk2.ucx_tah1k', view_dependencies=None) teardown: [WRONG_COMMAND_FOR_OBJECT_TYPE] The operation DROP TABLE requires a EXTERNAL or MANAGED. But hive_metastore.migrate_skbk2.ucx_tsicm is a VIEW. Use DROP VIEW instead. SQLSTATE: 42809 10:50 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 3 schema fixtures 10:50 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.migrate_skbk2', metastore_id=None, name='migrate_skbk2', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 10:50 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_ciklu', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_ciklu.migrate_skbk2', metastore_id=None, name='migrate_skbk2', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 10:51 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_sysov', metastore_id=None, name='ucx_sysov', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) [gw6] linux -- Python 3.10.14 /home/runner/work/ucx/ucx/.venv/bin/python ```

Running from acceptance #4253