search

databrickslabs / ucx

Your best companion for upgrading to Unity Catalog. UCX will guide you, the Databricks customer, through the process of upgrading your account, groups, workspaces, jobs etc. to Unity Catalog.
Other
196 stars 70 forks source link

manually whitelist core.dbclient and core.logging_utils (from DB SAT) #1971

Closed ericvergnaud closed 2 weeks ago

ericvergnaud commented 2 weeks ago

Changes

manually whitelist core.dbclient and core.logging_utils (from DB SAT)

Linked issues

Progresses #1901

Functionality

None

Tests

nfx commented 2 weeks ago

DB SAT is a prototype, we don't expect it to be used by customers

github-actions[bot] commented 2 weeks ago

❌ 131/140 passed, 9 failed, 22 skipped, 5h5m13s total

❌ test_table_migration_job_refreshes_migration_status[hiveserde-migrate-external-tables-ctas]: TimeoutError: Timed out after 0:05:00 (8m49.294s) ``` TimeoutError: Timed out after 0:05:00 17:52 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/Imt9 17:52 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.hiveserde_in_place_iyov6: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_iyov6 17:52 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.hiveserde_in_place_iyov6', metastore_id=None, name='hiveserde_in_place_iyov6', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 17:52 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_storage_dir fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_iyov6 17:52 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_iyov6.parquet_serde_iyov6: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_iyov6/parquet_serde_iyov6 17:52 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_iyov6.parquet_serde_iyov6', metastore_id=None, name='parquet_serde_iyov6', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='hiveserde_in_place_iyov6', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_iyov6/parquet_serde_iyov6', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 17:52 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_iyov6.orc_serde_iyov6: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_iyov6/orc_serde_iyov6 17:52 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_iyov6.orc_serde_iyov6', metastore_id=None, name='orc_serde_iyov6', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='hiveserde_in_place_iyov6', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_iyov6/orc_serde_iyov6', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 17:52 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_iyov6.avro_serde_iyov6: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_iyov6/avro_serde_iyov6 17:52 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_iyov6.avro_serde_iyov6', metastore_id=None, name='avro_serde_iyov6', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='hiveserde_in_place_iyov6', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_iyov6/avro_serde_iyov6', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 17:52 DEBUG [databricks.labs.ucx.mixins.fixtures] added catalog fixture: CatalogInfo(browse_only=False, catalog_type=, comment='', connection_name=None, created_at=1719942771405, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cacnr', isolation_mode=, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_cacnr', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1719942771405, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d') 17:52 INFO [databricks.labs.ucx.mixins.fixtures] Schema ucx_cacnr.hiveserde_in_place_iyov6: https://DATABRICKS_HOST/explore/data/ucx_cacnr/hiveserde_in_place_iyov6 17:52 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_cacnr', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cacnr.hiveserde_in_place_iyov6', metastore_id=None, name='hiveserde_in_place_iyov6', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 17:54 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_sqlsg: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_sqlsg 17:54 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_sqlsg', metastore_id=None, name='ucx_sqlsg', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) [gw2] linux -- Python 3.10.14 /home/runner/work/ucx/ucx/.venv/bin/python 17:52 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/Imt9 17:52 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.hiveserde_in_place_iyov6: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_iyov6 17:52 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.hiveserde_in_place_iyov6', metastore_id=None, name='hiveserde_in_place_iyov6', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 17:52 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_storage_dir fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_iyov6 17:52 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_iyov6.parquet_serde_iyov6: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_iyov6/parquet_serde_iyov6 17:52 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_iyov6.parquet_serde_iyov6', metastore_id=None, name='parquet_serde_iyov6', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='hiveserde_in_place_iyov6', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_iyov6/parquet_serde_iyov6', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 17:52 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_iyov6.orc_serde_iyov6: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_iyov6/orc_serde_iyov6 17:52 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_iyov6.orc_serde_iyov6', metastore_id=None, name='orc_serde_iyov6', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='hiveserde_in_place_iyov6', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_iyov6/orc_serde_iyov6', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 17:52 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_iyov6.avro_serde_iyov6: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_iyov6/avro_serde_iyov6 17:52 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_iyov6.avro_serde_iyov6', metastore_id=None, name='avro_serde_iyov6', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='hiveserde_in_place_iyov6', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_iyov6/avro_serde_iyov6', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 17:52 DEBUG [databricks.labs.ucx.mixins.fixtures] added catalog fixture: CatalogInfo(browse_only=False, catalog_type=, comment='', connection_name=None, created_at=1719942771405, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cacnr', isolation_mode=, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_cacnr', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1719942771405, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d') 17:52 INFO [databricks.labs.ucx.mixins.fixtures] Schema ucx_cacnr.hiveserde_in_place_iyov6: https://DATABRICKS_HOST/explore/data/ucx_cacnr/hiveserde_in_place_iyov6 17:52 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_cacnr', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cacnr.hiveserde_in_place_iyov6', metastore_id=None, name='hiveserde_in_place_iyov6', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 17:54 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_sqlsg: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_sqlsg 17:54 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_sqlsg', metastore_id=None, name='ucx_sqlsg', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 17:54 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.5qn6/config.yml) doesn't exist. 17:54 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration 17:55 INFO [databricks.labs.ucx.installer.hms_lineage] HMS Lineage feature creates one system table named system.hms_to_uc_migration.table_access and helps in your migration process from HMS to UC by allowing you to programmatically query HMS lineage data. 17:55 INFO [databricks.labs.ucx.install] Fetching installations... 17:55 WARNING [databricks.labs.ucx.install] Existing installation at /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.5qn6 is corrupted. Skipping... 17:55 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy. 17:55 DEBUG [tests.integration.conftest] Waiting for clusters to start... 17:55 DEBUG [tests.integration.conftest] Waiting for clusters to start... 17:55 INFO [databricks.labs.ucx.install] Installing UCX v0.27.2+6020240702175516 17:55 INFO [databricks.labs.ucx.install] Creating ucx schemas... 17:55 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=experimental-workflow-linter 17:55 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=failing 17:55 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-data-reconciliation 17:55 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-experimental 17:55 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions 17:55 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-tables-ctas 17:55 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups 17:55 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables 17:56 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-hiveserde-tables-in-place-experimental 17:56 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental 17:56 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment 17:56 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=scan-tables-in-mounts-experimental 17:56 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups 17:56 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.5qn6/README for the next steps. 17:56 DEBUG [databricks.labs.ucx.installer.workflows] starting migrate-external-tables-ctas job: https://DATABRICKS_HOST#job/616238125397648 17:52 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/Imt9 17:52 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.hiveserde_in_place_iyov6: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_iyov6 17:52 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.hiveserde_in_place_iyov6', metastore_id=None, name='hiveserde_in_place_iyov6', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 17:52 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_storage_dir fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_iyov6 17:52 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_iyov6.parquet_serde_iyov6: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_iyov6/parquet_serde_iyov6 17:52 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_iyov6.parquet_serde_iyov6', metastore_id=None, name='parquet_serde_iyov6', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='hiveserde_in_place_iyov6', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_iyov6/parquet_serde_iyov6', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 17:52 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_iyov6.orc_serde_iyov6: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_iyov6/orc_serde_iyov6 17:52 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_iyov6.orc_serde_iyov6', metastore_id=None, name='orc_serde_iyov6', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='hiveserde_in_place_iyov6', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_iyov6/orc_serde_iyov6', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 17:52 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_iyov6.avro_serde_iyov6: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_iyov6/avro_serde_iyov6 17:52 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_iyov6.avro_serde_iyov6', metastore_id=None, name='avro_serde_iyov6', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='hiveserde_in_place_iyov6', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_iyov6/avro_serde_iyov6', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 17:52 DEBUG [databricks.labs.ucx.mixins.fixtures] added catalog fixture: CatalogInfo(browse_only=False, catalog_type=, comment='', connection_name=None, created_at=1719942771405, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cacnr', isolation_mode=, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_cacnr', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1719942771405, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d') 17:52 INFO [databricks.labs.ucx.mixins.fixtures] Schema ucx_cacnr.hiveserde_in_place_iyov6: https://DATABRICKS_HOST/explore/data/ucx_cacnr/hiveserde_in_place_iyov6 17:52 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_cacnr', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cacnr.hiveserde_in_place_iyov6', metastore_id=None, name='hiveserde_in_place_iyov6', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 17:54 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_sqlsg: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_sqlsg 17:54 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_sqlsg', metastore_id=None, name='ucx_sqlsg', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 17:54 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.5qn6/config.yml) doesn't exist. 17:54 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration 17:55 INFO [databricks.labs.ucx.installer.hms_lineage] HMS Lineage feature creates one system table named system.hms_to_uc_migration.table_access and helps in your migration process from HMS to UC by allowing you to programmatically query HMS lineage data. 17:55 INFO [databricks.labs.ucx.install] Fetching installations... 17:55 WARNING [databricks.labs.ucx.install] Existing installation at /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.5qn6 is corrupted. Skipping... 17:55 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy. 17:55 DEBUG [tests.integration.conftest] Waiting for clusters to start... 17:55 DEBUG [tests.integration.conftest] Waiting for clusters to start... 17:55 INFO [databricks.labs.ucx.install] Installing UCX v0.27.2+6020240702175516 17:55 INFO [databricks.labs.ucx.install] Creating ucx schemas... 17:55 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=experimental-workflow-linter 17:55 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=failing 17:55 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-data-reconciliation 17:55 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-experimental 17:55 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions 17:55 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-tables-ctas 17:55 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups 17:55 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables 17:56 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-hiveserde-tables-in-place-experimental 17:56 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental 17:56 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment 17:56 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=scan-tables-in-mounts-experimental 17:56 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups 17:56 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.5qn6/README for the next steps. 17:56 DEBUG [databricks.labs.ucx.installer.workflows] starting migrate-external-tables-ctas job: https://DATABRICKS_HOST#job/616238125397648 18:00 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 make_storage_dir fixtures 18:00 DEBUG [databricks.labs.ucx.mixins.fixtures] removing make_storage_dir fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_iyov6 18:00 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 make_dbfs_data_copy fixtures 18:00 DEBUG [databricks.labs.ucx.mixins.fixtures] removing make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/Imt9 18:00 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 cluster fixtures 18:00 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 catalog fixtures 18:00 DEBUG [databricks.labs.ucx.mixins.fixtures] removing catalog fixture: CatalogInfo(browse_only=False, catalog_type=, comment='', connection_name=None, created_at=1719942771405, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cacnr', isolation_mode=, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_cacnr', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1719942771405, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d') 18:00 INFO [databricks.labs.ucx.install] Deleting UCX v0.27.2+6020240702175516 from https://DATABRICKS_HOST 18:00 INFO [databricks.labs.ucx.install] Deleting inventory database ucx_sqlsg 18:00 INFO [databricks.labs.ucx.install] Deleting jobs 18:00 INFO [databricks.labs.ucx.install] Deleting experimental-workflow-linter job_id=978059117695056. 18:00 INFO [databricks.labs.ucx.install] Deleting failing job_id=251185053827991. 18:00 INFO [databricks.labs.ucx.install] Deleting migrate-data-reconciliation job_id=454300187120166. 18:00 INFO [databricks.labs.ucx.install] Deleting migrate-groups-experimental job_id=669557071453900. 18:00 INFO [databricks.labs.ucx.install] Deleting validate-groups-permissions job_id=617415383481472. 18:00 INFO [databricks.labs.ucx.install] Deleting migrate-external-tables-ctas job_id=616238125397648. 18:00 INFO [databricks.labs.ucx.install] Deleting remove-workspace-local-backup-groups job_id=856878714220225. 18:00 INFO [databricks.labs.ucx.install] Deleting migrate-tables job_id=813608820648682. 18:00 INFO [databricks.labs.ucx.install] Deleting migrate-external-hiveserde-tables-in-place-experimental job_id=611997777779568. 18:00 INFO [databricks.labs.ucx.install] Deleting migrate-tables-in-mounts-experimental job_id=358158883252541. 18:00 INFO [databricks.labs.ucx.install] Deleting assessment job_id=874435102870140. 18:00 INFO [databricks.labs.ucx.install] Deleting scan-tables-in-mounts-experimental job_id=383679415479974. 18:00 INFO [databricks.labs.ucx.install] Deleting migrate-groups job_id=755999498862468. 18:00 INFO [databricks.labs.ucx.install] Deleting cluster policy 18:00 INFO [databricks.labs.ucx.install] Deleting secret scope 18:00 INFO [databricks.labs.ucx.install] UnInstalling UCX complete 18:00 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 workspace user fixtures 18:00 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 account group fixtures 18:00 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 workspace group fixtures 18:00 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 table fixtures 18:00 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 3 table fixtures 18:00 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_iyov6.parquet_serde_iyov6', metastore_id=None, name='parquet_serde_iyov6', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='hiveserde_in_place_iyov6', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_iyov6/parquet_serde_iyov6', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 18:00 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_iyov6.orc_serde_iyov6', metastore_id=None, name='orc_serde_iyov6', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='hiveserde_in_place_iyov6', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_iyov6/orc_serde_iyov6', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 18:00 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_iyov6.avro_serde_iyov6', metastore_id=None, name='avro_serde_iyov6', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='hiveserde_in_place_iyov6', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_iyov6/avro_serde_iyov6', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 18:00 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 3 schema fixtures 18:00 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.hiveserde_in_place_iyov6', metastore_id=None, name='hiveserde_in_place_iyov6', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 18:01 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_cacnr', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cacnr.hiveserde_in_place_iyov6', metastore_id=None, name='hiveserde_in_place_iyov6', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 18:01 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_sqlsg', metastore_id=None, name='ucx_sqlsg', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) [gw2] linux -- Python 3.10.14 /home/runner/work/ucx/ucx/.venv/bin/python ```
❌ test_table_migration_job_refreshes_migration_status[regular-migrate-tables]: TimeoutError: Timed out after 0:05:00 (11m22.42s) ``` TimeoutError: Timed out after 0:05:00 17:55 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/kPul 17:55 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.migrate_1v4sp: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_1v4sp 17:55 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.migrate_1v4sp', metastore_id=None, name='migrate_1v4sp', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 17:55 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_1v4sp.ucx_trmsy: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_1v4sp/ucx_trmsy 17:55 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_1v4sp.ucx_trmsy', metastore_id=None, name='ucx_trmsy', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='migrate_1v4sp', sql_path=None, storage_credential_name=None, storage_location='dbfs:/user/hive/warehouse/migrate_1v4sp/ucx_trmsy', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 17:55 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_1v4sp.ucx_tyaxy: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_1v4sp/ucx_tyaxy 17:55 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_1v4sp.ucx_tyaxy', metastore_id=None, name='ucx_tyaxy', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='migrate_1v4sp', sql_path=None, storage_credential_name=None, storage_location='dbfs:/tmp/ucx_test_X0H7', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 17:55 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_1v4sp.ucx_ta9o7: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_1v4sp/ucx_ta9o7 17:55 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_1v4sp.ucx_ta9o7', metastore_id=None, name='ucx_ta9o7', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='migrate_1v4sp', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/b/kPul', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 17:55 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_1v4sp.ucx_tagef: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_1v4sp/ucx_tagef 17:55 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_1v4sp.ucx_tagef', metastore_id=None, name='ucx_tagef', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='migrate_1v4sp', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_1v4sp.ucx_trmsy', view_dependencies=None) 17:55 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_1v4sp.ucx_t3o6b: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_1v4sp/ucx_t3o6b 17:55 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_1v4sp.ucx_t3o6b', metastore_id=None, name='ucx_t3o6b', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='migrate_1v4sp', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_1v4sp.ucx_tagef', view_dependencies=None) 17:55 DEBUG [databricks.labs.ucx.mixins.fixtures] added catalog fixture: CatalogInfo(browse_only=False, catalog_type=, comment='', connection_name=None, created_at=1719942947253, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cn4y8', isolation_mode=, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_cn4y8', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1719942947253, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d') 17:55 INFO [databricks.labs.ucx.mixins.fixtures] Schema ucx_cn4y8.migrate_1v4sp: https://DATABRICKS_HOST/explore/data/ucx_cn4y8/migrate_1v4sp 17:55 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_cn4y8', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cn4y8.migrate_1v4sp', metastore_id=None, name='migrate_1v4sp', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 17:56 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_sqcix: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_sqcix 17:56 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_sqcix', metastore_id=None, name='ucx_sqcix', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) [gw0] linux -- Python 3.10.14 /home/runner/work/ucx/ucx/.venv/bin/python 17:55 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/kPul 17:55 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.migrate_1v4sp: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_1v4sp 17:55 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.migrate_1v4sp', metastore_id=None, name='migrate_1v4sp', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 17:55 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_1v4sp.ucx_trmsy: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_1v4sp/ucx_trmsy 17:55 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_1v4sp.ucx_trmsy', metastore_id=None, name='ucx_trmsy', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='migrate_1v4sp', sql_path=None, storage_credential_name=None, storage_location='dbfs:/user/hive/warehouse/migrate_1v4sp/ucx_trmsy', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 17:55 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_1v4sp.ucx_tyaxy: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_1v4sp/ucx_tyaxy 17:55 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_1v4sp.ucx_tyaxy', metastore_id=None, name='ucx_tyaxy', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='migrate_1v4sp', sql_path=None, storage_credential_name=None, storage_location='dbfs:/tmp/ucx_test_X0H7', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 17:55 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_1v4sp.ucx_ta9o7: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_1v4sp/ucx_ta9o7 17:55 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_1v4sp.ucx_ta9o7', metastore_id=None, name='ucx_ta9o7', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='migrate_1v4sp', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/b/kPul', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 17:55 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_1v4sp.ucx_tagef: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_1v4sp/ucx_tagef 17:55 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_1v4sp.ucx_tagef', metastore_id=None, name='ucx_tagef', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='migrate_1v4sp', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_1v4sp.ucx_trmsy', view_dependencies=None) 17:55 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_1v4sp.ucx_t3o6b: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_1v4sp/ucx_t3o6b 17:55 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_1v4sp.ucx_t3o6b', metastore_id=None, name='ucx_t3o6b', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='migrate_1v4sp', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_1v4sp.ucx_tagef', view_dependencies=None) 17:55 DEBUG [databricks.labs.ucx.mixins.fixtures] added catalog fixture: CatalogInfo(browse_only=False, catalog_type=, comment='', connection_name=None, created_at=1719942947253, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cn4y8', isolation_mode=, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_cn4y8', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1719942947253, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d') 17:55 INFO [databricks.labs.ucx.mixins.fixtures] Schema ucx_cn4y8.migrate_1v4sp: https://DATABRICKS_HOST/explore/data/ucx_cn4y8/migrate_1v4sp 17:55 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_cn4y8', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cn4y8.migrate_1v4sp', metastore_id=None, name='migrate_1v4sp', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 17:56 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_sqcix: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_sqcix 17:56 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_sqcix', metastore_id=None, name='ucx_sqcix', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 17:57 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.Dwmo/config.yml) doesn't exist. 17:57 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration 17:57 INFO [databricks.labs.ucx.installer.hms_lineage] HMS Lineage feature creates one system table named system.hms_to_uc_migration.table_access and helps in your migration process from HMS to UC by allowing you to programmatically query HMS lineage data. 17:57 INFO [databricks.labs.ucx.install] Fetching installations... 17:58 WARNING [databricks.labs.ucx.install] Existing installation at /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.Dwmo is corrupted. Skipping... 17:58 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy. 17:58 DEBUG [tests.integration.conftest] Waiting for clusters to start... 17:58 DEBUG [tests.integration.conftest] Waiting for clusters to start... 17:58 INFO [databricks.labs.ucx.install] Installing UCX v0.27.2+6020240702175810 17:58 INFO [databricks.labs.ucx.install] Creating ucx schemas... 17:59 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions 17:59 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-experimental 17:59 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups 17:59 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental 17:59 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-data-reconciliation 17:59 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables 17:59 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups 17:59 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-hiveserde-tables-in-place-experimental 17:59 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=scan-tables-in-mounts-experimental 17:59 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=experimental-workflow-linter 17:59 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment 17:59 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-tables-ctas 17:59 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=failing 17:59 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.Dwmo/README for the next steps. 17:59 DEBUG [databricks.labs.ucx.installer.workflows] starting migrate-tables job: https://DATABRICKS_HOST#job/33094689566409 17:55 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/kPul 17:55 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.migrate_1v4sp: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_1v4sp 17:55 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.migrate_1v4sp', metastore_id=None, name='migrate_1v4sp', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 17:55 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_1v4sp.ucx_trmsy: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_1v4sp/ucx_trmsy 17:55 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_1v4sp.ucx_trmsy', metastore_id=None, name='ucx_trmsy', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='migrate_1v4sp', sql_path=None, storage_credential_name=None, storage_location='dbfs:/user/hive/warehouse/migrate_1v4sp/ucx_trmsy', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 17:55 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_1v4sp.ucx_tyaxy: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_1v4sp/ucx_tyaxy 17:55 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_1v4sp.ucx_tyaxy', metastore_id=None, name='ucx_tyaxy', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='migrate_1v4sp', sql_path=None, storage_credential_name=None, storage_location='dbfs:/tmp/ucx_test_X0H7', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 17:55 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_1v4sp.ucx_ta9o7: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_1v4sp/ucx_ta9o7 17:55 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_1v4sp.ucx_ta9o7', metastore_id=None, name='ucx_ta9o7', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='migrate_1v4sp', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/b/kPul', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 17:55 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_1v4sp.ucx_tagef: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_1v4sp/ucx_tagef 17:55 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_1v4sp.ucx_tagef', metastore_id=None, name='ucx_tagef', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='migrate_1v4sp', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_1v4sp.ucx_trmsy', view_dependencies=None) 17:55 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_1v4sp.ucx_t3o6b: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_1v4sp/ucx_t3o6b 17:55 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_1v4sp.ucx_t3o6b', metastore_id=None, name='ucx_t3o6b', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='migrate_1v4sp', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_1v4sp.ucx_tagef', view_dependencies=None) 17:55 DEBUG [databricks.labs.ucx.mixins.fixtures] added catalog fixture: CatalogInfo(browse_only=False, catalog_type=, comment='', connection_name=None, created_at=1719942947253, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cn4y8', isolation_mode=, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_cn4y8', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1719942947253, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d') 17:55 INFO [databricks.labs.ucx.mixins.fixtures] Schema ucx_cn4y8.migrate_1v4sp: https://DATABRICKS_HOST/explore/data/ucx_cn4y8/migrate_1v4sp 17:55 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_cn4y8', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cn4y8.migrate_1v4sp', metastore_id=None, name='migrate_1v4sp', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 17:56 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_sqcix: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_sqcix 17:56 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_sqcix', metastore_id=None, name='ucx_sqcix', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 17:57 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.Dwmo/config.yml) doesn't exist. 17:57 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration 17:57 INFO [databricks.labs.ucx.installer.hms_lineage] HMS Lineage feature creates one system table named system.hms_to_uc_migration.table_access and helps in your migration process from HMS to UC by allowing you to programmatically query HMS lineage data. 17:57 INFO [databricks.labs.ucx.install] Fetching installations... 17:58 WARNING [databricks.labs.ucx.install] Existing installation at /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.Dwmo is corrupted. Skipping... 17:58 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy. 17:58 DEBUG [tests.integration.conftest] Waiting for clusters to start... 17:58 DEBUG [tests.integration.conftest] Waiting for clusters to start... 17:58 INFO [databricks.labs.ucx.install] Installing UCX v0.27.2+6020240702175810 17:58 INFO [databricks.labs.ucx.install] Creating ucx schemas... 17:59 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions 17:59 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-experimental 17:59 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups 17:59 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental 17:59 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-data-reconciliation 17:59 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables 17:59 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups 17:59 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-hiveserde-tables-in-place-experimental 17:59 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=scan-tables-in-mounts-experimental 17:59 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=experimental-workflow-linter 17:59 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment 17:59 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-tables-ctas 17:59 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=failing 17:59 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.Dwmo/README for the next steps. 17:59 DEBUG [databricks.labs.ucx.installer.workflows] starting migrate-tables job: https://DATABRICKS_HOST#job/33094689566409 18:05 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 make_storage_dir fixtures 18:05 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 make_dbfs_data_copy fixtures 18:05 DEBUG [databricks.labs.ucx.mixins.fixtures] removing make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/kPul 18:05 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 cluster fixtures 18:05 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 catalog fixtures 18:05 DEBUG [databricks.labs.ucx.mixins.fixtures] removing catalog fixture: CatalogInfo(browse_only=False, catalog_type=, comment='', connection_name=None, created_at=1719942947253, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cn4y8', isolation_mode=, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_cn4y8', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1719942947253, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d') 18:05 INFO [databricks.labs.ucx.install] Deleting UCX v0.27.2+6020240702175810 from https://DATABRICKS_HOST 18:05 INFO [databricks.labs.ucx.install] Deleting inventory database ucx_sqcix 18:05 INFO [databricks.labs.ucx.install] Deleting jobs 18:05 INFO [databricks.labs.ucx.install] Deleting validate-groups-permissions job_id=1007232131410924. 18:05 INFO [databricks.labs.ucx.install] Deleting migrate-groups-experimental job_id=290673855040683. 18:05 INFO [databricks.labs.ucx.install] Deleting migrate-groups job_id=93108210342518. 18:05 INFO [databricks.labs.ucx.install] Deleting migrate-tables-in-mounts-experimental job_id=161118294449531. 18:05 INFO [databricks.labs.ucx.install] Deleting migrate-data-reconciliation job_id=1045854618320972. 18:05 INFO [databricks.labs.ucx.install] Deleting migrate-tables job_id=33094689566409. 18:05 INFO [databricks.labs.ucx.install] Deleting remove-workspace-local-backup-groups job_id=428950644092683. 18:05 INFO [databricks.labs.ucx.install] Deleting migrate-external-hiveserde-tables-in-place-experimental job_id=322351445197059. 18:05 INFO [databricks.labs.ucx.install] Deleting scan-tables-in-mounts-experimental job_id=691839228940581. 18:05 INFO [databricks.labs.ucx.install] Deleting experimental-workflow-linter job_id=1014101505710539. 18:05 INFO [databricks.labs.ucx.install] Deleting assessment job_id=40503676718701. 18:05 INFO [databricks.labs.ucx.install] Deleting migrate-external-tables-ctas job_id=220032307640192. 18:05 INFO [databricks.labs.ucx.install] Deleting failing job_id=744246748008382. 18:05 INFO [databricks.labs.ucx.install] Deleting cluster policy 18:05 INFO [databricks.labs.ucx.install] Deleting secret scope 18:05 INFO [databricks.labs.ucx.install] UnInstalling UCX complete 18:05 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 workspace user fixtures 18:05 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 account group fixtures 18:05 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 workspace group fixtures 18:05 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 table fixtures 18:05 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 5 table fixtures 18:05 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_1v4sp.ucx_trmsy', metastore_id=None, name='ucx_trmsy', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='migrate_1v4sp', sql_path=None, storage_credential_name=None, storage_location='dbfs:/user/hive/warehouse/migrate_1v4sp/ucx_trmsy', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 18:05 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_1v4sp.ucx_tyaxy', metastore_id=None, name='ucx_tyaxy', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='migrate_1v4sp', sql_path=None, storage_credential_name=None, storage_location='dbfs:/tmp/ucx_test_X0H7', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 18:05 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_1v4sp.ucx_ta9o7', metastore_id=None, name='ucx_ta9o7', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='migrate_1v4sp', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/b/kPul', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 18:05 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_1v4sp.ucx_tagef', metastore_id=None, name='ucx_tagef', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='migrate_1v4sp', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_1v4sp.ucx_trmsy', view_dependencies=None) 18:05 DEBUG [databricks.labs.ucx.mixins.fixtures] ignoring error while table TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_1v4sp.ucx_tagef', metastore_id=None, name='ucx_tagef', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='migrate_1v4sp', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_1v4sp.ucx_trmsy', view_dependencies=None) teardown: [WRONG_COMMAND_FOR_OBJECT_TYPE] The operation DROP TABLE requires a EXTERNAL or MANAGED. But hive_metastore.migrate_1v4sp.ucx_tagef is a VIEW. Use DROP VIEW instead. SQLSTATE: 42809 18:05 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_1v4sp.ucx_t3o6b', metastore_id=None, name='ucx_t3o6b', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='migrate_1v4sp', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_1v4sp.ucx_tagef', view_dependencies=None) 18:06 DEBUG [databricks.labs.ucx.mixins.fixtures] ignoring error while table TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_1v4sp.ucx_t3o6b', metastore_id=None, name='ucx_t3o6b', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='migrate_1v4sp', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_1v4sp.ucx_tagef', view_dependencies=None) teardown: [WRONG_COMMAND_FOR_OBJECT_TYPE] The operation DROP TABLE requires a EXTERNAL or MANAGED. But hive_metastore.migrate_1v4sp.ucx_t3o6b is a VIEW. Use DROP VIEW instead. SQLSTATE: 42809 18:06 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 3 schema fixtures 18:06 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.migrate_1v4sp', metastore_id=None, name='migrate_1v4sp', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 18:06 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_cn4y8', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cn4y8.migrate_1v4sp', metastore_id=None, name='migrate_1v4sp', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 18:06 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_sqcix', metastore_id=None, name='ucx_sqcix', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) [gw0] linux -- Python 3.10.14 /home/runner/work/ucx/ucx/.venv/bin/python ```
❌ test_table_migration_job_refreshes_migration_status[hiveserde-migrate-external-hiveserde-tables-in-place-experimental]: TimeoutError: Timed out after 0:05:00 (8m16.776s) ``` TimeoutError: Timed out after 0:05:00 18:06 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/hi0O 18:06 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.hiveserde_in_place_gueyb: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_gueyb 18:06 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.hiveserde_in_place_gueyb', metastore_id=None, name='hiveserde_in_place_gueyb', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 18:06 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_storage_dir fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_gueyb 18:06 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_gueyb.parquet_serde_gueyb: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_gueyb/parquet_serde_gueyb 18:06 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_gueyb.parquet_serde_gueyb', metastore_id=None, name='parquet_serde_gueyb', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='hiveserde_in_place_gueyb', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_gueyb/parquet_serde_gueyb', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 18:06 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_gueyb.orc_serde_gueyb: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_gueyb/orc_serde_gueyb 18:06 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_gueyb.orc_serde_gueyb', metastore_id=None, name='orc_serde_gueyb', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='hiveserde_in_place_gueyb', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_gueyb/orc_serde_gueyb', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 18:06 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_gueyb.avro_serde_gueyb: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_gueyb/avro_serde_gueyb 18:06 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_gueyb.avro_serde_gueyb', metastore_id=None, name='avro_serde_gueyb', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='hiveserde_in_place_gueyb', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_gueyb/avro_serde_gueyb', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 18:07 DEBUG [databricks.labs.ucx.mixins.fixtures] added catalog fixture: CatalogInfo(browse_only=False, catalog_type=, comment='', connection_name=None, created_at=1719943622378, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_ckyyw', isolation_mode=, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_ckyyw', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1719943622378, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d') 18:07 INFO [databricks.labs.ucx.mixins.fixtures] Schema ucx_ckyyw.hiveserde_in_place_gueyb: https://DATABRICKS_HOST/explore/data/ucx_ckyyw/hiveserde_in_place_gueyb 18:07 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_ckyyw', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_ckyyw.hiveserde_in_place_gueyb', metastore_id=None, name='hiveserde_in_place_gueyb', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 18:07 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_stica: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_stica 18:07 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_stica', metastore_id=None, name='ucx_stica', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) [gw0] linux -- Python 3.10.14 /home/runner/work/ucx/ucx/.venv/bin/python 18:06 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/hi0O 18:06 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.hiveserde_in_place_gueyb: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_gueyb 18:06 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.hiveserde_in_place_gueyb', metastore_id=None, name='hiveserde_in_place_gueyb', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 18:06 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_storage_dir fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_gueyb 18:06 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_gueyb.parquet_serde_gueyb: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_gueyb/parquet_serde_gueyb 18:06 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_gueyb.parquet_serde_gueyb', metastore_id=None, name='parquet_serde_gueyb', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='hiveserde_in_place_gueyb', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_gueyb/parquet_serde_gueyb', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 18:06 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_gueyb.orc_serde_gueyb: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_gueyb/orc_serde_gueyb 18:06 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_gueyb.orc_serde_gueyb', metastore_id=None, name='orc_serde_gueyb', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='hiveserde_in_place_gueyb', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_gueyb/orc_serde_gueyb', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 18:06 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_gueyb.avro_serde_gueyb: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_gueyb/avro_serde_gueyb 18:06 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_gueyb.avro_serde_gueyb', metastore_id=None, name='avro_serde_gueyb', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='hiveserde_in_place_gueyb', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_gueyb/avro_serde_gueyb', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 18:07 DEBUG [databricks.labs.ucx.mixins.fixtures] added catalog fixture: CatalogInfo(browse_only=False, catalog_type=, comment='', connection_name=None, created_at=1719943622378, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_ckyyw', isolation_mode=, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_ckyyw', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1719943622378, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d') 18:07 INFO [databricks.labs.ucx.mixins.fixtures] Schema ucx_ckyyw.hiveserde_in_place_gueyb: https://DATABRICKS_HOST/explore/data/ucx_ckyyw/hiveserde_in_place_gueyb 18:07 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_ckyyw', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_ckyyw.hiveserde_in_place_gueyb', metastore_id=None, name='hiveserde_in_place_gueyb', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 18:07 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_stica: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_stica 18:07 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_stica', metastore_id=None, name='ucx_stica', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 18:08 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.f1Qn/config.yml) doesn't exist. 18:08 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration 18:08 INFO [databricks.labs.ucx.installer.hms_lineage] HMS Lineage feature creates one system table named system.hms_to_uc_migration.table_access and helps in your migration process from HMS to UC by allowing you to programmatically query HMS lineage data. 18:08 INFO [databricks.labs.ucx.install] Fetching installations... 18:08 WARNING [databricks.labs.ucx.install] Existing installation at /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.f1Qn is corrupted. Skipping... 18:08 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy. 18:08 DEBUG [tests.integration.conftest] Waiting for clusters to start... 18:08 DEBUG [tests.integration.conftest] Waiting for clusters to start... 18:08 INFO [databricks.labs.ucx.install] Installing UCX v0.27.2+6020240702180829 18:08 INFO [databricks.labs.ucx.install] Creating ucx schemas... 18:09 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions 18:09 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-experimental 18:09 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups 18:09 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental 18:09 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-data-reconciliation 18:09 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables 18:09 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups 18:09 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-hiveserde-tables-in-place-experimental 18:09 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=scan-tables-in-mounts-experimental 18:09 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=experimental-workflow-linter 18:09 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment 18:09 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-tables-ctas 18:09 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=failing 18:09 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.f1Qn/README for the next steps. 18:09 DEBUG [databricks.labs.ucx.installer.workflows] starting migrate-external-hiveserde-tables-in-place-experimental job: https://DATABRICKS_HOST#job/614908448935600 18:06 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/hi0O 18:06 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.hiveserde_in_place_gueyb: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_gueyb 18:06 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.hiveserde_in_place_gueyb', metastore_id=None, name='hiveserde_in_place_gueyb', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 18:06 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_storage_dir fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_gueyb 18:06 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_gueyb.parquet_serde_gueyb: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_gueyb/parquet_serde_gueyb 18:06 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_gueyb.parquet_serde_gueyb', metastore_id=None, name='parquet_serde_gueyb', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='hiveserde_in_place_gueyb', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_gueyb/parquet_serde_gueyb', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 18:06 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_gueyb.orc_serde_gueyb: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_gueyb/orc_serde_gueyb 18:06 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_gueyb.orc_serde_gueyb', metastore_id=None, name='orc_serde_gueyb', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='hiveserde_in_place_gueyb', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_gueyb/orc_serde_gueyb', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 18:06 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_gueyb.avro_serde_gueyb: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_gueyb/avro_serde_gueyb 18:06 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_gueyb.avro_serde_gueyb', metastore_id=None, name='avro_serde_gueyb', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='hiveserde_in_place_gueyb', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_gueyb/avro_serde_gueyb', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 18:07 DEBUG [databricks.labs.ucx.mixins.fixtures] added catalog fixture: CatalogInfo(browse_only=False, catalog_type=, comment='', connection_name=None, created_at=1719943622378, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_ckyyw', isolation_mode=, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_ckyyw', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1719943622378, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d') 18:07 INFO [databricks.labs.ucx.mixins.fixtures] Schema ucx_ckyyw.hiveserde_in_place_gueyb: https://DATABRICKS_HOST/explore/data/ucx_ckyyw/hiveserde_in_place_gueyb 18:07 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_ckyyw', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_ckyyw.hiveserde_in_place_gueyb', metastore_id=None, name='hiveserde_in_place_gueyb', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 18:07 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_stica: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_stica 18:07 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_stica', metastore_id=None, name='ucx_stica', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 18:08 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.f1Qn/config.yml) doesn't exist. 18:08 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration 18:08 INFO [databricks.labs.ucx.installer.hms_lineage] HMS Lineage feature creates one system table named system.hms_to_uc_migration.table_access and helps in your migration process from HMS to UC by allowing you to programmatically query HMS lineage data. 18:08 INFO [databricks.labs.ucx.install] Fetching installations... 18:08 WARNING [databricks.labs.ucx.install] Existing installation at /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.f1Qn is corrupted. Skipping... 18:08 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy. 18:08 DEBUG [tests.integration.conftest] Waiting for clusters to start... 18:08 DEBUG [tests.integration.conftest] Waiting for clusters to start... 18:08 INFO [databricks.labs.ucx.install] Installing UCX v0.27.2+6020240702180829 18:08 INFO [databricks.labs.ucx.install] Creating ucx schemas... 18:09 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions 18:09 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-experimental 18:09 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups 18:09 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental 18:09 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-data-reconciliation 18:09 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables 18:09 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups 18:09 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-hiveserde-tables-in-place-experimental 18:09 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=scan-tables-in-mounts-experimental 18:09 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=experimental-workflow-linter 18:09 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment 18:09 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-tables-ctas 18:09 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=failing 18:09 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.f1Qn/README for the next steps. 18:09 DEBUG [databricks.labs.ucx.installer.workflows] starting migrate-external-hiveserde-tables-in-place-experimental job: https://DATABRICKS_HOST#job/614908448935600 18:14 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 make_storage_dir fixtures 18:14 DEBUG [databricks.labs.ucx.mixins.fixtures] removing make_storage_dir fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_gueyb 18:14 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 make_dbfs_data_copy fixtures 18:14 DEBUG [databricks.labs.ucx.mixins.fixtures] removing make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/hi0O 18:14 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 cluster fixtures 18:14 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 catalog fixtures 18:14 DEBUG [databricks.labs.ucx.mixins.fixtures] removing catalog fixture: CatalogInfo(browse_only=False, catalog_type=, comment='', connection_name=None, created_at=1719943622378, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_ckyyw', isolation_mode=, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_ckyyw', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1719943622378, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d') 18:14 INFO [databricks.labs.ucx.install] Deleting UCX v0.27.2+6020240702180829 from https://DATABRICKS_HOST 18:14 INFO [databricks.labs.ucx.install] Deleting inventory database ucx_stica 18:14 INFO [databricks.labs.ucx.install] Deleting jobs 18:14 INFO [databricks.labs.ucx.install] Deleting validate-groups-permissions job_id=1035504756051650. 18:14 INFO [databricks.labs.ucx.install] Deleting migrate-groups-experimental job_id=76360586899015. 18:14 INFO [databricks.labs.ucx.install] Deleting migrate-groups job_id=136797176303416. 18:14 INFO [databricks.labs.ucx.install] Deleting migrate-tables-in-mounts-experimental job_id=216008316444832. 18:14 INFO [databricks.labs.ucx.install] Deleting migrate-data-reconciliation job_id=10494338242904. 18:14 INFO [databricks.labs.ucx.install] Deleting migrate-tables job_id=753287296324178. 18:14 INFO [databricks.labs.ucx.install] Deleting remove-workspace-local-backup-groups job_id=241693541590681. 18:14 INFO [databricks.labs.ucx.install] Deleting migrate-external-hiveserde-tables-in-place-experimental job_id=614908448935600. 18:14 INFO [databricks.labs.ucx.install] Deleting scan-tables-in-mounts-experimental job_id=296479052950896. 18:14 INFO [databricks.labs.ucx.install] Deleting experimental-workflow-linter job_id=864174846237117. 18:14 INFO [databricks.labs.ucx.install] Deleting assessment job_id=1021223568986233. 18:14 INFO [databricks.labs.ucx.install] Deleting migrate-external-tables-ctas job_id=688146756331074. 18:14 INFO [databricks.labs.ucx.install] Deleting failing job_id=661016764032382. 18:14 INFO [databricks.labs.ucx.install] Deleting cluster policy 18:14 INFO [databricks.labs.ucx.install] Deleting secret scope 18:14 INFO [databricks.labs.ucx.install] UnInstalling UCX complete 18:14 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 workspace user fixtures 18:14 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 account group fixtures 18:14 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 workspace group fixtures 18:14 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 table fixtures 18:14 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 3 table fixtures 18:14 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_gueyb.parquet_serde_gueyb', metastore_id=None, name='parquet_serde_gueyb', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='hiveserde_in_place_gueyb', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_gueyb/parquet_serde_gueyb', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 18:14 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_gueyb.orc_serde_gueyb', metastore_id=None, name='orc_serde_gueyb', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='hiveserde_in_place_gueyb', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_gueyb/orc_serde_gueyb', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 18:14 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_gueyb.avro_serde_gueyb', metastore_id=None, name='avro_serde_gueyb', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070219'}, row_filter=None, schema_name='hiveserde_in_place_gueyb', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_gueyb/avro_serde_gueyb', table_constraints=None, table_id=None, table_type=, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None) 18:14 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 3 schema fixtures 18:14 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.hiveserde_in_place_gueyb', metastore_id=None, name='hiveserde_in_place_gueyb', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 18:14 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_ckyyw', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_ckyyw.hiveserde_in_place_gueyb', metastore_id=None, name='hiveserde_in_place_gueyb', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 18:14 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_stica', metastore_id=None, name='ucx_stica', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) [gw0] linux -- Python 3.10.14 /home/runner/work/ucx/ucx/.venv/bin/python ```
❌ test_running_real_workflow_linter_job: TimeoutError: timed out after 0:20:00: (21m54.803s) ``` TimeoutError: timed out after 0:20:00: [gw1] linux -- Python 3.10.14 /home/runner/work/ucx/ucx/.venv/bin/python 18:01 DEBUG [databricks.labs.ucx.mixins.fixtures] added directory fixture: /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/sdk-Xkfi 18:01 DEBUG [databricks.labs.ucx.mixins.fixtures] added notebook fixture: /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/sdk-Xkfi/notebook.ipynb 18:01 INFO [databricks.labs.ucx.mixins.fixtures] Job: https://DATABRICKS_HOST#job/932784898698018 18:01 DEBUG [databricks.labs.ucx.mixins.fixtures] added job fixture: CreateResponse(job_id=932784898698018) 18:01 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_sy5hj: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_sy5hj 18:01 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_sy5hj', metastore_id=None, name='ucx_sy5hj', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 18:01 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.FX7l/config.yml) doesn't exist. 18:01 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration 18:01 INFO [databricks.labs.ucx.installer.hms_lineage] HMS Lineage feature creates one system table named system.hms_to_uc_migration.table_access and helps in your migration process from HMS to UC by allowing you to programmatically query HMS lineage data. 18:01 INFO [databricks.labs.ucx.install] Fetching installations... 18:01 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy. 18:01 DEBUG [tests.integration.conftest] Waiting for clusters to start... 18:01 DEBUG [tests.integration.conftest] Waiting for clusters to start... 18:01 INFO [databricks.labs.ucx.install] Installing UCX v0.27.2+6020240702180155 18:01 INFO [databricks.labs.ucx.install] Creating ucx schemas... 18:03 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables 18:03 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-experimental 18:03 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental 18:03 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-hiveserde-tables-in-place-experimental 18:03 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=scan-tables-in-mounts-experimental 18:03 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-tables-ctas 18:03 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment 18:03 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions 18:03 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=failing 18:03 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups 18:03 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=experimental-workflow-linter 18:03 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-data-reconciliation 18:03 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups 18:03 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.FX7l/README for the next steps. 18:03 DEBUG [databricks.labs.ucx.installer.workflows] starting experimental-workflow-linter job: https://DATABRICKS_HOST#job/791604160972712 18:01 DEBUG [databricks.labs.ucx.mixins.fixtures] added directory fixture: /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/sdk-Xkfi 18:01 DEBUG [databricks.labs.ucx.mixins.fixtures] added notebook fixture: /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/sdk-Xkfi/notebook.ipynb 18:01 INFO [databricks.labs.ucx.mixins.fixtures] Job: https://DATABRICKS_HOST#job/932784898698018 18:01 DEBUG [databricks.labs.ucx.mixins.fixtures] added job fixture: CreateResponse(job_id=932784898698018) 18:01 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_sy5hj: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_sy5hj 18:01 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_sy5hj', metastore_id=None, name='ucx_sy5hj', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 18:01 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.FX7l/config.yml) doesn't exist. 18:01 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration 18:01 INFO [databricks.labs.ucx.installer.hms_lineage] HMS Lineage feature creates one system table named system.hms_to_uc_migration.table_access and helps in your migration process from HMS to UC by allowing you to programmatically query HMS lineage data. 18:01 INFO [databricks.labs.ucx.install] Fetching installations... 18:01 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy. 18:01 DEBUG [tests.integration.conftest] Waiting for clusters to start... 18:01 DEBUG [tests.integration.conftest] Waiting for clusters to start... 18:01 INFO [databricks.labs.ucx.install] Installing UCX v0.27.2+6020240702180155 18:01 INFO [databricks.labs.ucx.install] Creating ucx schemas... 18:03 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables 18:03 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-experimental 18:03 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental 18:03 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-hiveserde-tables-in-place-experimental 18:03 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=scan-tables-in-mounts-experimental 18:03 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-tables-ctas 18:03 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment 18:03 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions 18:03 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=failing 18:03 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups 18:03 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=experimental-workflow-linter 18:03 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-data-reconciliation 18:03 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups 18:03 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.FX7l/README for the next steps. 18:03 DEBUG [databricks.labs.ucx.installer.workflows] starting experimental-workflow-linter job: https://DATABRICKS_HOST#job/791604160972712 18:23 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 job fixtures 18:23 DEBUG [databricks.labs.ucx.mixins.fixtures] removing job fixture: CreateResponse(job_id=932784898698018) 18:23 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 directory fixtures 18:23 DEBUG [databricks.labs.ucx.mixins.fixtures] removing directory fixture: /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/sdk-Xkfi 18:23 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 notebook fixtures 18:23 DEBUG [databricks.labs.ucx.mixins.fixtures] removing notebook fixture: /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/sdk-Xkfi/notebook.ipynb 18:23 DEBUG [databricks.labs.ucx.mixins.fixtures] ignoring error while notebook /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/sdk-Xkfi/notebook.ipynb teardown: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/sdk-Xkfi/notebook.ipynb) doesn't exist. 18:23 INFO [databricks.labs.ucx.install] Deleting UCX v0.27.2+6020240702180155 from https://DATABRICKS_HOST 18:23 INFO [databricks.labs.ucx.install] Deleting inventory database ucx_sy5hj 18:23 INFO [databricks.labs.ucx.install] Deleting jobs 18:23 INFO [databricks.labs.ucx.install] Deleting migrate-tables job_id=971801852478076. 18:23 INFO [databricks.labs.ucx.install] Deleting migrate-groups-experimental job_id=1049197412088660. 18:23 INFO [databricks.labs.ucx.install] Deleting migrate-tables-in-mounts-experimental job_id=750929139362483. 18:23 INFO [databricks.labs.ucx.install] Deleting migrate-external-hiveserde-tables-in-place-experimental job_id=974333011107675. 18:23 INFO [databricks.labs.ucx.install] Deleting scan-tables-in-mounts-experimental job_id=539145098496622. 18:23 INFO [databricks.labs.ucx.install] Deleting migrate-external-tables-ctas job_id=844642106962447. 18:23 INFO [databricks.labs.ucx.install] Deleting assessment job_id=331246576575932. 18:23 INFO [databricks.labs.ucx.install] Deleting validate-groups-permissions job_id=730759039021536. 18:23 INFO [databricks.labs.ucx.install] Deleting failing job_id=946347749638163. 18:23 INFO [databricks.labs.ucx.install] Deleting remove-workspace-local-backup-groups job_id=44033678602597. 18:23 INFO [databricks.labs.ucx.install] Deleting experimental-workflow-linter job_id=791604160972712. 18:23 INFO [databricks.labs.ucx.install] Deleting migrate-data-reconciliation job_id=137335678553251. 18:23 INFO [databricks.labs.ucx.install] Deleting migrate-groups job_id=570174638506660. 18:23 INFO [databricks.labs.ucx.install] Deleting cluster policy 18:23 INFO [databricks.labs.ucx.install] Deleting secret scope 18:23 INFO [databricks.labs.ucx.install] UnInstalling UCX complete 18:23 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 workspace user fixtures 18:23 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 account group fixtures 18:23 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 workspace group fixtures 18:23 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 table fixtures 18:23 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 table fixtures 18:23 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 schema fixtures 18:23 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_sy5hj', metastore_id=None, name='ucx_sy5hj', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) [gw1] linux -- Python 3.10.14 /home/runner/work/ucx/ucx/.venv/bin/python ```
❌ test_tokens[True]: databricks.labs.blueprint.parallel.ManyError: Detected 1 failures: NotImplemented: Permission migration API is not enabled for workspace: 7342989205138882 (9.533s) ``` databricks.labs.blueprint.parallel.ManyError: Detected 1 failures: NotImplemented: Permission migration API is not enabled for workspace: 7342989205138882 18:30 INFO [databricks.sdk] Using Databricks Metadata Service authentication 18:30 INFO [databricks.sdk] Using Databricks Metadata Service authentication 18:30 INFO [databricks.labs.ucx.mixins.fixtures] Workspace group sdk-FSZF: https://DATABRICKS_HOST#setting/accounts/groups/260176751592012 18:30 DEBUG [databricks.labs.ucx.mixins.fixtures] added workspace group fixture: Group(display_name='sdk-FSZF', entitlements=[], external_id=None, groups=[], id='260176751592012', members=[], meta=ResourceMeta(resource_type='WorkspaceGroup'), roles=[], schemas=[]) 18:30 INFO [databricks.labs.ucx.mixins.fixtures] Account group sdk-3seH: https://accounts.CLOUD_ENVdatabricks.net/users/groups/565064144888444/members 18:30 DEBUG [databricks.labs.ucx.mixins.fixtures] added account group fixture: Group(display_name='sdk-3seH', entitlements=[], external_id=None, groups=[], id='565064144888444', members=[], meta=None, roles=[], schemas=[]) [gw1] linux -- Python 3.10.14 /home/runner/work/ucx/ucx/.venv/bin/python 18:30 INFO [databricks.sdk] Using Databricks Metadata Service authentication 18:30 INFO [databricks.sdk] Using Databricks Metadata Service authentication 18:30 INFO [databricks.labs.ucx.mixins.fixtures] Workspace group sdk-FSZF: https://DATABRICKS_HOST#setting/accounts/groups/260176751592012 18:30 DEBUG [databricks.labs.ucx.mixins.fixtures] added workspace group fixture: Group(display_name='sdk-FSZF', entitlements=[], external_id=None, groups=[], id='260176751592012', members=[], meta=ResourceMeta(resource_type='WorkspaceGroup'), roles=[], schemas=[]) 18:30 INFO [databricks.labs.ucx.mixins.fixtures] Account group sdk-3seH: https://accounts.CLOUD_ENVdatabricks.net/users/groups/565064144888444/members 18:30 DEBUG [databricks.labs.ucx.mixins.fixtures] added account group fixture: Group(display_name='sdk-3seH', entitlements=[], external_id=None, groups=[], id='565064144888444', members=[], meta=None, roles=[], schemas=[]) 18:30 DEBUG [databricks.labs.ucx.mixins.fixtures] added authorization permissions fixture: tokens [] -> [group_name sdk-FSZF CAN_USE] 18:30 INFO [databricks.labs.ucx.workspace_access.groups] Migrating permissions for 1 account groups. 18:30 INFO [databricks.labs.ucx.workspace_access.groups] Migrating permissions: sdk-FSZF (workspace) -> sdk-3seH (account) starting 18:30 ERROR [databricks.labs.ucx.workspace_access.groups] Migration of group permissions failed: sdk-FSZF (workspace) -> sdk-3seH (account) Traceback (most recent call last): File "/home/runner/work/ucx/ucx/src/databricks/labs/ucx/workspace_access/groups.py", line 115, in _apply_to_groups group_permissions = self._migrate_group_permissions_paginated(ws, name_in_workspace, name_in_account) File "/home/runner/work/ucx/ucx/src/databricks/labs/ucx/workspace_access/groups.py", line 138, in _migrate_group_permissions_paginated result = ws.permission_migration.migrate_permissions( File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/service/iam.py", line 2533, in migrate_permissions res = self._api.do('POST', '/api/2.0/permissionmigration', body=body, headers=headers) File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/core.py", line 153, in do response = retryable(self._perform)(method, File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/retries.py", line 54, in wrapper raise err File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/retries.py", line 33, in wrapper return func(*args, **kwargs) File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/core.py", line 264, in _perform raise self._make_nicer_error(response=response, **payload) from None databricks.sdk.errors.platform.NotImplemented: Permission migration API is not enabled for workspace: 7342989205138882 18:30 INFO [databricks.labs.ucx.workspace_access.groups] Migrated 0 permissions for 0/1 groups successfully. 18:30 ERROR [databricks.labs.ucx.workspace_access.groups] Migrating permissions failed for 1/1 groups. 18:30 INFO [databricks.sdk] Using Databricks Metadata Service authentication 18:30 INFO [databricks.sdk] Using Databricks Metadata Service authentication 18:30 INFO [databricks.labs.ucx.mixins.fixtures] Workspace group sdk-FSZF: https://DATABRICKS_HOST#setting/accounts/groups/260176751592012 18:30 DEBUG [databricks.labs.ucx.mixins.fixtures] added workspace group fixture: Group(display_name='sdk-FSZF', entitlements=[], external_id=None, groups=[], id='260176751592012', members=[], meta=ResourceMeta(resource_type='WorkspaceGroup'), roles=[], schemas=[]) 18:30 INFO [databricks.labs.ucx.mixins.fixtures] Account group sdk-3seH: https://accounts.CLOUD_ENVdatabricks.net/users/groups/565064144888444/members 18:30 DEBUG [databricks.labs.ucx.mixins.fixtures] added account group fixture: Group(display_name='sdk-3seH', entitlements=[], external_id=None, groups=[], id='565064144888444', members=[], meta=None, roles=[], schemas=[]) 18:30 DEBUG [databricks.labs.ucx.mixins.fixtures] added authorization permissions fixture: tokens [] -> [group_name sdk-FSZF CAN_USE] 18:30 INFO [databricks.labs.ucx.workspace_access.groups] Migrating permissions for 1 account groups. 18:30 INFO [databricks.labs.ucx.workspace_access.groups] Migrating permissions: sdk-FSZF (workspace) -> sdk-3seH (account) starting 18:30 ERROR [databricks.labs.ucx.workspace_access.groups] Migration of group permissions failed: sdk-FSZF (workspace) -> sdk-3seH (account) Traceback (most recent call last): File "/home/runner/work/ucx/ucx/src/databricks/labs/ucx/workspace_access/groups.py", line 115, in _apply_to_groups group_permissions = self._migrate_group_permissions_paginated(ws, name_in_workspace, name_in_account) File "/home/runner/work/ucx/ucx/src/databricks/labs/ucx/workspace_access/groups.py", line 138, in _migrate_group_permissions_paginated result = ws.permission_migration.migrate_permissions( File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/service/iam.py", line 2533, in migrate_permissions res = self._api.do('POST', '/api/2.0/permissionmigration', body=body, headers=headers) File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/core.py", line 153, in do response = retryable(self._perform)(method, File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/retries.py", line 54, in wrapper raise err File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/retries.py", line 33, in wrapper return func(*args, **kwargs) File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/core.py", line 264, in _perform raise self._make_nicer_error(response=response, **payload) from None databricks.sdk.errors.platform.NotImplemented: Permission migration API is not enabled for workspace: 7342989205138882 18:30 INFO [databricks.labs.ucx.workspace_access.groups] Migrated 0 permissions for 0/1 groups successfully. 18:30 ERROR [databricks.labs.ucx.workspace_access.groups] Migrating permissions failed for 1/1 groups. 18:30 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 authorization permissions fixtures 18:30 DEBUG [databricks.labs.ucx.mixins.fixtures] removing authorization permissions fixture: tokens [] -> [group_name sdk-FSZF CAN_USE] 18:30 DEBUG [databricks.labs.ucx.mixins.fixtures] ignoring error while authorization permissions tokens [] -> [group_name sdk-FSZF CAN_USE] teardown: Cannot remove admins's CAN_MANAGE permission on tokens 18:30 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 account group fixtures 18:30 DEBUG [databricks.labs.ucx.mixins.fixtures] removing account group fixture: Group(display_name='sdk-3seH', entitlements=[], external_id=None, groups=[], id='565064144888444', members=[], meta=None, roles=[], schemas=[]) 18:30 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 workspace group fixtures 18:30 DEBUG [databricks.labs.ucx.mixins.fixtures] removing workspace group fixture: Group(display_name='sdk-FSZF', entitlements=[], external_id=None, groups=[], id='260176751592012', members=[], meta=ResourceMeta(resource_type='WorkspaceGroup'), roles=[], schemas=[]) [gw1] linux -- Python 3.10.14 /home/runner/work/ucx/ucx/.venv/bin/python ```
❌ test_endpoints[True]: databricks.labs.blueprint.parallel.ManyError: Detected 1 failures: NotImplemented: Permission migration API is not enabled for workspace: 7342989205138882 (9.584s) ``` databricks.labs.blueprint.parallel.ManyError: Detected 1 failures: NotImplemented: Permission migration API is not enabled for workspace: 7342989205138882 18:31 INFO [databricks.sdk] Using Databricks Metadata Service authentication 18:31 INFO [databricks.sdk] Using Databricks Metadata Service authentication 18:31 INFO [databricks.labs.ucx.mixins.fixtures] Workspace group sdk-8y6J: https://DATABRICKS_HOST#setting/accounts/groups/1089038680325189 18:31 DEBUG [databricks.labs.ucx.mixins.fixtures] added workspace group fixture: Group(display_name='sdk-8y6J', entitlements=[], external_id=None, groups=[], id='1089038680325189', members=[], meta=ResourceMeta(resource_type='WorkspaceGroup'), roles=[], schemas=[]) 18:31 INFO [databricks.labs.ucx.mixins.fixtures] Account group sdk-NhTl: https://accounts.CLOUD_ENVdatabricks.net/users/groups/242870607189947/members 18:31 DEBUG [databricks.labs.ucx.mixins.fixtures] added account group fixture: Group(display_name='sdk-NhTl', entitlements=[], external_id=None, groups=[], id='242870607189947', members=[], meta=None, roles=[], schemas=[]) [gw1] linux -- Python 3.10.14 /home/runner/work/ucx/ucx/.venv/bin/python 18:31 INFO [databricks.sdk] Using Databricks Metadata Service authentication 18:31 INFO [databricks.sdk] Using Databricks Metadata Service authentication 18:31 INFO [databricks.labs.ucx.mixins.fixtures] Workspace group sdk-8y6J: https://DATABRICKS_HOST#setting/accounts/groups/1089038680325189 18:31 DEBUG [databricks.labs.ucx.mixins.fixtures] added workspace group fixture: Group(display_name='sdk-8y6J', entitlements=[], external_id=None, groups=[], id='1089038680325189', members=[], meta=ResourceMeta(resource_type='WorkspaceGroup'), roles=[], schemas=[]) 18:31 INFO [databricks.labs.ucx.mixins.fixtures] Account group sdk-NhTl: https://accounts.CLOUD_ENVdatabricks.net/users/groups/242870607189947/members 18:31 DEBUG [databricks.labs.ucx.mixins.fixtures] added account group fixture: Group(display_name='sdk-NhTl', entitlements=[], external_id=None, groups=[], id='242870607189947', members=[], meta=None, roles=[], schemas=[]) 18:31 DEBUG [databricks.labs.ucx.mixins.fixtures] added model fixture: ModelDatabricks(creation_timestamp=1719945069331, description=None, id='870acb47272b47419058a2664605a1ed', last_updated_timestamp=1719945069331, latest_versions=[], name='sdk-2cDG', permission_level=, tags=[], user_id='0a330eb5-dd51-4d97-b6e4-c474356b1d5d') 18:31 DEBUG [databricks.labs.ucx.mixins.fixtures] added Serving endpoint fixture: 18:31 DEBUG [databricks.labs.ucx.mixins.fixtures] added serving_endpoint permissions fixture: 0c1cd4b4cfa140b685c24a6aa773e988 [group_name admins CAN_MANAGE] -> [group_name sdk-8y6J CAN_QUERY] 18:31 INFO [databricks.labs.ucx.workspace_access.groups] Migrating permissions for 1 account groups. 18:31 INFO [databricks.labs.ucx.workspace_access.groups] Migrating permissions: sdk-8y6J (workspace) -> sdk-NhTl (account) starting 18:31 ERROR [databricks.labs.ucx.workspace_access.groups] Migration of group permissions failed: sdk-8y6J (workspace) -> sdk-NhTl (account) Traceback (most recent call last): File "/home/runner/work/ucx/ucx/src/databricks/labs/ucx/workspace_access/groups.py", line 115, in _apply_to_groups group_permissions = self._migrate_group_permissions_paginated(ws, name_in_workspace, name_in_account) File "/home/runner/work/ucx/ucx/src/databricks/labs/ucx/workspace_access/groups.py", line 138, in _migrate_group_permissions_paginated result = ws.permission_migration.migrate_permissions( File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/service/iam.py", line 2533, in migrate_permissions res = self._api.do('POST', '/api/2.0/permissionmigration', body=body, headers=headers) File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/core.py", line 153, in do response = retryable(self._perform)(method, File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/retries.py", line 54, in wrapper raise err File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/retries.py", line 33, in wrapper return func(*args, **kwargs) File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/core.py", line 264, in _perform raise self._make_nicer_error(response=response, **payload) from None databricks.sdk.errors.platform.NotImplemented: Permission migration API is not enabled for workspace: 7342989205138882 18:31 INFO [databricks.labs.ucx.workspace_access.groups] Migrated 0 permissions for 0/1 groups successfully. 18:31 ERROR [databricks.labs.ucx.workspace_access.groups] Migrating permissions failed for 1/1 groups. 18:31 INFO [databricks.sdk] Using Databricks Metadata Service authentication 18:31 INFO [databricks.sdk] Using Databricks Metadata Service authentication 18:31 INFO [databricks.labs.ucx.mixins.fixtures] Workspace group sdk-8y6J: https://DATABRICKS_HOST#setting/accounts/groups/1089038680325189 18:31 DEBUG [databricks.labs.ucx.mixins.fixtures] added workspace group fixture: Group(display_name='sdk-8y6J', entitlements=[], external_id=None, groups=[], id='1089038680325189', members=[], meta=ResourceMeta(resource_type='WorkspaceGroup'), roles=[], schemas=[]) 18:31 INFO [databricks.labs.ucx.mixins.fixtures] Account group sdk-NhTl: https://accounts.CLOUD_ENVdatabricks.net/users/groups/242870607189947/members 18:31 DEBUG [databricks.labs.ucx.mixins.fixtures] added account group fixture: Group(display_name='sdk-NhTl', entitlements=[], external_id=None, groups=[], id='242870607189947', members=[], meta=None, roles=[], schemas=[]) 18:31 DEBUG [databricks.labs.ucx.mixins.fixtures] added model fixture: ModelDatabricks(creation_timestamp=1719945069331, description=None, id='870acb47272b47419058a2664605a1ed', last_updated_timestamp=1719945069331, latest_versions=[], name='sdk-2cDG', permission_level=, tags=[], user_id='0a330eb5-dd51-4d97-b6e4-c474356b1d5d') 18:31 DEBUG [databricks.labs.ucx.mixins.fixtures] added Serving endpoint fixture: 18:31 DEBUG [databricks.labs.ucx.mixins.fixtures] added serving_endpoint permissions fixture: 0c1cd4b4cfa140b685c24a6aa773e988 [group_name admins CAN_MANAGE] -> [group_name sdk-8y6J CAN_QUERY] 18:31 INFO [databricks.labs.ucx.workspace_access.groups] Migrating permissions for 1 account groups. 18:31 INFO [databricks.labs.ucx.workspace_access.groups] Migrating permissions: sdk-8y6J (workspace) -> sdk-NhTl (account) starting 18:31 ERROR [databricks.labs.ucx.workspace_access.groups] Migration of group permissions failed: sdk-8y6J (workspace) -> sdk-NhTl (account) Traceback (most recent call last): File "/home/runner/work/ucx/ucx/src/databricks/labs/ucx/workspace_access/groups.py", line 115, in _apply_to_groups group_permissions = self._migrate_group_permissions_paginated(ws, name_in_workspace, name_in_account) File "/home/runner/work/ucx/ucx/src/databricks/labs/ucx/workspace_access/groups.py", line 138, in _migrate_group_permissions_paginated result = ws.permission_migration.migrate_permissions( File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/service/iam.py", line 2533, in migrate_permissions res = self._api.do('POST', '/api/2.0/permissionmigration', body=body, headers=headers) File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/core.py", line 153, in do response = retryable(self._perform)(method, File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/retries.py", line 54, in wrapper raise err File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/retries.py", line 33, in wrapper return func(*args, **kwargs) File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/core.py", line 264, in _perform raise self._make_nicer_error(response=response, **payload) from None databricks.sdk.errors.platform.NotImplemented: Permission migration API is not enabled for workspace: 7342989205138882 18:31 INFO [databricks.labs.ucx.workspace_access.groups] Migrated 0 permissions for 0/1 groups successfully. 18:31 ERROR [databricks.labs.ucx.workspace_access.groups] Migrating permissions failed for 1/1 groups. 18:31 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 serving_endpoint permissions fixtures 18:31 DEBUG [databricks.labs.ucx.mixins.fixtures] removing serving_endpoint permissions fixture: 0c1cd4b4cfa140b685c24a6aa773e988 [group_name admins CAN_MANAGE] -> [group_name sdk-8y6J CAN_QUERY] 18:31 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 Serving endpoint fixtures 18:31 DEBUG [databricks.labs.ucx.mixins.fixtures] removing Serving endpoint fixture: 18:31 DEBUG [databricks.labs.ucx.mixins.fixtures] ignoring error while Serving endpoint teardown: Endpoint does not exist. 18:31 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 model fixtures 18:31 DEBUG [databricks.labs.ucx.mixins.fixtures] removing model fixture: ModelDatabricks(creation_timestamp=1719945069331, description=None, id='870acb47272b47419058a2664605a1ed', last_updated_timestamp=1719945069331, latest_versions=[], name='sdk-2cDG', permission_level=, tags=[], user_id='0a330eb5-dd51-4d97-b6e4-c474356b1d5d') 18:31 DEBUG [databricks.labs.ucx.mixins.fixtures] ignoring error while model ModelDatabricks(creation_timestamp=1719945069331, description=None, id='870acb47272b47419058a2664605a1ed', last_updated_timestamp=1719945069331, latest_versions=[], name='sdk-2cDG', permission_level=, tags=[], user_id='0a330eb5-dd51-4d97-b6e4-c474356b1d5d') teardown: RegisteredModel '870acb47272b47419058a2664605a1ed' does not exist. It might have been deleted. 18:31 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 account group fixtures 18:31 DEBUG [databricks.labs.ucx.mixins.fixtures] removing account group fixture: Group(display_name='sdk-NhTl', entitlements=[], external_id=None, groups=[], id='242870607189947', members=[], meta=None, roles=[], schemas=[]) 18:31 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 workspace group fixtures 18:31 DEBUG [databricks.labs.ucx.mixins.fixtures] removing workspace group fixture: Group(display_name='sdk-8y6J', entitlements=[], external_id=None, groups=[], id='1089038680325189', members=[], meta=ResourceMeta(resource_type='WorkspaceGroup'), roles=[], schemas=[]) [gw1] linux -- Python 3.10.14 /home/runner/work/ucx/ucx/.venv/bin/python ```
❌ test_feature_tables[True]: databricks.labs.blueprint.parallel.ManyError: Detected 1 failures: NotImplemented: Permission migration API is not enabled for workspace: 7342989205138882 (11.815s) ``` databricks.labs.blueprint.parallel.ManyError: Detected 1 failures: NotImplemented: Permission migration API is not enabled for workspace: 7342989205138882 18:31 INFO [databricks.sdk] Using Databricks Metadata Service authentication 18:31 INFO [databricks.sdk] Using Databricks Metadata Service authentication 18:32 INFO [databricks.labs.ucx.mixins.fixtures] Workspace group sdk-cus3: https://DATABRICKS_HOST#setting/accounts/groups/872249740590174 18:32 DEBUG [databricks.labs.ucx.mixins.fixtures] added workspace group fixture: Group(display_name='sdk-cus3', entitlements=[], external_id=None, groups=[], id='872249740590174', members=[], meta=ResourceMeta(resource_type='WorkspaceGroup'), roles=[], schemas=[]) 18:32 INFO [databricks.labs.ucx.mixins.fixtures] Account group sdk-cwaM: https://accounts.CLOUD_ENVdatabricks.net/users/groups/394061995058781/members 18:32 DEBUG [databricks.labs.ucx.mixins.fixtures] added account group fixture: Group(display_name='sdk-cwaM', entitlements=[], external_id=None, groups=[], id='394061995058781', members=[], meta=None, roles=[], schemas=[]) [gw1] linux -- Python 3.10.14 /home/runner/work/ucx/ucx/.venv/bin/python 18:31 INFO [databricks.sdk] Using Databricks Metadata Service authentication 18:31 INFO [databricks.sdk] Using Databricks Metadata Service authentication 18:32 INFO [databricks.labs.ucx.mixins.fixtures] Workspace group sdk-cus3: https://DATABRICKS_HOST#setting/accounts/groups/872249740590174 18:32 DEBUG [databricks.labs.ucx.mixins.fixtures] added workspace group fixture: Group(display_name='sdk-cus3', entitlements=[], external_id=None, groups=[], id='872249740590174', members=[], meta=ResourceMeta(resource_type='WorkspaceGroup'), roles=[], schemas=[]) 18:32 INFO [databricks.labs.ucx.mixins.fixtures] Account group sdk-cwaM: https://accounts.CLOUD_ENVdatabricks.net/users/groups/394061995058781/members 18:32 DEBUG [databricks.labs.ucx.mixins.fixtures] added account group fixture: Group(display_name='sdk-cwaM', entitlements=[], external_id=None, groups=[], id='394061995058781', members=[], meta=None, roles=[], schemas=[]) 18:32 DEBUG [databricks.labs.ucx.mixins.fixtures] added Feature table fixture: {'name': 'SSz4W0.F91tj5', 'creation_timestamp': 1719945122332, 'last_updated_timestamp': 1719945122332, 'creator_id': '0a330eb5-dd51-4d97-b6e4-c474356b1d5d', 'primary_keys': ['pk'], 'features': ['pk'], 'last_update_user_id': '0a330eb5-dd51-4d97-b6e4-c474356b1d5d', 'id': 'e205680d1a2642a081c5309e52bd5cd3', 'permission_level': 'CAN_MANAGE', 'is_imported': False} 18:32 DEBUG [databricks.labs.ucx.mixins.fixtures] added feature_table permissions fixture: e205680d1a2642a081c5309e52bd5cd3 [group_name admins CAN_MANAGE] -> [group_name sdk-cus3 CAN_EDIT_METADATA] 18:32 INFO [databricks.labs.ucx.workspace_access.groups] Migrating permissions for 1 account groups. 18:32 INFO [databricks.labs.ucx.workspace_access.groups] Migrating permissions: sdk-cus3 (workspace) -> sdk-cwaM (account) starting 18:32 ERROR [databricks.labs.ucx.workspace_access.groups] Migration of group permissions failed: sdk-cus3 (workspace) -> sdk-cwaM (account) Traceback (most recent call last): File "/home/runner/work/ucx/ucx/src/databricks/labs/ucx/workspace_access/groups.py", line 115, in _apply_to_groups group_permissions = self._migrate_group_permissions_paginated(ws, name_in_workspace, name_in_account) File "/home/runner/work/ucx/ucx/src/databricks/labs/ucx/workspace_access/groups.py", line 138, in _migrate_group_permissions_paginated result = ws.permission_migration.migrate_permissions( File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/service/iam.py", line 2533, in migrate_permissions res = self._api.do('POST', '/api/2.0/permissionmigration', body=body, headers=headers) File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/core.py", line 153, in do response = retryable(self._perform)(method, File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/retries.py", line 54, in wrapper raise err File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/retries.py", line 33, in wrapper return func(*args, **kwargs) File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/core.py", line 264, in _perform raise self._make_nicer_error(response=response, **payload) from None databricks.sdk.errors.platform.NotImplemented: Permission migration API is not enabled for workspace: 7342989205138882 18:32 INFO [databricks.labs.ucx.workspace_access.groups] Migrated 0 permissions for 0/1 groups successfully. 18:32 ERROR [databricks.labs.ucx.workspace_access.groups] Migrating permissions failed for 1/1 groups. 18:31 INFO [databricks.sdk] Using Databricks Metadata Service authentication 18:31 INFO [databricks.sdk] Using Databricks Metadata Service authentication 18:32 INFO [databricks.labs.ucx.mixins.fixtures] Workspace group sdk-cus3: https://DATABRICKS_HOST#setting/accounts/groups/872249740590174 18:32 DEBUG [databricks.labs.ucx.mixins.fixtures] added workspace group fixture: Group(display_name='sdk-cus3', entitlements=[], external_id=None, groups=[], id='872249740590174', members=[], meta=ResourceMeta(resource_type='WorkspaceGroup'), roles=[], schemas=[]) 18:32 INFO [databricks.labs.ucx.mixins.fixtures] Account group sdk-cwaM: https://accounts.CLOUD_ENVdatabricks.net/users/groups/394061995058781/members 18:32 DEBUG [databricks.labs.ucx.mixins.fixtures] added account group fixture: Group(display_name='sdk-cwaM', entitlements=[], external_id=None, groups=[], id='394061995058781', members=[], meta=None, roles=[], schemas=[]) 18:32 DEBUG [databricks.labs.ucx.mixins.fixtures] added Feature table fixture: {'name': 'SSz4W0.F91tj5', 'creation_timestamp': 1719945122332, 'last_updated_timestamp': 1719945122332, 'creator_id': '0a330eb5-dd51-4d97-b6e4-c474356b1d5d', 'primary_keys': ['pk'], 'features': ['pk'], 'last_update_user_id': '0a330eb5-dd51-4d97-b6e4-c474356b1d5d', 'id': 'e205680d1a2642a081c5309e52bd5cd3', 'permission_level': 'CAN_MANAGE', 'is_imported': False} 18:32 DEBUG [databricks.labs.ucx.mixins.fixtures] added feature_table permissions fixture: e205680d1a2642a081c5309e52bd5cd3 [group_name admins CAN_MANAGE] -> [group_name sdk-cus3 CAN_EDIT_METADATA] 18:32 INFO [databricks.labs.ucx.workspace_access.groups] Migrating permissions for 1 account groups. 18:32 INFO [databricks.labs.ucx.workspace_access.groups] Migrating permissions: sdk-cus3 (workspace) -> sdk-cwaM (account) starting 18:32 ERROR [databricks.labs.ucx.workspace_access.groups] Migration of group permissions failed: sdk-cus3 (workspace) -> sdk-cwaM (account) Traceback (most recent call last): File "/home/runner/work/ucx/ucx/src/databricks/labs/ucx/workspace_access/groups.py", line 115, in _apply_to_groups group_permissions = self._migrate_group_permissions_paginated(ws, name_in_workspace, name_in_account) File "/home/runner/work/ucx/ucx/src/databricks/labs/ucx/workspace_access/groups.py", line 138, in _migrate_group_permissions_paginated result = ws.permission_migration.migrate_permissions( File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/service/iam.py", line 2533, in migrate_permissions res = self._api.do('POST', '/api/2.0/permissionmigration', body=body, headers=headers) File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/core.py", line 153, in do response = retryable(self._perform)(method, File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/retries.py", line 54, in wrapper raise err File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/retries.py", line 33, in wrapper return func(*args, **kwargs) File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/core.py", line 264, in _perform raise self._make_nicer_error(response=response, **payload) from None databricks.sdk.errors.platform.NotImplemented: Permission migration API is not enabled for workspace: 7342989205138882 18:32 INFO [databricks.labs.ucx.workspace_access.groups] Migrated 0 permissions for 0/1 groups successfully. 18:32 ERROR [databricks.labs.ucx.workspace_access.groups] Migrating permissions failed for 1/1 groups. 18:32 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 feature_table permissions fixtures 18:32 DEBUG [databricks.labs.ucx.mixins.fixtures] removing feature_table permissions fixture: e205680d1a2642a081c5309e52bd5cd3 [group_name admins CAN_MANAGE] -> [group_name sdk-cus3 CAN_EDIT_METADATA] 18:32 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 Feature table fixtures 18:32 DEBUG [databricks.labs.ucx.mixins.fixtures] removing Feature table fixture: {'name': 'SSz4W0.F91tj5', 'creation_timestamp': 1719945122332, 'last_updated_timestamp': 1719945122332, 'creator_id': '0a330eb5-dd51-4d97-b6e4-c474356b1d5d', 'primary_keys': ['pk'], 'features': ['pk'], 'last_update_user_id': '0a330eb5-dd51-4d97-b6e4-c474356b1d5d', 'id': 'e205680d1a2642a081c5309e52bd5cd3', 'permission_level': 'CAN_MANAGE', 'is_imported': False} 18:32 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 account group fixtures 18:32 DEBUG [databricks.labs.ucx.mixins.fixtures] removing account group fixture: Group(display_name='sdk-cwaM', entitlements=[], external_id=None, groups=[], id='394061995058781', members=[], meta=None, roles=[], schemas=[]) 18:32 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 workspace group fixtures 18:32 DEBUG [databricks.labs.ucx.mixins.fixtures] removing workspace group fixture: Group(display_name='sdk-cus3', entitlements=[], external_id=None, groups=[], id='872249740590174', members=[], meta=ResourceMeta(resource_type='WorkspaceGroup'), roles=[], schemas=[]) [gw1] linux -- Python 3.10.14 /home/runner/work/ucx/ucx/.venv/bin/python ```
❌ test_feature_store_root_page[True]: databricks.labs.blueprint.parallel.ManyError: Detected 1 failures: NotImplemented: Permission migration API is not enabled for workspace: 7342989205138882 (9.447s) ``` databricks.labs.blueprint.parallel.ManyError: Detected 1 failures: NotImplemented: Permission migration API is not enabled for workspace: 7342989205138882 18:32 INFO [databricks.sdk] Using Databricks Metadata Service authentication 18:32 INFO [databricks.sdk] Using Databricks Metadata Service authentication 18:32 INFO [databricks.labs.ucx.mixins.fixtures] Workspace group sdk-SXFC: https://DATABRICKS_HOST#setting/accounts/groups/757757440387499 18:32 DEBUG [databricks.labs.ucx.mixins.fixtures] added workspace group fixture: Group(display_name='sdk-SXFC', entitlements=[], external_id=None, groups=[], id='757757440387499', members=[], meta=ResourceMeta(resource_type='WorkspaceGroup'), roles=[], schemas=[]) 18:32 INFO [databricks.labs.ucx.mixins.fixtures] Account group sdk-ew0z: https://accounts.CLOUD_ENVdatabricks.net/users/groups/693893000406016/members 18:32 DEBUG [databricks.labs.ucx.mixins.fixtures] added account group fixture: Group(display_name='sdk-ew0z', entitlements=[], external_id=None, groups=[], id='693893000406016', members=[], meta=None, roles=[], schemas=[]) [gw1] linux -- Python 3.10.14 /home/runner/work/ucx/ucx/.venv/bin/python 18:32 INFO [databricks.sdk] Using Databricks Metadata Service authentication 18:32 INFO [databricks.sdk] Using Databricks Metadata Service authentication 18:32 INFO [databricks.labs.ucx.mixins.fixtures] Workspace group sdk-SXFC: https://DATABRICKS_HOST#setting/accounts/groups/757757440387499 18:32 DEBUG [databricks.labs.ucx.mixins.fixtures] added workspace group fixture: Group(display_name='sdk-SXFC', entitlements=[], external_id=None, groups=[], id='757757440387499', members=[], meta=ResourceMeta(resource_type='WorkspaceGroup'), roles=[], schemas=[]) 18:32 INFO [databricks.labs.ucx.mixins.fixtures] Account group sdk-ew0z: https://accounts.CLOUD_ENVdatabricks.net/users/groups/693893000406016/members 18:32 DEBUG [databricks.labs.ucx.mixins.fixtures] added account group fixture: Group(display_name='sdk-ew0z', entitlements=[], external_id=None, groups=[], id='693893000406016', members=[], meta=None, roles=[], schemas=[]) 18:32 INFO [databricks.labs.ucx.workspace_access.groups] Migrating permissions for 1 account groups. 18:32 INFO [databricks.labs.ucx.workspace_access.groups] Migrating permissions: sdk-SXFC (workspace) -> sdk-ew0z (account) starting 18:32 ERROR [databricks.labs.ucx.workspace_access.groups] Migration of group permissions failed: sdk-SXFC (workspace) -> sdk-ew0z (account) Traceback (most recent call last): File "/home/runner/work/ucx/ucx/src/databricks/labs/ucx/workspace_access/groups.py", line 115, in _apply_to_groups group_permissions = self._migrate_group_permissions_paginated(ws, name_in_workspace, name_in_account) File "/home/runner/work/ucx/ucx/src/databricks/labs/ucx/workspace_access/groups.py", line 138, in _migrate_group_permissions_paginated result = ws.permission_migration.migrate_permissions( File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/service/iam.py", line 2533, in migrate_permissions res = self._api.do('POST', '/api/2.0/permissionmigration', body=body, headers=headers) File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/core.py", line 153, in do response = retryable(self._perform)(method, File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/retries.py", line 54, in wrapper raise err File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/retries.py", line 33, in wrapper return func(*args, **kwargs) File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/core.py", line 264, in _perform raise self._make_nicer_error(response=response, **payload) from None databricks.sdk.errors.platform.NotImplemented: Permission migration API is not enabled for workspace: 7342989205138882 18:32 INFO [databricks.labs.ucx.workspace_access.groups] Migrated 0 permissions for 0/1 groups successfully. 18:32 ERROR [databricks.labs.ucx.workspace_access.groups] Migrating permissions failed for 1/1 groups. 18:32 INFO [databricks.sdk] Using Databricks Metadata Service authentication 18:32 INFO [databricks.sdk] Using Databricks Metadata Service authentication 18:32 INFO [databricks.labs.ucx.mixins.fixtures] Workspace group sdk-SXFC: https://DATABRICKS_HOST#setting/accounts/groups/757757440387499 18:32 DEBUG [databricks.labs.ucx.mixins.fixtures] added workspace group fixture: Group(display_name='sdk-SXFC', entitlements=[], external_id=None, groups=[], id='757757440387499', members=[], meta=ResourceMeta(resource_type='WorkspaceGroup'), roles=[], schemas=[]) 18:32 INFO [databricks.labs.ucx.mixins.fixtures] Account group sdk-ew0z: https://accounts.CLOUD_ENVdatabricks.net/users/groups/693893000406016/members 18:32 DEBUG [databricks.labs.ucx.mixins.fixtures] added account group fixture: Group(display_name='sdk-ew0z', entitlements=[], external_id=None, groups=[], id='693893000406016', members=[], meta=None, roles=[], schemas=[]) 18:32 INFO [databricks.labs.ucx.workspace_access.groups] Migrating permissions for 1 account groups. 18:32 INFO [databricks.labs.ucx.workspace_access.groups] Migrating permissions: sdk-SXFC (workspace) -> sdk-ew0z (account) starting 18:32 ERROR [databricks.labs.ucx.workspace_access.groups] Migration of group permissions failed: sdk-SXFC (workspace) -> sdk-ew0z (account) Traceback (most recent call last): File "/home/runner/work/ucx/ucx/src/databricks/labs/ucx/workspace_access/groups.py", line 115, in _apply_to_groups group_permissions = self._migrate_group_permissions_paginated(ws, name_in_workspace, name_in_account) File "/home/runner/work/ucx/ucx/src/databricks/labs/ucx/workspace_access/groups.py", line 138, in _migrate_group_permissions_paginated result = ws.permission_migration.migrate_permissions( File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/service/iam.py", line 2533, in migrate_permissions res = self._api.do('POST', '/api/2.0/permissionmigration', body=body, headers=headers) File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/core.py", line 153, in do response = retryable(self._perform)(method, File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/retries.py", line 54, in wrapper raise err File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/retries.py", line 33, in wrapper return func(*args, **kwargs) File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/core.py", line 264, in _perform raise self._make_nicer_error(response=response, **payload) from None databricks.sdk.errors.platform.NotImplemented: Permission migration API is not enabled for workspace: 7342989205138882 18:32 INFO [databricks.labs.ucx.workspace_access.groups] Migrated 0 permissions for 0/1 groups successfully. 18:32 ERROR [databricks.labs.ucx.workspace_access.groups] Migrating permissions failed for 1/1 groups. 18:32 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 account group fixtures 18:32 DEBUG [databricks.labs.ucx.mixins.fixtures] removing account group fixture: Group(display_name='sdk-ew0z', entitlements=[], external_id=None, groups=[], id='693893000406016', members=[], meta=None, roles=[], schemas=[]) 18:32 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 workspace group fixtures 18:32 DEBUG [databricks.labs.ucx.mixins.fixtures] removing workspace group fixture: Group(display_name='sdk-SXFC', entitlements=[], external_id=None, groups=[], id='757757440387499', members=[], meta=ResourceMeta(resource_type='WorkspaceGroup'), roles=[], schemas=[]) [gw1] linux -- Python 3.10.14 /home/runner/work/ucx/ucx/.venv/bin/python ```
❌ test_models_root_page[True]: databricks.labs.blueprint.parallel.ManyError: Detected 1 failures: NotImplemented: Permission migration API is not enabled for workspace: 7342989205138882 (8.838s) ``` databricks.labs.blueprint.parallel.ManyError: Detected 1 failures: NotImplemented: Permission migration API is not enabled for workspace: 7342989205138882 18:32 INFO [databricks.sdk] Using Databricks Metadata Service authentication 18:32 INFO [databricks.sdk] Using Databricks Metadata Service authentication 18:32 INFO [databricks.labs.ucx.mixins.fixtures] Workspace group sdk-YY57: https://DATABRICKS_HOST#setting/accounts/groups/702577790255713 18:32 DEBUG [databricks.labs.ucx.mixins.fixtures] added workspace group fixture: Group(display_name='sdk-YY57', entitlements=[], external_id=None, groups=[], id='702577790255713', members=[], meta=ResourceMeta(resource_type='WorkspaceGroup'), roles=[], schemas=[]) 18:32 INFO [databricks.labs.ucx.mixins.fixtures] Account group sdk-oW5V: https://accounts.CLOUD_ENVdatabricks.net/users/groups/143570554656729/members 18:32 DEBUG [databricks.labs.ucx.mixins.fixtures] added account group fixture: Group(display_name='sdk-oW5V', entitlements=[], external_id=None, groups=[], id='143570554656729', members=[], meta=None, roles=[], schemas=[]) [gw1] linux -- Python 3.10.14 /home/runner/work/ucx/ucx/.venv/bin/python 18:32 INFO [databricks.sdk] Using Databricks Metadata Service authentication 18:32 INFO [databricks.sdk] Using Databricks Metadata Service authentication 18:32 INFO [databricks.labs.ucx.mixins.fixtures] Workspace group sdk-YY57: https://DATABRICKS_HOST#setting/accounts/groups/702577790255713 18:32 DEBUG [databricks.labs.ucx.mixins.fixtures] added workspace group fixture: Group(display_name='sdk-YY57', entitlements=[], external_id=None, groups=[], id='702577790255713', members=[], meta=ResourceMeta(resource_type='WorkspaceGroup'), roles=[], schemas=[]) 18:32 INFO [databricks.labs.ucx.mixins.fixtures] Account group sdk-oW5V: https://accounts.CLOUD_ENVdatabricks.net/users/groups/143570554656729/members 18:32 DEBUG [databricks.labs.ucx.mixins.fixtures] added account group fixture: Group(display_name='sdk-oW5V', entitlements=[], external_id=None, groups=[], id='143570554656729', members=[], meta=None, roles=[], schemas=[]) 18:32 INFO [databricks.labs.ucx.workspace_access.groups] Migrating permissions for 1 account groups. 18:32 INFO [databricks.labs.ucx.workspace_access.groups] Migrating permissions: sdk-YY57 (workspace) -> sdk-oW5V (account) starting 18:32 ERROR [databricks.labs.ucx.workspace_access.groups] Migration of group permissions failed: sdk-YY57 (workspace) -> sdk-oW5V (account) Traceback (most recent call last): File "/home/runner/work/ucx/ucx/src/databricks/labs/ucx/workspace_access/groups.py", line 115, in _apply_to_groups group_permissions = self._migrate_group_permissions_paginated(ws, name_in_workspace, name_in_account) File "/home/runner/work/ucx/ucx/src/databricks/labs/ucx/workspace_access/groups.py", line 138, in _migrate_group_permissions_paginated result = ws.permission_migration.migrate_permissions( File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/service/iam.py", line 2533, in migrate_permissions res = self._api.do('POST', '/api/2.0/permissionmigration', body=body, headers=headers) File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/core.py", line 153, in do response = retryable(self._perform)(method, File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/retries.py", line 54, in wrapper raise err File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/retries.py", line 33, in wrapper return func(*args, **kwargs) File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/core.py", line 264, in _perform raise self._make_nicer_error(response=response, **payload) from None databricks.sdk.errors.platform.NotImplemented: Permission migration API is not enabled for workspace: 7342989205138882 18:32 INFO [databricks.labs.ucx.workspace_access.groups] Migrated 0 permissions for 0/1 groups successfully. 18:32 ERROR [databricks.labs.ucx.workspace_access.groups] Migrating permissions failed for 1/1 groups. 18:32 INFO [databricks.sdk] Using Databricks Metadata Service authentication 18:32 INFO [databricks.sdk] Using Databricks Metadata Service authentication 18:32 INFO [databricks.labs.ucx.mixins.fixtures] Workspace group sdk-YY57: https://DATABRICKS_HOST#setting/accounts/groups/702577790255713 18:32 DEBUG [databricks.labs.ucx.mixins.fixtures] added workspace group fixture: Group(display_name='sdk-YY57', entitlements=[], external_id=None, groups=[], id='702577790255713', members=[], meta=ResourceMeta(resource_type='WorkspaceGroup'), roles=[], schemas=[]) 18:32 INFO [databricks.labs.ucx.mixins.fixtures] Account group sdk-oW5V: https://accounts.CLOUD_ENVdatabricks.net/users/groups/143570554656729/members 18:32 DEBUG [databricks.labs.ucx.mixins.fixtures] added account group fixture: Group(display_name='sdk-oW5V', entitlements=[], external_id=None, groups=[], id='143570554656729', members=[], meta=None, roles=[], schemas=[]) 18:32 INFO [databricks.labs.ucx.workspace_access.groups] Migrating permissions for 1 account groups. 18:32 INFO [databricks.labs.ucx.workspace_access.groups] Migrating permissions: sdk-YY57 (workspace) -> sdk-oW5V (account) starting 18:32 ERROR [databricks.labs.ucx.workspace_access.groups] Migration of group permissions failed: sdk-YY57 (workspace) -> sdk-oW5V (account) Traceback (most recent call last): File "/home/runner/work/ucx/ucx/src/databricks/labs/ucx/workspace_access/groups.py", line 115, in _apply_to_groups group_permissions = self._migrate_group_permissions_paginated(ws, name_in_workspace, name_in_account) File "/home/runner/work/ucx/ucx/src/databricks/labs/ucx/workspace_access/groups.py", line 138, in _migrate_group_permissions_paginated result = ws.permission_migration.migrate_permissions( File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/service/iam.py", line 2533, in migrate_permissions res = self._api.do('POST', '/api/2.0/permissionmigration', body=body, headers=headers) File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/core.py", line 153, in do response = retryable(self._perform)(method, File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/retries.py", line 54, in wrapper raise err File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/retries.py", line 33, in wrapper return func(*args, **kwargs) File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/core.py", line 264, in _perform raise self._make_nicer_error(response=response, **payload) from None databricks.sdk.errors.platform.NotImplemented: Permission migration API is not enabled for workspace: 7342989205138882 18:32 INFO [databricks.labs.ucx.workspace_access.groups] Migrated 0 permissions for 0/1 groups successfully. 18:32 ERROR [databricks.labs.ucx.workspace_access.groups] Migrating permissions failed for 1/1 groups. 18:32 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 account group fixtures 18:32 DEBUG [databricks.labs.ucx.mixins.fixtures] removing account group fixture: Group(display_name='sdk-oW5V', entitlements=[], external_id=None, groups=[], id='143570554656729', members=[], meta=None, roles=[], schemas=[]) 18:32 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 workspace group fixtures 18:32 DEBUG [databricks.labs.ucx.mixins.fixtures] removing workspace group fixture: Group(display_name='sdk-YY57', entitlements=[], external_id=None, groups=[], id='702577790255713', members=[], meta=ResourceMeta(resource_type='WorkspaceGroup'), roles=[], schemas=[]) [gw1] linux -- Python 3.10.14 /home/runner/work/ucx/ucx/.venv/bin/python ```

Running from acceptance #4211