search

databrickslabs / ucx

Automated migrations to Unity Catalog
Other
231 stars 79 forks source link

Test failure: `test_reflect_account_groups_on_workspace_skips_groups_that_already_exists_in_the_workspace` #2623

Closed github-actions[bot] closed 1 month ago

github-actions[bot] commented 1 month ago
❌ test_reflect_account_groups_on_workspace_skips_groups_that_already_exists_in_the_workspace: databricks.labs.blueprint.parallel.ManyError: Detected 1 failures: TimeoutError: Timed out after 0:20:00 (20m21.536s) ``` databricks.labs.blueprint.parallel.ManyError: Detected 1 failures: TimeoutError: Timed out after 0:20:00 05:32 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_sr3el: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_sr3el 05:32 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_sr3el', metastore_id=None, name='ucx_sr3el', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) [gw3] linux -- Python 3.10.14 /home/runner/work/ucx/ucx/.venv/bin/python 05:32 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_sr3el: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_sr3el 05:32 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_sr3el', metastore_id=None, name='ucx_sr3el', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 05:32 DEBUG [databricks.labs.ucx.mixins.fixtures] added workspace user fixture: User(active=True, display_name='sdk-kbjr-ra78a52eab@example.com', emails=[ComplexValue(display=None, primary=True, ref=None, type='work', value='sdk-kbjr-ra78a52eab@example.com')], entitlements=[], external_id=None, groups=[], id='4961563233899540', name=Name(family_name=None, given_name='sdk-kbjr-ra78a52eab@example.com'), roles=[], schemas=[, ], user_name='sdk-kbjr-ra78a52eab@example.com') 05:32 INFO [databricks.labs.ucx.mixins.fixtures] Workspace group ucx_Gct3R-ra78a52eab: https://DATABRICKS_HOST#setting/accounts/groups/35156646578503 05:32 DEBUG [databricks.labs.ucx.mixins.fixtures] added workspace group fixture: Group(display_name='ucx_Gct3R-ra78a52eab', entitlements=[ComplexValue(display=None, primary=None, ref=None, type=None, value='allow-cluster-create')], external_id=None, groups=[], id='35156646578503', members=[ComplexValue(display='sdk-kbjr-ra78a52eab@example.com', primary=None, ref='Users/4961563233899540', type=None, value='4961563233899540')], meta=ResourceMeta(resource_type='WorkspaceGroup'), roles=[], schemas=[]) 05:32 INFO [databricks.labs.ucx.mixins.fixtures] Account group ucx_Gct3R-ra78a52eab: https://accounts.CLOUD_ENVdatabricks.net/users/groups/398958227453548/members 05:32 DEBUG [databricks.labs.ucx.mixins.fixtures] added account group fixture: Group(display_name='ucx_Gct3R-ra78a52eab', entitlements=[], external_id=None, groups=[], id='398958227453548', members=[ComplexValue(display='sdk-kbjr-ra78a52eab@example.com', primary=None, ref='Users/4961563233899540', type=None, value='4961563233899540')], meta=None, roles=[], schemas=[]) 05:32 INFO [databricks.labs.ucx.workspace_access.groups] Listing account groups with id,displayName,externalId... 05:32 INFO [databricks.labs.ucx.workspace_access.groups] Found 69 account groups 05:32 INFO [databricks.labs.ucx.workspace_access.groups] Listing workspace groups (resource_type=Group) with id,displayName,externalId,meta ... 05:32 INFO [databricks.labs.ucx.workspace_access.groups] Found 57 Group 05:32 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.ucx_sr3el.groups] fetching groups inventory 05:32 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.ucx_sr3el.groups] crawling new set of snapshot data for groups 05:32 INFO [databricks.labs.ucx.workspace_access.groups] Listing workspace groups (resource_type=WorkspaceGroup) with id,displayName,meta,externalId,members,roles,entitlements ... 05:32 INFO [databricks.labs.ucx.workspace_access.groups] Found 4 WorkspaceGroup 05:32 INFO [databricks.labs.ucx.workspace_access.groups] Listing account groups with id,displayName,externalId... 05:32 INFO [databricks.labs.ucx.workspace_access.groups] Found 69 account groups 05:32 INFO [databricks.labs.ucx.workspace_access.groups] Group listing provided, a subset of all groups will be migrated 05:32 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.ucx_sr3el.groups] found 1 new records for groups 05:32 INFO [databricks.labs.ucx.workspace_access.groups] Starting to reflect 1 account groups into workspace for migration... 05:52 ERROR [databricks.labs.blueprint.parallel] reflect account groups on this workspace('398958227453548') task failed: Timed out after 0:20:00 Traceback (most recent call last): File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/retries.py", line 33, in wrapper return func(*args, **kwargs) File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/labs/blueprint/limiter.py", line 65, in wrapper return func(*args, **kwargs) File "/home/runner/work/ucx/ucx/src/databricks/labs/ucx/workspace_access/groups.py", line 883, in _reflect_account_group_to_workspace self._ws.api_client.do("PUT", path, data=json.dumps({"permissions": ["USER"]})) File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/core.py", line 155, in do response = retryable(self._perform)(method, File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/retries.py", line 54, in wrapper raise err File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/retries.py", line 33, in wrapper return func(*args, **kwargs) File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/core.py", line 273, in _perform raise error from None databricks.sdk.errors.platform.ResourceConflict: Workspace group with name ucx_Gct3R-ra78a52eab already exists. The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/labs/blueprint/parallel.py", line 158, in inner return func(*args, **kwargs), None File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/retries.py", line 59, in wrapper raise TimeoutError(f'Timed out after {timeout}') from last_err TimeoutError: Timed out after 0:20:00 05:52 CRITICAL [databricks.labs.blueprint.parallel] All 'reflect account groups on this workspace' tasks failed!!! 05:32 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_sr3el: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_sr3el 05:32 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_sr3el', metastore_id=None, name='ucx_sr3el', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 05:32 DEBUG [databricks.labs.ucx.mixins.fixtures] added workspace user fixture: User(active=True, display_name='sdk-kbjr-ra78a52eab@example.com', emails=[ComplexValue(display=None, primary=True, ref=None, type='work', value='sdk-kbjr-ra78a52eab@example.com')], entitlements=[], external_id=None, groups=[], id='4961563233899540', name=Name(family_name=None, given_name='sdk-kbjr-ra78a52eab@example.com'), roles=[], schemas=[, ], user_name='sdk-kbjr-ra78a52eab@example.com') 05:32 INFO [databricks.labs.ucx.mixins.fixtures] Workspace group ucx_Gct3R-ra78a52eab: https://DATABRICKS_HOST#setting/accounts/groups/35156646578503 05:32 DEBUG [databricks.labs.ucx.mixins.fixtures] added workspace group fixture: Group(display_name='ucx_Gct3R-ra78a52eab', entitlements=[ComplexValue(display=None, primary=None, ref=None, type=None, value='allow-cluster-create')], external_id=None, groups=[], id='35156646578503', members=[ComplexValue(display='sdk-kbjr-ra78a52eab@example.com', primary=None, ref='Users/4961563233899540', type=None, value='4961563233899540')], meta=ResourceMeta(resource_type='WorkspaceGroup'), roles=[], schemas=[]) 05:32 INFO [databricks.labs.ucx.mixins.fixtures] Account group ucx_Gct3R-ra78a52eab: https://accounts.CLOUD_ENVdatabricks.net/users/groups/398958227453548/members 05:32 DEBUG [databricks.labs.ucx.mixins.fixtures] added account group fixture: Group(display_name='ucx_Gct3R-ra78a52eab', entitlements=[], external_id=None, groups=[], id='398958227453548', members=[ComplexValue(display='sdk-kbjr-ra78a52eab@example.com', primary=None, ref='Users/4961563233899540', type=None, value='4961563233899540')], meta=None, roles=[], schemas=[]) 05:32 INFO [databricks.labs.ucx.workspace_access.groups] Listing account groups with id,displayName,externalId... 05:32 INFO [databricks.labs.ucx.workspace_access.groups] Found 69 account groups 05:32 INFO [databricks.labs.ucx.workspace_access.groups] Listing workspace groups (resource_type=Group) with id,displayName,externalId,meta ... 05:32 INFO [databricks.labs.ucx.workspace_access.groups] Found 57 Group 05:32 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.ucx_sr3el.groups] fetching groups inventory 05:32 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.ucx_sr3el.groups] crawling new set of snapshot data for groups 05:32 INFO [databricks.labs.ucx.workspace_access.groups] Listing workspace groups (resource_type=WorkspaceGroup) with id,displayName,meta,externalId,members,roles,entitlements ... 05:32 INFO [databricks.labs.ucx.workspace_access.groups] Found 4 WorkspaceGroup 05:32 INFO [databricks.labs.ucx.workspace_access.groups] Listing account groups with id,displayName,externalId... 05:32 INFO [databricks.labs.ucx.workspace_access.groups] Found 69 account groups 05:32 INFO [databricks.labs.ucx.workspace_access.groups] Group listing provided, a subset of all groups will be migrated 05:32 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.ucx_sr3el.groups] found 1 new records for groups 05:32 INFO [databricks.labs.ucx.workspace_access.groups] Starting to reflect 1 account groups into workspace for migration... 05:52 ERROR [databricks.labs.blueprint.parallel] reflect account groups on this workspace('398958227453548') task failed: Timed out after 0:20:00 Traceback (most recent call last): File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/retries.py", line 33, in wrapper return func(*args, **kwargs) File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/labs/blueprint/limiter.py", line 65, in wrapper return func(*args, **kwargs) File "/home/runner/work/ucx/ucx/src/databricks/labs/ucx/workspace_access/groups.py", line 883, in _reflect_account_group_to_workspace self._ws.api_client.do("PUT", path, data=json.dumps({"permissions": ["USER"]})) File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/core.py", line 155, in do response = retryable(self._perform)(method, File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/retries.py", line 54, in wrapper raise err File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/retries.py", line 33, in wrapper return func(*args, **kwargs) File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/core.py", line 273, in _perform raise error from None databricks.sdk.errors.platform.ResourceConflict: Workspace group with name ucx_Gct3R-ra78a52eab already exists. The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/labs/blueprint/parallel.py", line 158, in inner return func(*args, **kwargs), None File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/retries.py", line 59, in wrapper raise TimeoutError(f'Timed out after {timeout}') from last_err TimeoutError: Timed out after 0:20:00 05:52 CRITICAL [databricks.labs.blueprint.parallel] All 'reflect account groups on this workspace' tasks failed!!! 05:52 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 schema fixtures 05:52 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_sr3el', metastore_id=None, name='ucx_sr3el', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 05:52 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 workspace user fixtures 05:52 DEBUG [databricks.labs.ucx.mixins.fixtures] removing workspace user fixture: User(active=True, display_name='sdk-kbjr-ra78a52eab@example.com', emails=[ComplexValue(display=None, primary=True, ref=None, type='work', value='sdk-kbjr-ra78a52eab@example.com')], entitlements=[], external_id=None, groups=[], id='4961563233899540', name=Name(family_name=None, given_name='sdk-kbjr-ra78a52eab@example.com'), roles=[], schemas=[, ], user_name='sdk-kbjr-ra78a52eab@example.com') 05:52 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 account group fixtures 05:52 DEBUG [databricks.labs.ucx.mixins.fixtures] removing account group fixture: Group(display_name='ucx_Gct3R-ra78a52eab', entitlements=[], external_id=None, groups=[], id='398958227453548', members=[ComplexValue(display='sdk-kbjr-ra78a52eab@example.com', primary=None, ref='Users/4961563233899540', type=None, value='4961563233899540')], meta=None, roles=[], schemas=[]) 05:52 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 workspace group fixtures 05:52 DEBUG [databricks.labs.ucx.mixins.fixtures] removing workspace group fixture: Group(display_name='ucx_Gct3R-ra78a52eab', entitlements=[ComplexValue(display=None, primary=None, ref=None, type=None, value='allow-cluster-create')], external_id=None, groups=[], id='35156646578503', members=[ComplexValue(display='sdk-kbjr-ra78a52eab@example.com', primary=None, ref='Users/4961563233899540', type=None, value='4961563233899540')], meta=ResourceMeta(resource_type='WorkspaceGroup'), roles=[], schemas=[]) [gw3] linux -- Python 3.10.14 /home/runner/work/ucx/ucx/.venv/bin/python ```

Running from nightly #194

github-actions[bot] commented 1 month ago
❌ test_reflect_account_groups_on_workspace_skips_groups_that_already_exists_in_the_workspace: databricks.labs.blueprint.parallel.ManyError: Detected 1 failures: TimeoutError: Timed out after 0:20:00 (20m19.781s) ``` databricks.labs.blueprint.parallel.ManyError: Detected 1 failures: TimeoutError: Timed out after 0:20:00 12:08 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_s02by: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_s02by 12:08 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_s02by', metastore_id=None, name='ucx_s02by', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) [gw4] linux -- Python 3.10.14 /home/runner/work/ucx/ucx/.venv/bin/python 12:08 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_s02by: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_s02by 12:08 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_s02by', metastore_id=None, name='ucx_s02by', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 12:08 DEBUG [databricks.labs.ucx.mixins.fixtures] added workspace user fixture: User(active=True, display_name='sdk-yxfs-ra78a52eb2@example.com', emails=[ComplexValue(display=None, primary=True, ref=None, type='work', value='sdk-yxfs-ra78a52eb2@example.com')], entitlements=[], external_id=None, groups=[], id='163843573986372', name=Name(family_name=None, given_name='sdk-yxfs-ra78a52eb2@example.com'), roles=[], schemas=[, ], user_name='sdk-yxfs-ra78a52eb2@example.com') 12:08 INFO [databricks.labs.ucx.mixins.fixtures] Workspace group ucx_Gyh25-ra78a52eb2: https://DATABRICKS_HOST#setting/accounts/groups/367431752349390 12:08 DEBUG [databricks.labs.ucx.mixins.fixtures] added workspace group fixture: Group(display_name='ucx_Gyh25-ra78a52eb2', entitlements=[ComplexValue(display=None, primary=None, ref=None, type=None, value='allow-cluster-create')], external_id=None, groups=[], id='367431752349390', members=[ComplexValue(display='sdk-yxfs-ra78a52eb2@example.com', primary=None, ref='Users/163843573986372', type=None, value='163843573986372')], meta=ResourceMeta(resource_type='WorkspaceGroup'), roles=[], schemas=[]) 12:08 INFO [databricks.labs.ucx.mixins.fixtures] Account group ucx_Gyh25-ra78a52eb2: https://accounts.CLOUD_ENVdatabricks.net/users/groups/268481593188409/members 12:08 DEBUG [databricks.labs.ucx.mixins.fixtures] added account group fixture: Group(display_name='ucx_Gyh25-ra78a52eb2', entitlements=[], external_id=None, groups=[], id='268481593188409', members=[ComplexValue(display='sdk-yxfs-ra78a52eb2@example.com', primary=None, ref='Users/163843573986372', type=None, value='163843573986372')], meta=None, roles=[], schemas=[]) 12:08 INFO [databricks.labs.ucx.workspace_access.groups] Listing account groups with id,displayName,externalId... 12:08 INFO [databricks.labs.ucx.workspace_access.groups] Found 72 account groups 12:08 INFO [databricks.labs.ucx.workspace_access.groups] Listing workspace groups (resource_type=Group) with id,displayName,externalId,meta ... 12:08 INFO [databricks.labs.ucx.workspace_access.groups] Found 57 Group 12:08 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.ucx_s02by.groups] fetching groups inventory 12:08 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.ucx_s02by.groups] crawling new set of snapshot data for groups 12:08 INFO [databricks.labs.ucx.workspace_access.groups] Listing workspace groups (resource_type=WorkspaceGroup) with id,displayName,meta,externalId,members,roles,entitlements ... 12:08 INFO [databricks.labs.ucx.workspace_access.groups] Found 4 WorkspaceGroup 12:08 INFO [databricks.labs.ucx.workspace_access.groups] Listing account groups with id,displayName,externalId... 12:08 INFO [databricks.labs.ucx.workspace_access.groups] Found 73 account groups 12:08 INFO [databricks.labs.ucx.workspace_access.groups] Group listing provided, a subset of all groups will be migrated 12:08 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.ucx_s02by.groups] found 1 new records for groups 12:08 INFO [databricks.labs.ucx.workspace_access.groups] Starting to reflect 1 account groups into workspace for migration... 12:28 ERROR [databricks.labs.blueprint.parallel] reflect account groups on this workspace('268481593188409') task failed: Timed out after 0:20:00 Traceback (most recent call last): File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/retries.py", line 33, in wrapper return func(*args, **kwargs) File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/labs/blueprint/limiter.py", line 65, in wrapper return func(*args, **kwargs) File "/home/runner/work/ucx/ucx/src/databricks/labs/ucx/workspace_access/groups.py", line 883, in _reflect_account_group_to_workspace self._ws.api_client.do("PUT", path, data=json.dumps({"permissions": ["USER"]})) File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/core.py", line 155, in do response = retryable(self._perform)(method, File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/retries.py", line 54, in wrapper raise err File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/retries.py", line 33, in wrapper return func(*args, **kwargs) File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/core.py", line 273, in _perform raise error from None databricks.sdk.errors.platform.ResourceConflict: Workspace group with name ucx_Gyh25-ra78a52eb2 already exists. The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/labs/blueprint/parallel.py", line 158, in inner return func(*args, **kwargs), None File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/retries.py", line 59, in wrapper raise TimeoutError(f'Timed out after {timeout}') from last_err TimeoutError: Timed out after 0:20:00 12:28 CRITICAL [databricks.labs.blueprint.parallel] All 'reflect account groups on this workspace' tasks failed!!! 12:08 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_s02by: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_s02by 12:08 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_s02by', metastore_id=None, name='ucx_s02by', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 12:08 DEBUG [databricks.labs.ucx.mixins.fixtures] added workspace user fixture: User(active=True, display_name='sdk-yxfs-ra78a52eb2@example.com', emails=[ComplexValue(display=None, primary=True, ref=None, type='work', value='sdk-yxfs-ra78a52eb2@example.com')], entitlements=[], external_id=None, groups=[], id='163843573986372', name=Name(family_name=None, given_name='sdk-yxfs-ra78a52eb2@example.com'), roles=[], schemas=[, ], user_name='sdk-yxfs-ra78a52eb2@example.com') 12:08 INFO [databricks.labs.ucx.mixins.fixtures] Workspace group ucx_Gyh25-ra78a52eb2: https://DATABRICKS_HOST#setting/accounts/groups/367431752349390 12:08 DEBUG [databricks.labs.ucx.mixins.fixtures] added workspace group fixture: Group(display_name='ucx_Gyh25-ra78a52eb2', entitlements=[ComplexValue(display=None, primary=None, ref=None, type=None, value='allow-cluster-create')], external_id=None, groups=[], id='367431752349390', members=[ComplexValue(display='sdk-yxfs-ra78a52eb2@example.com', primary=None, ref='Users/163843573986372', type=None, value='163843573986372')], meta=ResourceMeta(resource_type='WorkspaceGroup'), roles=[], schemas=[]) 12:08 INFO [databricks.labs.ucx.mixins.fixtures] Account group ucx_Gyh25-ra78a52eb2: https://accounts.CLOUD_ENVdatabricks.net/users/groups/268481593188409/members 12:08 DEBUG [databricks.labs.ucx.mixins.fixtures] added account group fixture: Group(display_name='ucx_Gyh25-ra78a52eb2', entitlements=[], external_id=None, groups=[], id='268481593188409', members=[ComplexValue(display='sdk-yxfs-ra78a52eb2@example.com', primary=None, ref='Users/163843573986372', type=None, value='163843573986372')], meta=None, roles=[], schemas=[]) 12:08 INFO [databricks.labs.ucx.workspace_access.groups] Listing account groups with id,displayName,externalId... 12:08 INFO [databricks.labs.ucx.workspace_access.groups] Found 72 account groups 12:08 INFO [databricks.labs.ucx.workspace_access.groups] Listing workspace groups (resource_type=Group) with id,displayName,externalId,meta ... 12:08 INFO [databricks.labs.ucx.workspace_access.groups] Found 57 Group 12:08 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.ucx_s02by.groups] fetching groups inventory 12:08 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.ucx_s02by.groups] crawling new set of snapshot data for groups 12:08 INFO [databricks.labs.ucx.workspace_access.groups] Listing workspace groups (resource_type=WorkspaceGroup) with id,displayName,meta,externalId,members,roles,entitlements ... 12:08 INFO [databricks.labs.ucx.workspace_access.groups] Found 4 WorkspaceGroup 12:08 INFO [databricks.labs.ucx.workspace_access.groups] Listing account groups with id,displayName,externalId... 12:08 INFO [databricks.labs.ucx.workspace_access.groups] Found 73 account groups 12:08 INFO [databricks.labs.ucx.workspace_access.groups] Group listing provided, a subset of all groups will be migrated 12:08 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.ucx_s02by.groups] found 1 new records for groups 12:08 INFO [databricks.labs.ucx.workspace_access.groups] Starting to reflect 1 account groups into workspace for migration... 12:28 ERROR [databricks.labs.blueprint.parallel] reflect account groups on this workspace('268481593188409') task failed: Timed out after 0:20:00 Traceback (most recent call last): File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/retries.py", line 33, in wrapper return func(*args, **kwargs) File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/labs/blueprint/limiter.py", line 65, in wrapper return func(*args, **kwargs) File "/home/runner/work/ucx/ucx/src/databricks/labs/ucx/workspace_access/groups.py", line 883, in _reflect_account_group_to_workspace self._ws.api_client.do("PUT", path, data=json.dumps({"permissions": ["USER"]})) File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/core.py", line 155, in do response = retryable(self._perform)(method, File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/retries.py", line 54, in wrapper raise err File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/retries.py", line 33, in wrapper return func(*args, **kwargs) File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/core.py", line 273, in _perform raise error from None databricks.sdk.errors.platform.ResourceConflict: Workspace group with name ucx_Gyh25-ra78a52eb2 already exists. The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/labs/blueprint/parallel.py", line 158, in inner return func(*args, **kwargs), None File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/retries.py", line 59, in wrapper raise TimeoutError(f'Timed out after {timeout}') from last_err TimeoutError: Timed out after 0:20:00 12:28 CRITICAL [databricks.labs.blueprint.parallel] All 'reflect account groups on this workspace' tasks failed!!! 12:28 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 schema fixtures 12:28 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_s02by', metastore_id=None, name='ucx_s02by', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None) 12:28 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 workspace user fixtures 12:28 DEBUG [databricks.labs.ucx.mixins.fixtures] removing workspace user fixture: User(active=True, display_name='sdk-yxfs-ra78a52eb2@example.com', emails=[ComplexValue(display=None, primary=True, ref=None, type='work', value='sdk-yxfs-ra78a52eb2@example.com')], entitlements=[], external_id=None, groups=[], id='163843573986372', name=Name(family_name=None, given_name='sdk-yxfs-ra78a52eb2@example.com'), roles=[], schemas=[, ], user_name='sdk-yxfs-ra78a52eb2@example.com') 12:28 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 account group fixtures 12:28 DEBUG [databricks.labs.ucx.mixins.fixtures] removing account group fixture: Group(display_name='ucx_Gyh25-ra78a52eb2', entitlements=[], external_id=None, groups=[], id='268481593188409', members=[ComplexValue(display='sdk-yxfs-ra78a52eb2@example.com', primary=None, ref='Users/163843573986372', type=None, value='163843573986372')], meta=None, roles=[], schemas=[]) 12:28 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 workspace group fixtures 12:28 DEBUG [databricks.labs.ucx.mixins.fixtures] removing workspace group fixture: Group(display_name='ucx_Gyh25-ra78a52eb2', entitlements=[ComplexValue(display=None, primary=None, ref=None, type=None, value='allow-cluster-create')], external_id=None, groups=[], id='367431752349390', members=[ComplexValue(display='sdk-yxfs-ra78a52eb2@example.com', primary=None, ref='Users/163843573986372', type=None, value='163843573986372')], meta=ResourceMeta(resource_type='WorkspaceGroup'), roles=[], schemas=[]) [gw4] linux -- Python 3.10.14 /home/runner/work/ucx/ucx/.venv/bin/python ```

Running from nightly #194