[BUG]: External Location permissions are overwritten causing failure

[P164-slalom]

Is there an existing issue for this?

• [X] I have searched the existing issues

Current Behavior

When we execute the table migration workflow, the tables are created as per the mapping csv. However, when the tables are created in the corresponding external location, the grants are not applied causing the table permissions to overwrite the external location permissions. This is causing any jobs from other workspaces and service principals that previously had access to the external location are failing.

Expected Behavior

The expectation is apply the grants immediately to the tables as they are created. Also, making sure the external location permissions are applied to the table grants so the failures do not occur.

Steps To Reproduce

No response

Cloud

Azure

Operating System

Windows

Version

latest via Databricks CLI

Relevant log output

Py4JJavaError: An error occurred while calling o371.sql. : com.databricks.sql.managedcatalog.acl.UnauthorizedAccessException: PERMISSION_DENIED: Catalog 'prod_master_data' is not accessible in current workspace at com.databricks.managedcatalog.TypeConversionUtils$.toUnityCatalogDeniedException(TypeConversionUtils.scala:2246) at com.databricks.managedcatalog.TypeConversionUtils$.toCatalyst(TypeConversionUtils.scala:2210) at com.databricks.managedcatalog.ManagedCatalogClientImpl.$anonfun$checkPathAccess$1(ManagedCatalogClientImpl.scala:4150) at com.databricks.managedcatalog.ManagedCatalogClientImpl.$anonfun$recordAndWrapException$2(ManagedCatalogClientImpl.scala:4629)

[HariGS-DB]

@P164-slalom Thanks for raising this issue. Could you please give more details on the issue.

1. Table migration is creating the tables correctly ?
2. Table permissions are applied correctly ? if grants are not applied what is the error you are getting. is it possible to share the log file of the job

[dipankarkush-db]

@HariGS-DB - Table migration is working correctly as well as the grants. There are some scenarios where same abfss (ADLS) location is access from more than one workspace. When workspace 1 is migrating to UC tables and grants then workspace 2 is losing access to the abfss location as there is a UC table on that location and principals from workspace 2 has not been granted permission on those tables yet.

[HariGS-DB]

Conclusion: This is not technically a bug in UCX. The issue is occurring due to the fact that when you create table on a location, the permission defined on the external location are ignored. If a user has access to the external location and was able to access before, but once a table is created and if the user is not given access to select/modify the table, then the user can read from the external location, This is standard UC behaviour. Customers need to make sure while migrating tables and acl to UC, there shouldn't be any existing principals having access to the underlying location