[FEATURE] Expose argument `workspace_id` in resource `databricks_permission_assignment`

[camilo-s]

Use-cases

Here's a summary of my use-case (also described in #2239):

• I'm managing an Azure Databricks deployment with Terraform providers azurerm, azuread, databricks.
• I've configured SCIM provisioning using Azure AD according to the documentation.
• I've assigned groups to the Databricks' Enterprise Application, which I can successfully see in the Databricks Account console after sync.
• I'd like to provision these groups at the workspace level, to be able to assign them workspace-level permissions, grants and entitlements.

Attempted Solutions

My current solution proceeds as follows:

• Read the account-level groups with the databricks_group data source using the account-level provider (which is authenticated using Azure MSI): This works.
• Assign workspace permissions to the group with the databricks_permission_assignment resource using the workspace-level provider: The resource's documentation indicates this is the right way to enable groups at workspace level, but it doesn't work because it can't find the groups by ID. This is kind of understandable: This resource will talk to the Permission Assignment Workspace API /api/2.0/preview/permissionassignments/principals/{principal_id}, which has no way to know which groups exist at account-level but not at workspace-level.

data "databricks_group" "users" {
  for_each = var.azure_ad_groups

  display_name = each.key

  provider = databricks.account
}

resource "databricks_permission_assignment" "users" {
  for_each = var.azure_ad_groups

  principal_id = data.databricks_group.users[each.key].id
  permissions  = ["USERS"]
}

Proposal

My proposals are:

1. To enable this resource for use with an account provider, so it can talk to the Permission Assignment Account API, and correspondingly expose the workspace_id parameter of this API.

   resource "databricks_permission_assignment" "users" {
     for_each = var.azure_ad_groups
   
     principal_id = data.databricks_group.users[each.key].id
     permissions  = ["USERS"]
     workspace_id = var.databricks_workspace_id
   
     provider = databricks.account
   }

2. To create an AzAPI-provider of sorts for Databricks, as a thin layer on top of the Databricks REST APIs to be able to interact with evolving APIs that are not yet properly incorporated in the Databricks provider.

References

• I've filed closely related issue #2239, with a full account of the error message.

[camilo-s]

Ideally this should follow the schema for databricks_mws_permission_assignment, which does have a workspace_id attribute.

[nkvuong]

@camilo-s it is a bit confusing, but databricks_mws_permission_assignment is meant to be used at account level, and databricks_permission_assignment is meant to be used at workspace level (and therefore does not require a workspace id)

[camilo-s]

@nkvuong I understand this. But then how might databricks_permission_assignment possibly be used to provision existing account-level groups at the workspace level at all? It acts at workspace level (i.e. calling the Permission Assignment Workspace API) which is agnostic to account-level groups, so it rightfully returns an error for not being able to find the account-level groups by ID, as I've indicated in #2239.

I made an attempt to leverage databricks_mws_permission_assignment to call the Permission Assignment Account API for my Azure Databricks account, but that API doesn't seem to work for Azure at the moment (documented in #2239).