[ISSUE] Support assignment of users/groups/SPs to workspaces at account level

[bkvarda]

When Identity Federation rolled out, it disabled user/group creation at the workspace level and forces the use of account-level groups/users. In order to create workspace permissions for these users/groups, they need to first be assigned to the workspace. There appears to not currently be a way to do this assignment in Terraform, which basically breaks existing code for onboarding of new groups to workspaces.

In the UI when you assign users to a workspace it sends a PUT to https://accounts.cloud.databricks.com/api/2.0/preview/accounts/4c302ea8-1234-1234-1234-D1234567/roleassignments/principals/1234567891011

Provider version

0.6.0

Reproduction

After Identity Federation is enabled, creating a group at the workspace-level no longer works.

resource "databricks_group" "d_workspace" {
  provider = databricks.created_workspace
  display_name               = "Group D"
}

Error: cannot create group: group Account group with name Group D not found. Using basic auth: host=https://foo1234719374198374.cloud.databricks.com, username=foo@foo.com, password=***REDACTED***
│ 
│   with databricks_group.d_workspace,
│   on main.tf line 1141, in resource "databricks_group" "d_workspace":
│ 1141: resource "databricks_group" "d_workspace" {
│ 

If you create the group at the account-level instead, that works. But you can not reference that group in workspace permissions:

resource "databricks_group" "d_workspace" {
  provider = databricks.mws
  display_name               = "Group D"
}

resource "databricks_group_instance_profile" "group_d_instance_profile" {
  provider = databricks.created_workspace
  group_id            = databricks_group.d_workspace.id
  instance_profile_id = databricks_instance_profile.groupb-profile.id
}

The plan:

# databricks_group.d_workspace will be created
  + resource "databricks_group" "d_workspace" {
      + allow_cluster_create       = false
      + allow_instance_pool_create = false
      + databricks_sql_access      = false
      + display_name               = "Group D"
      + id                         = (known after apply)
      + url                        = (known after apply)
      + workspace_access           = false
    }
# databricks_group_instance_profile.group_d_instance_profile will be created
  + resource "databricks_group_instance_profile" "group_d_instance_profile" {
      + group_id            = (known after apply)
      + id                  = (known after apply)
      + instance_profile_id = "arn:aws:iam::123456678391:instance-profile/group_b_instance_profile"
    }

Then fails:

databricks_group.d_workspace: Creation complete after 2s [id=1105040545683737]
databricks_group_instance_profile.group_d_instance_profile: Creating.
│ Error: cannot create group instance profile: Group with id 1105040545683737 not found.
│ 
│   with databricks_group_instance_profile.group_d_instance_profile,
│   on main.tf line 1182, in resource "databricks_group_instance_profile" "group_d_instance_profile":
│ 1182: resource "databricks_group_instance_profile" "group_d_instance_profile" {
│ 
╵

Workaround

It seems like as a workaround you can do something like this:

# Create at account level
resource "databricks_group" "d_workspace" {
  provider = databricks.mws
  display_name               = "Group D"
}

# Create at workspace level
resource "databricks_group" "d_workspace1" {
  provider = databricks.created_workspace
  display_name               = "Group D"
}

# Then do workspace-level assignments on the group
resource "databricks_group_instance_profile" "group_d_instance_profile" {
  provider = databricks.created_workspace
  group_id            = databricks_group.d_workspace.id
  instance_profile_id = databricks_instance_profile.groupb-profile.id
}

This also appears to assign them to the workspace in the accounts UI, but not sure if there are any implications of doing it this way nor whether this is a long-term solution.

[baxen]

+1 - we're waiting to enable identity federation to have a plan so we can continue to support adding new groups (without initial members) to the workspace and assign an instance profile.

The workaround is interesting but seems to imply that the account level groups and workspace level groups are associated by display_name - if that's true and the intended way to associate the two then we could get everything working with that example terraform

[paiyar]

:+1: Also looking for official guidance on the long term solution (@bkvarda thank you for sharing the workaround)

[nfx]

Support is coming very soon =)