[FEATURE] Add Account SCIM For Workspaces support

[nkvuong]

Even though workspace admins cannot create or manage account groups, they can retrieve details about account-level groups, users & service principals via the Account SCIM API by calling {workspace-domain}/api/2.0/account/scim/v2/

This is most relevant to managing groups use case, as groups created via Workspace SCIM (Groups) REST API will be workspace-local groups, and cannot leverage Unity Catalog.

Suggestion is to create a databricks_account_group data source, similar to databricks_group

Usage would be like below

data "databricks_account_group" "dataeng_admins" {
  display_name = "dataeng-admins"
}

resource "databricks_permission_assignment" "add_group" {
  principal_id = databricks_account_group.dataeng_admins.id
  permissions  = ["USER"]
}

[guderkar]

This feature would be very helpful but I see the PR is closed. Do you have any workaround suggestion for this?

Using http request would do the trick

But the authorization section is ugly. Any idea how to get some authorization context from databricks.workspace provider which can be passed to this http resource?

[mgyucht]

Is this a Terraform problem or an issue with Databricks account-level SCIM APIs? Outside of Terraform, is it possible for WS admins to create groups that can use UC?

[onefifth]

I really hope this part of the API is actively being worked on. I've spent far too much time messing around with Databricks groups over the last couple weeks.

The documentation explicitly references an Account Groups API endpoint that should be used in the Workspace domain by Workspace Admins to manage membership in Account groups. This endpoint is completely undocumented in the Workspace API reference, and based on this comment, that is (for some reason) intentional?

These (mostly) undocumented account-api-in-workspace-domain endpoints exist for many of the Workspace API endpoints that are currently listed as "public preview" in the documentation, and can be used by swapping preview/scim to account/scim in the URL. From the outside it almost looks like a mistake? It seems like the documented "Public Preview" endpoints should actually be referencing these undocumented "account" endpoints which, as far as I can tell, do exactly what you'd expect the set of public preview Workspace Group/User/ServicePrincipal endpoints to do...

Either a databricks_account_group resource should exist (along with matching ones for the other account-api-in-workspace-domain endpoints) or logic should be introduced to check if the authenticated workspace provider is a workspace admin (this already kinda exists) and alter the endpoint URL as required (this also weirdly half-exists already??).

I realize this is mostly a grievance with the REST API, but without these resources/endpoints, things like the Group Manager role make little to no sense outside of the web UIs and actively mislead users into thinking it's possible for non-account admins to be able to manage groups. I've upgraded several Workspace groups to Account groups only to discover I need to revert my changes or dole out account admin privileges to automation SPs that previously only needed to be workspace admins. Frankly, encouraging users to upgrade to Account Groups and calling Workspace Groups "legacy" comes across as an extremely premature decision given the current state of the docs and APIs.

[AleCeli]

+1