[ISSUE] Issue with destroying `databricks_group` and `databricks_group_member` resource.

[cesarnascimentto]

## Configuration

## main.tf
provider "databricks" {
  host       = var.host
  account_id = var.account_id
}

resource "databricks_group" "unit_catalog_group" {
  for_each = { for index, group in var.unit_catalog_group : group.group_name => group }

  display_name = each.key

}

data "databricks_user" "unit_catalog_group_user" {
  for_each = { for user in toset(local.uniq_list_users) : user => user }

  user_name = each.value

  depends_on = [
    databricks_group.unit_catalog_group
  ]
}

data "databricks_group" "unit_catalog_groups" {
  for_each = { for group in toset(local.uniq_list_groups) : group => group }

  display_name = each.value

  depends_on = [
    databricks_group.unit_catalog_group
  ]
}

data "databricks_service_principal" "spn" {
  for_each = { for spn in toset(local.uniq_list_spn) : spn => spn }

  application_id = each.value

  depends_on = [
    databricks_group.unit_catalog_group
  ]
}
resource "databricks_group_member" "member_group" {
  for_each = local.all_members_merged

  group_id  = databricks_group.unit_catalog_group[each.value.group].id
  member_id = each.value.user

  depends_on = [
    databricks_group.unit_catalog_group,
    data.databricks_user.unit_catalog_group_user,
    data.databricks_group.unit_catalog_groups,
    data.databricks_service_principal.spn
  ]
}

## local.tf

locals {
  unit_catalog_users_members  = { for i, catalog in var.unit_catalog_group : i.group_name => catalog.user }
  unit_catalog_groups_members = { for i, catalog in var.unit_catalog_group : i.group_name => catalog.group }
  unit_catalog_spn_members    = { for i, catalog in var.unit_catalog_group : i.group_name => catalog.spn }

  uniq_list_users  = flatten([for i, lista in local.unit_catalog_users_members : flatten(lista)])
  uniq_list_groups = flatten([for i, lista in local.unit_catalog_groups_members : flatten(lista)])
  uniq_list_spn    = flatten([for i, lista in local.unit_catalog_spn_members : flatten(lista)])

  unit_catalog_users_object = flatten([
    for group in var.unit_catalog_group : {
      for user in group.user : replace("${group.group_name}-${user}", "/\\W|_|\\s/", "-") => {
        group = group.group_name
        user  = data.databricks_user.unit_catalog_group_user[user].id
      }
    }
    ]
  )

  unit_catalog_groups_object = flatten([
    for group in var.unit_catalog_group : {
      for member in group.group : replace("${group.group_name}-${member}", "/\\W|_|\\s/", "-") => {
        group = group.group_name
        user  = data.databricks_group.unit_catalog_groups[member].id
      }
    }
    ]
  )

  unit_catalog_spn_object = flatten([
    for group in var.unit_catalog_group : {
      for member in group.spn : replace("${group.group_name}-${member}", "/\\W|_|\\s/", "-") => {
        group = group.group_name
        user  = data.databricks_service_principal.spn[member].id
      }
    }
    ]
  )

  catalog_users_members_list  = merge(local.unit_catalog_users_object...)
  catalog_groups_members_list = merge(local.unit_catalog_groups_object...)
  catalog_spns_members_list   = merge(local.unit_catalog_spn_object...)
  all_members_merged          = merge(local.catalog_users_members_list, local.catalog_groups_members_list, local.catalog_spns_members_list)

}

Populating the variables in terraform.auto.tfvars

## terraform.auto.tfvars
unit_catalog_group = [
  {
    group_name = "TEAM_BUILD_CI_TEST_1"
    user       = ["user1@xx-xxxxx.com"]
    #spn        = []
  },
  {
    group_name = "TEAM_BUILD_CI_1"
    user       = ["user2@xx-xxxxx.com"]
    group      = ["TEAM_BUILD_CI_TEST_1"]
    spn        = []
  }
]

### Expected Behavior All resources are being destroyed.

### Actual Behavior We were previously using this approach with version 1.14.0 of the Databricks provider. However, we started receiving errors with the plan and apply commands. We updated the provider to the current version (1.37.1), and the apply and plan commands were restored. However, when running the terraform destroy command, we encountered other errors.

## Log Errors Error: cannot delete group member: Unrecognized field "value" (class com.unboundid.scim2.common.messages.PatchOperation$RemoveOperation), not marked as ignorable (one known property: "path"]) │ at [Source: (String)"{"schemas":["urn:ietf:params:scim:api:messages:2.0:PatchOp"],"Operations":[{"op":"remove","path":"members[value eq \"0X0X0X10XX000X00\"]","value":""}]}"; line: 1, column: 148] (through reference chain: com.unboundid.scim2.common.messages.PatchRequest["Operations"]->java.util.ArrayList[0]->com.unboundid.scim2.common.messages.PatchOperation$RemoveOperation["value"])

PS: The IDs have been masked for client security.

### Steps to Reproduce terraform destroy

### Terraform and provider versions azurerm = 3.93.0 databricks = 1.37.1 terraform = 1.3.0

### Is it a regression? Resource deletion

### Debug Output < "schemas": [ < "urn:ietf:params:scim:api:messages:2.0:Error" < ], < "status": "400" < }: tf_rpc=ApplyResourceChange tf_provider_addr=registry.terraform.io/databricks/databricks tf_req_id=000000-00000000-00000000-000 @caller=/home/runner/work/terraform-provider-databricks/terraform-provider-databricks/logger/logger.go:33 @module=databricks tf_resource_type=databricks_group_member timestamp=2024-02-23T17:57:41.405-0300 2024-02-23T17:57:41.407-0300 [ERROR] provider.terraform-provider-databricks_v1.37.1: Response contains error diagnostic: @module=sdk.proto diagnostic_detail="" tf_provider_addr=registry.terraform.io/databricks/databricks @caller=/home/runner/work/terraform-provider-databricks/terraform-provider-databricks/vendor/github.com/hashicorp/terraform-plugin-go/tfprotov5/internal/diag/diagnostics.go:62 tf_req_id=000001000-00000000-0000 diagnostic_summary= | cannot delete group member: Unrecognized field "value" (class com.unboundid.scim2.common.messages.PatchOperation$RemoveOperation), not marked as ignorable (one known property: "path"]) | at [Source: (String)"{"schemas":["urn:ietf:params:scim:api:messages:2.0:PatchOp"],"Operations":[{"op":"remove","path":"members[value eq \"123456789101112\"]","value":""}]}"; line: 1, column: 148] (through reference chain: com.unboundid.scim2.common.messages.PatchRequest["Operations"]->java.util.ArrayList[0]->com.unboundid.scim2.common.messages.PatchOperation$RemoveOperation["value"]) tf_resource_type=databricks_group_member tf_rpc=ApplyResourceChange diagnostic_severity=ERROR tf_proto_version=5.4 timestamp=2024-02-23T17:57:41.405-0300 2024-02-23T17:57:41.407-0300 [DEBUG] provider.terraform-provider-databricks_v1.37.1: non-retriable error: Unrecognized field "value" (class com.unboundid.scim2.common.messages.PatchOperation$RemoveOperation), not marked as ignorable (one known property: "path"]) at [Source: (String)"{"schemas":["urn:ietf:params:scim:api:messages:2.0:PatchOp"],"Operations":[{"op":"remove","path":"members[value eq \"3702515137500277\"]","value":""}]}"; line: 1, column: 148] (through reference chain: com.unboundid.scim2.common.messages.PatchRequest["Operations"]->java.util.ArrayList[0]->com.unboundid.scim2.common.messages.PatchOperation$RemoveOperation["value"]): tf_req_id=X0000000-X0X0-0xx0-xX0x-0x00X0xxx0x0 tf_resource_type=databricks_group_member @caller=/home/runner/work/terraform-provider-databricks/terraform-provider-databricks/logger/logger.go:33 tf_provider_addr=registry.terraform.io/databricks/databricks tf_rpc=ApplyResourceChange @module=databricks timestamp=2024-02-23T17:57:41.405-0300 2024-02-23T17:57:41.407-0300 [DEBUG] provider.terraform-provider-databricks_v1.37.1: PATCH /api/2.0/accounts/1x11xx1x-01x1-0xx0-xx00-0x0000x0x00x/scim/v2/Groups/1211109876543210

{ "Operations": [ { "op": "remove", "path": "members[value eq \"123456789101112\"]", "value": "" } ], "schemas": [ "urn:ietf:params:scim:api:messages:2.0:PatchOp" ] } < HTTP/2.0 400 Bad Request < { < "detail": "Unrecognized field \"value\" (class com.unboundid.scim2.common.messages.PatchOperation$RemoveOpera... (449 more bytes)", < "schemas": [ < "urn:ietf:params:scim:api:messages:2.0:Error" < ], < "status": "400" < }: tf_resource_type=databricks_group_member @module=databricks tf_req_id=1111111-x0x0-1xX0-xx0x-0x00x8xxX0x0 tf_rpc=ApplyResourceChange @caller=/home/runner/work/terraform-provider-databricks/terraform-provider-databricks/logger/logger.go:33 tf_provider_addr=registry.terraform.io/databricks/databricks timestamp=2024-02-23T17:57:41.405-0300 2024-02-23T17:57:41.407-0300 [ERROR] provider.terraform-provider-databricks_v1.37.1: Response contains error diagnostic: diagnostic_severity=ERROR tf_provider_addr=registry.terraform.io/databricks/databricks @caller=/home/runner/work/terraform-provider-databricks/terraform-provider-databricks/vendor/github.com/hashicorp/terraform-plugin-go/tfprotov5/internal/diag/diagnostics.go:62 @module=sdk.proto tf_proto_version=5.4 tf_req_id=d4965107-e8c3-7ef5-ef6d-9f39c8cdd2e7 tf_rpc=ApplyResourceChange diagnostic_summary= | cannot delete group member: Unrecognized field "value" (class com.unboundid.scim2.common.messages.PatchOperation$RemoveOperation), not marked as ignorable (one known property: "path"]) | at [Source: (String)"{"schemas":["urn:ietf:params:scim:api:messages:2.0:PatchOp"],"Operations":[{"op":"remove","path":"members[value eq \"123456789101112\"]","value":""}]}"; line: 1, column: 148] (through reference chain: com.unboundid.scim2.common.messages.PatchRequest["Operations"]-

### Important Factoids Before updating the Databricks provider, the 'terraform plan', 'apply', and 'destroy' commands were not working. After updating to the current provider version, 'terraform destroy' presents the mentioned issue.

[alexott]

Most probably you have problem with the TF code itself. You didn't post the full code, for example, I don't see local.all_members_merged definition...

[cesarnascimentto]

Hello @alexott ! I included the local.tf file as you mentioned. But, could it be this? Considering that apply and plan work? It seems to me an error from the Databricks API in constructing the request for destroy.

[cesarnascimentto]

I performed a test by removing all lists and creating the group, associating the user directly. The plan and apply ran smoothly. However, the same error occurred during the destroy operation:

Error: cannot delete group member: Unrecognized field "value" (class com.unboundid.scim2.common.messages.PatchOperation$RemoveOperation), not marked as ignorable (one known property: "path"]) │ at [Source: (String)"{"schemas":["urn:ietf:params:scim:api:messages:2.0:PatchOp"],"Operations":[{"op":"remove","path":"members[value eq \"4804081037358757\"]","value":""}]}"; line: 1, column: 148] (through reference chain: com.unboundid.scim2.common.messages.PatchRequest["Operations"]->java.util.ArrayList[0]->com.unboundid.scim2.common.messages.PatchOperation$RemoveOperation["value"])

provider "databricks" {
  host       = var.host
  account_id = var.account_id
}

resource "databricks_group" "unit_catalog_group" {

  display_name = "Group_test_1" 

}

data "databricks_user" "unit_catalog_group_user" {

  user_name = "user1-ext@xx-xxxxx.com" 

  depends_on = [
    databricks_group.unit_catalog_group
  ]
}

resource "databricks_group_member" "member_group" {

  group_id  = databricks_group.unit_catalog_group.id          
  member_id = data.databricks_user.unit_catalog_group_user.id 

  depends_on = [
    databricks_group.unit_catalog_group,
    data.databricks_user.unit_catalog_group_user,
  ]
}

[jorgenj-cit]

Hello Databricks team, any updates regarding this situation?

[sjrusso8]

Hello! Any update on this error?