[ISSUE] Issue with `databricks_cluster_policy` resource. Two different type of libraries doesn't get installed in one go

[pcsrijith]

Configuration

locals {
  default_policy = {
    "dbus_per_hour" : {
      "type" : "range",
      "maxValue" : 10
    },
    "autotermination_minutes" : {
      "type" : "fixed",
      "value" : 20,
      "hidden" : true
    },
    "custom_tags.Team" : {
      "type" : "fixed",
      "value" : var.team
    }
  }
}

resource "databricks_cluster_policy" "fair_use" {
  name       = "${var.team} cluster policy"
  definition = jsonencode(merge(local.default_policy, var.policy_overrides))

  libraries {
    pypi {
      package = "databricks-sdk==0.12.0"
      // repo can also be specified here
    }
    maven {
      package = "com.oracle.database.jdbc:ojdbc8:XXXX"
      // repo can also be specified here
    }
  }
}

Expected Behavior

Databricks cluster policy should have two libraries installed. One pypi and another maven type

Actual Behavior

Even though terraform plan shows that it will add both libraries, it doesn't add them

Steps to Reproduce

Terraform and provider versions

terraform version: v1.8.4 Databricks provider version: 1.44.0

Is it a regression?

No

Debug Output

Important Factoids

No

Would you like to implement a fix?

No

Extra Notes

I'm able to create policies having different types of libraries (maven, pypi) via Databricks UI.

[nkvuong]

this looks like an underlying issue with the API, Terraform provider is sending both libraries in the definition

locals {
  default_policy = {
    "dbus_per_hour" : {
      "type" : "range",
      "maxValue" : 10
    },
    "autotermination_minutes" : {
      "type" : "fixed",
      "value" : 20,
      "hidden" : true
    },
    "custom_tags.Team" : {
      "type" : "fixed",
      "value" : "test"
    }
  }
}

resource "databricks_cluster_policy" "fair_use" {
  name       = "test cluster policy"
  definition = jsonencode(local.default_policy)

  libraries {
    pypi {
      package = "databricks-sdk==0.12.0"
    }
    maven {
      coordinates = "com.oracle.database.jdbc:ojdbc8:XXXX"
    }
  }
}

the debug log

2024-06-25T14:27:32.659+0100 [DEBUG] provider.terraform-provider-databricks_v1.48.0: POST /api/2.0/policies/clusters/edit
> {
>   "definition": "{\"autotermination_minutes\":{\"hidden\":true,\"type\":\"fixed\",\"value\":20},\"custom_tags.Team\":{\"type\":... (71 more bytes)",
>   "libraries": [
>     {
>       "maven": {
>         "coordinates": "com.oracle.database.jdbc:ojdbc8:XXXX"
>       },
>       "pypi": {
>         "package": "databricks-sdk==0.12.0"
>       }
>     }
>   ],
>   "name": "test cluster policy",
>   "policy_id": "001C043340CD70D0"
> }
< HTTP/2.0 200 OK
< {}: @module=databricks tf_provider_addr=registry.terraform.io/databricks/databricks @caller=/home/runner/work/terraform-provider-databricks/terraform-provider-databricks/logger/logger.go:33 tf_rpc=ApplyResourceChange tf_req_id=74d2d271-e26e-1592-452a-98b7cc2575db tf_resource_type=databricks_cluster_policy timestamp="2024-06-25T14:27:32.658+0100"
2024-06-25T14:27:32.888+0100 [DEBUG] provider.terraform-provider-databricks_v1.48.0: GET /api/2.0/policies/clusters/get?policy_id=001C043340CD70D0
< HTTP/2.0 200 OK
< {
<   "created_at_timestamp": 1719321848000,
<   "definition": "{\"autotermination_minutes\":{\"hidden\":true,\"type\":\"fixed\",\"value\":20},\"custom_tags.Team\":{\"type\":... (71 more bytes)",
<   "is_default": false,
<   "libraries": [
<     {
<       "maven": {
<         "coordinates": "com.oracle.database.jdbc:ojdbc8:XXXX"
<       }
<     }
<   ],
<   "name": "test cluster policy",
<   "policy_id": "001C043340CD70D0"
< }

[nkvuong]

the issue is with the user code - this is how you specify multiple libraries.

@edwardfeng-db normally, we need to alias plural to singular, i.e. library to avoid confusion for users. Especially as clusters & jobs use singular library

resource "databricks_cluster_policy" "fair_use" {
  name       = "test cluster policy"
  definition = jsonencode(local.default_policy)

  libraries {
    maven {
      coordinates = "com.oracle.database.jdbc:ojdbc8:XXXX"
    }
  }
  libraries {
    pypi {
      package = "databricks-sdk==0.12.0"
    }
  }
}