[ISSUE] Issue with `databricks_permissions` resource. Cannot read permissions: resource type for /warehouses not found

[NicholasFiorentini]

Configuration

terraform {
  backend "remote" {
    organization = "my-org"
    workspaces {
      name = "databricks-workspace"
    }
  }
  required_providers {
    databricks = {
      source  = "databricks/databricks"
      version = "~> 1.36, != 1.39.0"
    }
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.0"
    }
  }
  required_version = "~> 1.8"
}

provider "databricks" {
  alias = "workspace"
  host  = "my-ws-domain"
  token = var.databricks_token
}

resource "databricks_sql_global_config" "this" {
  security_policy      = "DATA_ACCESS_CONTROL"
  instance_profile_arn = "AWS IP ARN"
  provider             = databricks.workspace
}

resource "databricks_sql_endpoint" "this" {
  provider         = databricks.workspace
  name             = "Endpoint for team"
  cluster_size     = "Medium"
  max_num_clusters = 1
  auto_stop_mins   = 10
}

resource "databricks_permissions" "sql_endpoint" {
  provider        = databricks.workspace
  sql_endpoint_id = databricks_sql_endpoint.this.id

  access_control {
    user_name        = "my_user@my_domain.com"
    permission_level = "IS_OWNER"
  }
  access_control {
    group_name       = "users"
    permission_level = "CAN_USE"
  }

Expected Behavior

The plan should not fail.

Actual Behavior

Terraform plan fails with

Error: cannot read permissions: resource type for /warehouses/*** not found
with *** on .terraform/***.tf line 9, in resource "databricks_permissions" "sql_endpoint":

resource "databricks_permissions" "sql_endpoint" {

Steps to Reproduce

1. terraform plan

Terraform and provider versions

Terraform v1.8.3
+ provider registry.terraform.io/databricks/databricks v1.54.0
+ provider registry.terraform.io/hashicorp/aws v5.72.1
+ provider registry.terraform.io/hashicorp/random v3.6.3

Is it a regression?

This configuration has been working fine since we last applied a plan in this workspace (15 days ago). We haven't changed any configuration; the token is valid and associated with a service account in the admin group.

The SQL endpoint is visible from Databricks UI and starts correctly. The ID is the same as the one reported in the error.

Debug Output

To be provided.

Important Factoids

This fails for both "classic" and "serverless" SQL endpoints in two different workspaces.

Output of databricks warehouses list -o json:

[
  {
    "auto_stop_mins": 10,
    "channel": {},
    "cluster_size": "Medium",
    "enable_photon": true,
    "enable_serverless_compute": false,
    "id": "***",
    "jdbc_url": "jdbc:spark://***.cloud.databricks.com:443/default;transportMode=http;ssl=1;AuthMech=3;httpPath=/sql/1.0/warehouses/***;",
    "max_num_clusters": 1,
    "min_num_clusters": 1,
    "name": "Endpoint for team",
    "num_active_sessions": 0,
    "num_clusters": 0,
    "odbc_params": {
      "hostname": "***cloud.databricks.com",
      "path": "/sql/1.0/warehouses/***",
      "port": 443,
      "protocol": "https"
    },
    "spot_instance_policy": "COST_OPTIMIZED",
    "state": "STOPPED",
    "tags": {},
    "warehouse_type": "CLASSIC"
  }
]

Output for databricks warehouses get-permissions ***:

{
  "access_control_list": [
    {
      "all_permissions": [
        {
          "inherited":false,
          "permission_level":"IS_OWNER"
        }
      ],
      "display_name":"my_user",
      "user_name":"my_user@my_domain.com"
    },
    {
      "all_permissions": [
        {
          "inherited":false,
          "permission_level":"CAN_USE"
        }
      ],
      "group_name":"users"
    },
    {
      "all_permissions": [
        {
          "inherited":true,
          "inherited_from_object": [
            "/sql/warehouses/"
          ],
          "permission_level":"CAN_MANAGE"
        }
      ],
      "group_name":"admins"
    }
  ],
  "object_id":"/sql/warehouses/***",
  "object_type":"warehouses"
}

Would you like to implement a fix?

No

[NicholasFiorentini]

I tested it with 1.52, and the plan works. This looks like a regression introduced by 1.53. Possibly due to this change?

databricks = {
   source  = "databricks/databricks"
  version = "< 1.53"
}

[mgyucht]

Definitely is caused by that. The IDs for this resource are a bit weird, especially for the SQL endpoints. I assumed that the ID always began with /sql/warehouses but apparently sometimes it starts with just /warehouses. I should have a fix ready for this by this afternoon.

[NicholasFiorentini]

Thank you very much @mgyucht

[NicholasFiorentini]

Thanks @mgyucht for the PR! Really appreciated! Do you know when it will be released?

[NicholasFiorentini]

Nevermind, just saw it.

[NicholasFiorentini]

Confirmed as fixed.