[ISSUE] Issue with `databricks_sql_table` resource

[mahmoud-masmoudi-dev]

Issue with databricks_sql_table resource. The "warehouse_id" seems to be ignored, and Terraform is always creating a special cluster "terraform-sql-table" for CRUD operations.

Configuration

resource "databricks_sql_table" "this" {
  provider           = databricks.dataengineer
  name               = var.table
  catalog_name       = var.catalog
  schema_name        = var.schema
  table_type         = "EXTERNAL"
  data_source_format = "DELTA"
  storage_location   = var.location

  comment = var.comment

  warehouse_id = "893a0ff662ac09de"
}

Expected Behavior

I expected terraform/databricks to use the specified warehouse to create the databricks_sql_table resources

Actual Behavior

terraform/databricks is always creating a cluster "terraform-sql-table" and ignoring the specified warehouse. Even the plan shows that terraform is generating a "cluster_id", even though, warehouse_id is explicitely specified

module.external_delta_table["bronze_gtfs_trips"].databricks_sql_table.this will be created

• resource "databricks_sql_table" "this" {
  • catalog_name = ""
  • cluster_id = (known after apply)
  • comment = ""
  • data_source_format = "DELTA"
  • id = (known after apply)
  • name = "bronze_gtfs_trips"
  • properties = (known after apply)
  • schema_name = ""
  • storage_location = ""
  • table_type = "EXTERNAL"
  • warehouse_id = "893a0ff662ac09de" }

Steps to Reproduce

1. terraform apply

Terraform and provider versions

Terraform v1.6.5 on linux_amd64

• provider registry.terraform.io/databricks/databricks v1.30.0
• provider registry.terraform.io/hashicorp/aws v5.29.0
• provider registry.terraform.io/hashicorp/null v3.2.2

[nkvuong]

@mahmoud-masmoudi-dev could you share debug logs per the template so we could investigate further?

[soujyosen]

This is the same behavior for us. The warehouse_id or cluster_id field is just ignored and a default cluster "terraform-sql-table" is created always in a never-ending loop. Please upvote this issue and help get resolved.

[CarreauClement]

Having the same issue here.

[Dedvall]

We´re not having this issue with provider >= 1.38, has been this resolved @nkvuong

[bjakubiak]

The issue still exists. This is the function that defines cluster to create: https://github.com/databricks/terraform-provider-databricks/blob/b827aeccb86c1cd7c5803e8a755d7085ebbdb207/access/resource_sql_permissions.go#L279 It defines cluster with name "terraform-table-acl" now. And calls this function: https://github.com/databricks/terraform-provider-databricks/blob/b827aeccb86c1cd7c5803e8a755d7085ebbdb207/clusters/clusters_api.go#L881 which checks if "terraform-table-acl" named cluster exists. If it doesn't, it creates new one according to passed definition.

It looks like the idea was rather to have attribute "cluster_name" instead of "cluster_id". Then to have this attribute used instead of hardcoded "terraform-table-acl" so if such cluster exists it can be used.