[ISSUE] Issue with `databricks_sql_endpoint` resource attributes drift showing during TF plan for Serverless compute

[abiola-alowosile]

Configuration

# sql_endpoint.tf module file
resource "databricks_sql_endpoint" "sql_endpoint" {
  count                     = var.prepare_for_deletion ? 0 : 1
  name                      = var.name
  cluster_size              = var.cluster_size
  min_num_clusters          = var.min_num_clusters
  max_num_clusters          = var.max_num_clusters
  auto_stop_mins            = var.auto_stop_mins
  spot_instance_policy      = var.spot_instance_policy
  enable_photon             = var.enable_photon
  enable_serverless_compute = var.enable_serverless_compute
  warehouse_type            = var.warehouse_type

  channel {
    name = var.channel
  }

  tags {
    dynamic "custom_tags" {
      for_each = var.tags
      content {
        key   = custom_tags.key
        value = custom_tags.value
      }
    }
  }
}

#excerpt from main.tf

module "looker_sql_endpoint" {
  source = "../../../../modules/depop_databricks/sql_endpoint"

  providers = {
    databricks = databricks.workspace
  }

  name = "looker"

  cluster_size     = "Small"
  min_num_clusters = 1
  max_num_clusters = 2
  auto_stop_mins   = 1

  enable_serverless_compute = true
  warehouse_type            = "PRO"

  tags = local.default_tags

  management_teams = [
    module.depop_databricks_staging_workspace.group_data_engineering.display_name,
    module.depop_databricks_staging_workspace.group_analytics_engineering.display_name,
  ]

  using_principals = [
    module.staging_workspace_config.braze_service_principal_application_id,
    module.staging_workspace_config.looker_service_principle_application_id
  ]
}

Expected Behavior

Dynamic elements of serveless cluster ignored or not tracked by Terraform or clarification on expected behaviour.

Actual Behavior

I do not expect dynamic attributes of a serveless cluster like for example number of clusters (num_clusters) and state to be tracked during a Terraform plan as shown below:

I have tried ignoring the attributes in question using the ignore_lifecycle Terraform meta argument but the following error appears:

Steps to Reproduce

Please list the steps required to reproduce the issue, for example:

1. create serverless clusters
2. start serverless clusters
3. terraform plan

Terraform and provider versions

Terraform = v1.5.7 Terraform <> Databricks = 1.51.0

Is it a regression?

Debug Output

debug.log

[alexott]

Can you try with the latest provider version?