[ISSUE] Issue with `databricks_cluster` resource when configuring a single node cluster

[willgraybeal]

Configuration

resource "databricks_cluster" "cluster" {
  cluster_name            = local.stack_name
  autotermination_minutes = 30
  is_pinned               = true
  spark_version           = var.databricks_spark_version_id
  instance_pool_id        = var.databricks_instance_pool_id
  driver_instance_pool_id = var.databricks_instance_pool_id

  ssh_public_keys = []

  spark_conf = {
    "spark.databricks.cluster.profile" : "singleNode"
    "spark.master" : "local[*]"
     ...
  }

  custom_tags = {
    "ResourceClass" = "SingleNode"
    ...
  }

  aws_attributes {
    instance_profile_arn = some_arn
  }
}

Expected Behavior

Running terraform plan generates a plan successfully

Actual Behavior

Running terraform plan returns the following error

╷
│ Error: Provider produced invalid plan
│
│ Provider "registry.terraform.io/databricks/databricks" planned an invalid value for
│ module.main.module.databricks_cluster[0].databricks_cluster.cluster.num_workers: planned value cty.NumberIntVal(0) for a non-computed attribute.
│
│ This is a bug in the provider, which should be reported in the provider's own issue tracker.

Steps to Reproduce

1. Configure a databricks_cluster resource as a Single Node cluster, per the docs here
2. Run terraform plan

Terraform and provider versions

Terraform v1.1.4
on darwin_amd64
+ provider registry.terraform.io/databricks/databricks v1.51.0

Is it a regression?

Yes, I tried version 1.50.0 and it succeeded

Debug Output

Important Factoids

Would you like to implement a fix?

[alexott]

you need to add explicit num_workers = 0 - most probably it's related to the SDK migration

[willgraybeal]

you need to add explicit num_workers = 0 - most probably it's related to the SDK migration

Adding num_workers = 0 results in the same plan error. Also worth pointing out that the existing docs say num_workers "can be set to 0 explicitly, or simply not specified, as it defaults to 0. "

[840]

Could you try to upgrade the Terraform CLI to v1.1.5 or higher? I was able to reproduce this issue only on v1.1.4 and lower.

[840]

To add to my previous message, Terraform CLI v1.1.5 appears to be the earliest version that supports protocol v6, which is required for the latest Databricks Terraform Provider (v1.51.0).

The docs are now updated to indicate v1.1.5 as a minimum requirement (#3965)