databricks / terraform-provider-databricks

Databricks Terraform Provider
https://registry.terraform.io/providers/databricks/databricks/latest
Other
456 stars 393 forks source link

[ISSUE] Issue with `databricks_jobs` resource when tasks are not sorted alphabetically #4159

Open goda opened 1 month ago

goda commented 1 month ago

Same issue as #3126 which was incorrectly closed. Verified that this issue still exists with provider v1.54.0

Configuration

resource "databricks_job" "this" {
  name = "Test_Job"

  task {
    task_key        = "ZZ_test"
    job_cluster_key = "my-cluster"

    notebook_task {
      notebook_path = "test_task_key_order/helloworld"
    }
  }

  task {
    task_key        = "AA_test"
    job_cluster_key = "my-cluster"

    depends_on {
      task_key = "ZZ_test"
    }

    notebook_task {
      notebook_path = "test_task_key_order/helloworld"
    }
  }

  job_cluster {
    job_cluster_key = "my-cluster"
    new_cluster {
      spark_version       = "13.3.x-cpu-ml-scala2.12"
      node_type_id        = "Standard_F4s"
      driver_node_type_id = "Standard_F4s"
      spark_env_vars      = local.env_vars
      runtime_engine      = "STANDARD"

      custom_tags = local.job_tags

      autoscale {
        min_workers = 1
        max_workers = 20
      }

      spark_conf = local.spark_conf
    }
  }

}

Expected Behavior

When creating a new change, unrelated to this resource, such as adding a new variable, the terraform plan for this resource should show no changes. This is exact same issue identified in #3126 which was closed by mistake.

Actual Behavior

Terraform will perform the following actions:

  # module.test_task_key_order.databricks_job.this will be updated in-place
  ~ resource "databricks_job" "this" {
        id                        = "<redacted>"
        name                      = "Test_Job"
        tags                      = {<redacted> }
        # (9 unchanged attributes hidden)

      ~ task {
          ~ task_key                  = "AA_test" -> "ZZ_test"
            # (6 unchanged attributes hidden)

          - depends_on {
              - task_key = "ZZ_test" -> null
            }

            # (2 unchanged blocks hidden)
        }
      ~ task {
          ~ task_key                  = "ZZ_test" -> "AA_test"
            # (6 unchanged attributes hidden)

          + depends_on {
              + task_key = "ZZ_test"
            }

            # (2 unchanged blocks hidden)
        }

        # (7 unchanged blocks hidden)
    }

Plan: 0 to add, 1 to change, 0 to destroy.

Steps to Reproduce

terraform apply

Terraform and provider versions

Terraform v1.9.8
on windows_amd64
+ provider registry.terraform.io/databricks/databricks v1.54.0

Is it a regression?

Not that I know of.

Debug Output

Important Factoids

Would you like to implement a fix?