"azurerm_databricks_workspace" forces replacement on public_network_access_enabled while it never existed

[donheerschap]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform (and AzureRM Provider) Version

terraform 1.0.10 azurerm 2.94.0

Affected Resource(s)

• azurerm_databricks_workspace

Terraform Configuration Files

resource "azurerm_databricks_workspace" "databricks" {
  count                         = var.enabled ? 1 : 0
  name                          = var.name
  location                      = var.resource_group_location
  resource_group_name           = var.resource_group_name
  sku                           = "premium"

  dynamic "custom_parameters" {
    for_each = var.virtualnetwork_enabled ? [1] : []
    content {
      virtual_network_id  = var.virtualnetwork_id
      no_public_ip        = !var.enable_public_ip
      private_subnet_name = "databricks-private"
      public_subnet_name  = "databricks-public"

      public_subnet_network_security_group_association_id  = var.public_network_security_group_association_id
      private_subnet_network_security_group_association_id = var.private_network_security_group_association_id

      #aml_workspace_id = var.machine_learning_workspace_id this shows up in the docs but isn't supported
    }
  }
  tags = merge(var.generic_tags, { Category = local.category })
}

Debug Output

# module.databricks.azurerm_databricks_workspace.databricks[0] must be replaced
-/+ resource "azurerm_databricks_workspace" "databricks" {
      ~ id                                    = "/subscriptions/.../resourceGroups/.../providers/Microsoft.Databricks/workspaces/..." -> (known after apply)
      ~ managed_resource_group_id             = "/subscriptions/.../resourceGroups/databricks-rg-..." -> (known after apply)
      ~ managed_resource_group_name           = "databricks-rg-..." -> (known after apply)
        name                                  = "..."
      + network_security_group_rules_required = (known after apply)
      + public_network_access_enabled         = true # forces replacement
      ~ storage_account_identity              = [] -> (known after apply)
        tags                                  = {
            "Category"   = "Processing"
            "Deployment" = "Automated"
        }
      ~ workspace_id                          = "..." -> (known after apply)
      ~ workspace_url                         = "adb-....14.azuredatabricks.net" -> (known after apply)
        # (5 unchanged attributes hidden)

      ~ custom_parameters {
          + nat_gateway_name                                     = (known after apply)
          + private_subnet_network_security_group_association_id = "/subscriptions/.../resourceGroups/.../providers/Microsoft.Network/virtualNetworks/.../subnets/databricks-private"
          + public_ip_name                                       = (known after apply)
          + public_subnet_network_security_group_association_id  = "/subscriptions/.../resourceGroups/..../providers/Microsoft.Network/virtualNetworks/.../subnets/databricks-public"
          ~ storage_account_name                                 = "..." -> (known after apply)
          ~ storage_account_sku_name                             = "Standard_GRS" -> (known after apply)
          ~ vnet_address_prefix                                  = "10.139" -> (known after apply)
            # (4 unchanged attributes hidden)
        }

      - timeouts {}
    }

Expected Behaviour

The resource "azurerm_databricks_workspace" should not be in the plan at all, as nothing changed

Actual Behaviour

It forces a replacement to set a non-existent variable in the configuration to a default which already should be true.

Steps to Reproduce

1. terraform plan

Other scenarios:

1. add the public_network_access_enabled and set to false -> results also in forcing replacement but then set to false
2. removing and importing state while the public_network_access_enabled is either true or false -> both gives a force replacement.

Important Factoids

References

• 0000

[donheerschap]

After some further investigation, de state file itself showed "public_network_access_enabled": null, for this resource, while other environments which use the exact same configuration already had "public_network_access_enabled": true, because that's the default, and how the environment in azure actually is configured aswell.

As this was one of our oldest environments I guess that somewhere within updating the databricks workspace resource had correct handling of adding the public_network_access_enabled parameter in previous provider versions in the refresh, while now it just forces it without refreshing what the current sitiuation is.

[sehgalnamit]

"public_network_access_enabled": null doe snot work for me. It still shows as

• public_network_access_enabled = true # forces replacement

[sehgalnamit]

I tried , it works without replacement provider "azurerm" { features { }

version = "<= 2.73.0" }

My issue was using tf12 upgrading from tf11

[heathen1878]

I've seen this, my scenario is moving from the Azure provider version 2.61.0 to 2.87.0 - public_network_access_enabled wasn't even a valid attribute in version 2.61.0 and my state file doesn't even list the public_network_access_enabled attribute.

As a workaround for existing deployments of databricks, i've used the lifecycle meta-argument, for example

resource "azurerm_databricks_workspace" "databricks" {
...
    lifecycle {
        ignore_changes = [
           public_network_access_enabled
        ]
    }
...
}

[Alex0M]

Terraform forces a replacement starting from Azure provider version 2.82.0. In version 2.81.0 all is fine but when I try to update to version 2.82.0 I see this issue.

[Triamus]

This still happens for me on a newly (manually) deployed Azure Databricks workspace with Terraform v1.3.0 and azurerm v3.0.0. I had imported the resource into tf and terraform plan produces the same behavior for following attribute settings:

• public_network_access_enabled = null
• public_network_access_enabled = true
• public_network_access_enabled = false
• leave public_network_access_enabled out of config

So does that mean that the import is not properly importing the attribute? The state file after import showed "public_network_access_enabled": "".