azurerm_databricks_workspace you must define a value for 'public_subnet_network_security_group_association_id'

onxades commented 2 years ago

Is there an existing issue for this?

[X] I have searched the existing issues

Community Note

Please vote on this issue by adding a :thumbsup: reaction to the original issue to help the community and maintainers prioritize this request
Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform Version

1.3.3

AzureRM Provider Version

3.29.1

Affected Resource(s)/Data Source(s)

azurerm_databricks_workspace

Terraform Configuration Files

module "subnet_public" {
  source = "../subnet"

  custom_name            = var.network_configuration.public_subnet.name
  resource_group_name    = var.network_configuration.virtual_network.rg
  instance_code          = var.instance_code
  location               = var.location
  environment            = var.environment
  environment_short_code = var.environment_short_code
  extra_tags             = merge(local.default_tags, var.extra_tags)

  address_prefixes           = var.network_configuration.public_subnet.address_prefixes
  virtual_network_name       = var.network_configuration.virtual_network.name
  delegation_name            = var.network_configuration.public_subnet.delegation_name
  delegation_service_name    = var.network_configuration.public_subnet.delegation_service_name
  delegation_service_actions = var.network_configuration.public_subnet.delegation_service_actions
}

module "subnet_private" {
  source = "../subnet"

  custom_name            = var.network_configuration.private_subnet.name
  resource_group_name    = var.network_configuration.virtual_network.rg
  instance_code          = var.instance_code
  location               = var.location
  environment            = var.environment
  environment_short_code = var.environment_short_code
  extra_tags             = merge(local.default_tags, var.extra_tags)

  address_prefixes           = var.network_configuration.private_subnet.address_prefixes
  virtual_network_name       = var.network_configuration.virtual_network.name
  delegation_name            = var.network_configuration.private_subnet.delegation_name
  delegation_service_name    = var.network_configuration.private_subnet.delegation_service_name
  delegation_service_actions = var.network_configuration.private_subnet.delegation_service_actions
}

resource "azurerm_subnet_network_security_group_association" "public" {
  subnet_id                 = module.subnet_public.id
  network_security_group_id = var.network_configuration.databricks_nsg_id
}

resource "azurerm_subnet_network_security_group_association" "private" {
  subnet_id                 = module.subnet_private.id
  network_security_group_id = var.network_configuration.databricks_nsg_id
}

resource "azurerm_databricks_workspace" "main" {
  name                = local.name
  resource_group_name = var.resource_group_name
  location            = var.location
  tags                = merge(local.default_tags, var.extra_tags)

  sku                           = local.sku
  managed_resource_group_name   = local.managed_resource_group_name
  public_network_access_enabled = var.public_network_access_enabled

  custom_parameters {
    machine_learning_workspace_id = var.machine_learning_workspace_id

    public_subnet_name                                   = module.subnet_public.name
    private_subnet_name                                  = module.subnet_private.name
    public_subnet_network_security_group_association_id  = azurerm_subnet_network_security_group_association.public.id
    private_subnet_network_security_group_association_id = azurerm_subnet_network_security_group_association.private.id
    virtual_network_id                                   = var.network_configuration.virtual_network.id
  }

  lifecycle {
    ignore_changes = [
      custom_parameters["machine_learning_workspace_id"],
      public_network_access_enabled
    ]
  }
}

Debug Output/Panic Output

│ Error: you must define a value for 'public_subnet_network_security_group_association_id' if 'public_subnet_name' is set
│ 
│   with module.prism.module.test_adb_abcinc.azurerm_databricks_workspace.main,
│   on ../../../../prism_terraform_modules/modules/azure/databricks _vnet_injected/main.tf line 48, in resource "azurerm_databricks_workspace" "main":
│   48: resource "azurerm_databricks_workspace" "main" {

Expected Behaviour

Terraform apply should update resource without issue.

Actual Behaviour

Terraform plan runs fine. On running terraform apply: Error: you must define a value for 'public_subnet_network_security_group_association_id' if 'public_subnet_name' is set

the public_subnet_network_security_group_association_id is set

Steps to Reproduce

Run terraform plan Run terraform apply

Important Factoids

No response

References

No response

onxades commented 2 years ago

Only other note I can add, I imported the ADB, subnets, and nsgs, when I run plan it shows tags to update for the resource, but it's the apply running into the error above shows up even though both public and private nsgs in custom parameters is set.

sinbai commented 2 years ago

@onxades thanks for opening this issue. Per the code below, the above error occurs when public_subnet_name is empty and public_subnet_network_security_group_association_id is nil, Could you try outputting the actual values of public_subnet_name and public_subnet_network_security_group_association_id to help troubleshoot? Also, could you provide a full tf configuration and detailed reproduce steps to help reproduce above error?

BTW, the following tf configuration works fine on my local.

terraform {
  required_providers {
    azurerm = {
      source = "hashicorp/azurerm"
      version = "3.29.0"
    }
  }
}

provider "azurerm" {
  features {}
}

resource "azurerm_resource_group" "test" {
  name     = "acctestRG-databricks-1101"
  location = "eastus"
}

resource "azurerm_virtual_network" "test" {
  name                = "acctest-vnet-1101"
  location            = azurerm_resource_group.test.location
  resource_group_name = azurerm_resource_group.test.name
  address_space       = ["10.0.0.0/16"]
}

resource "azurerm_subnet" "public" {
  name                 = "acctest-sn-public-1101"
  resource_group_name  = azurerm_resource_group.test.name
  virtual_network_name = azurerm_virtual_network.test.name
  address_prefixes     = ["10.0.1.0/24"]

  delegation {
    name = "acctest"

    service_delegation {
      name = "Microsoft.Databricks/workspaces"

      actions = [
        "Microsoft.Network/virtualNetworks/subnets/join/action",
        "Microsoft.Network/virtualNetworks/subnets/prepareNetworkPolicies/action",
        "Microsoft.Network/virtualNetworks/subnets/unprepareNetworkPolicies/action",
      ]
    }
  }
}

resource "azurerm_subnet" "private" {
  name                 = "acctest-sn-private-1101"
  resource_group_name  = azurerm_resource_group.test.name
  virtual_network_name = azurerm_virtual_network.test.name
  address_prefixes     = ["10.0.2.0/24"]

  delegation {
    name = "acctest"

    service_delegation {
      name = "Microsoft.Databricks/workspaces"

      actions = [
        "Microsoft.Network/virtualNetworks/subnets/join/action",
        "Microsoft.Network/virtualNetworks/subnets/prepareNetworkPolicies/action",
        "Microsoft.Network/virtualNetworks/subnets/unprepareNetworkPolicies/action",
      ]
    }
  }
}

resource "azurerm_network_security_group" "nsg" {
  name                = "acctest-nsg-private-1101"
  location            = azurerm_resource_group.test.location
  resource_group_name = azurerm_resource_group.test.name
}

resource "azurerm_subnet_network_security_group_association" "public" {
  subnet_id                 = azurerm_subnet.public.id
  network_security_group_id = azurerm_network_security_group.nsg.id
}

resource "azurerm_subnet_network_security_group_association" "private" {
  subnet_id                 = azurerm_subnet.private.id
  network_security_group_id = azurerm_network_security_group.nsg.id
}

resource "azurerm_databricks_workspace" "test" {
  depends_on = [azurerm_subnet_network_security_group_association.public, azurerm_subnet_network_security_group_association.private]

  name                        = "acctestDBW-1101"
  resource_group_name         = azurerm_resource_group.test.name
  location                    = azurerm_resource_group.test.location
  sku                         = "standard"
  managed_resource_group_name = "acctestRG-DBW-1101-managed"

  custom_parameters {
    no_public_ip        = true
    public_subnet_name  = azurerm_subnet.public.name
    private_subnet_name = azurerm_subnet.private.name
    virtual_network_id  = azurerm_virtual_network.test.id

    public_subnet_network_security_group_association_id  = azurerm_subnet_network_security_group_association.public.id
    private_subnet_network_security_group_association_id = azurerm_subnet_network_security_group_association.private.id
  }

  tags = {
    Pricing = "Standard"
  }
}

onxades commented 2 years ago

I'll take a look at this and get back with the outputs, thank you for the point out on the validations!

onxades commented 2 years ago

I was able to fix the issue. It looks like the following was causing the issue:

A custom_parameters block supports the following:

machine_learning_workspace_id - (Optional) The ID of a Azure Machine Learning workspace to link with Databricks workspace. Changing this forces a new resource to be created.

I had originally set the machine_learning_workspace_id to an empty string "" and this was causing the ADB module to not see the nsg association ids to be seen.

I think this is still valid as a bug, as an empty machine_learning_workspace_id doesn't have any validation that prevents this from being set to "", and it causes the nsg associations to not be seen when running a plan or apply.

onxades commented 2 years ago

also adding a lifecycle hook to ignore machine_learning_workspace_id also causes the issue.

hashicorp / terraform-provider-azurerm