Azure / terraform-azurerm-aks

Terraform Module for deploying an AKS cluster
MIT License
327 stars 445 forks source link

Error: Invalid for_each argument | create_role_assignment_network_contributor #572

Open prosaied opened 3 days ago

prosaied commented 3 days ago

Is there an existing issue for this?

Greenfield/Brownfield provisioning

greenfield

Terraform Version

1.5.4

Module Version

9.0.0

AzureRM Provider Version

3.109.0

Affected Resource(s)/Data Source(s)

azurerm_kubernetes_cluster_node_pool

Terraform Configuration Files

module "aks" {
  source  = "Azure/aks/azurerm"
  version = "7.3.2"

  prefix                                     = "example"
  cluster_name                               = var.cluster_name
  enable_auto_scaling                        = true
  agents_pool_name                           = "systempool"
  agents_size                                = var.node_size
  agents_max_count                           = var.node_count.max
  agents_min_count                           = var.node_count.min
  agents_labels                              = var.agents_labels
  agents_taints                              = var.agents_taints
  agents_tags                                = var.agents_tags
  only_critical_addons_enabled               = true
  orchestrator_version                       = var.kubernetes_version
  kubernetes_version                         = var.kubernetes_version
  os_disk_type                               = "Managed"
  os_sku                                     = "Ubuntu"
  temporary_name_for_rotation                = var.temporary_name_for_rotation # enter this value when changing any of the default/system nodepool properties
  vnet_subnet_id                             = var.vnet_subnet_id
  agents_availability_zones                  = var.availability_zones
  resource_group_name                        = var.resource_group_name
  os_disk_size_gb                            = var.os_disk_size_gb
  sku_tier                                   = "Standard"
  node_pools                                 = var.node_pools
  network_plugin                             = "azure"
  network_plugin_mode                        = "overlay"
  net_profile_outbound_type                  = "userAssignedNATGateway"
  role_based_access_control_enabled          = true
  rbac_aad_managed                           = true
  local_account_disabled                     = true
  log_analytics_workspace_enabled            = false
  oidc_issuer_enabled                        = true
  workload_identity_enabled                  = true
  rbac_aad_azure_rbac_enabled                = true
  rbac_aad_admin_group_object_ids            = var.rbac_aad_admin_group_object_ids
  create_role_assignment_network_contributor = var.create_role_assignment_network_contributor
  web_app_routing                            = var.web_app_routing

tfvars variables values

resource_group_name = "staging"

node_size                   = "Standard_D8s_v3"
os_disk_size                = 100
temporary_name_for_rotation = null
environment                 = "staging"
kubernetes_version          = "1.27.9"

agents_labels = {}
agents_taints = []
agents_tags   = {}

availability_zones         = ["1", "2", "3"]
create_platform_namespaces = true
create_rbac_roles          = true

subscription_id = "XXXXXXXXXXXXXXXX"
env_prefix      = "pg"

create_role_assignment_network_contributor = true

Debug Output/Panic Output

╷
│ Error: Invalid for_each argument
│
│   on .terraform/modules/aks.aks.aks/main.tf line 790, in resource "azurerm_role_assignment" "network_contributor":
│  790:   for_each = var.create_role_assignment_network_contributor && (var.client_id == "" || var.client_secret == "") ? local.subnet_ids : []
│     ├────────────────
│     │ local.subnet_ids is a set of dynamic, known only after apply
│     │ var.client_id is ""
│     │ var.client_secret is ""
│     │ var.create_role_assignment_network_contributor is true
│
│ The "for_each" set includes values derived from resource attributes that cannot be determined until apply, and so Terraform cannot determine the full set of keys that will identify the instances of this
│ resource.
│
│ When working with unknown values in for_each, it's better to use a map value where the keys are defined statically in your configuration and where only the values contain apply-time results.
│
│ Alternatively, you could use the -target planning option to first apply only the resources that the for_each value depends on, and then apply a second time to fully converge.

Expected Behaviour

When the variable create_role_assignment_network_contributor is set to true, it should create and iterate over resources.

Actual Behaviour

On the first run, you will encounter an invalid for_each loop if the variable create_role_assignment_network_contributor is set to true. To avoid this, set the variable to false initially. After the cluster is created, you can then set it to true, and it will work fine. Additionally, any changes to the VNet resource can also cause this error.

We are currently using version 7.3.2 of the "Azure/aks/azurerm" module in our environments. Both new and old deployments use this version, but the issue persists regardless of the deployment time. Notably, this bug still exists in version 9.0.0.

Steps to Reproduce

No response

Important Factoids

No response

References

No response

zioproto commented 3 days ago

hello @prosaied

this is a known problem and it was fixed in PR https://github.com/Azure/terraform-azurerm-aks/pull/410

the variable create_role_assignment_network_contributor is deprecated and will go away in a future version.

Please migrate to the new network_contributor_role_assigned_subnet_ids variable.