Terraform Plan deleting/replacing AKS Node Pool

[chkp-oferb]

Is there an existing issue for this?

• [X] I have searched the existing issues

Community Note

• Please vote on this issue by adding a :thumbsup: reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform Version

1.2.2

AzureRM Provider Version

3.8.0

Affected Resource(s)/Data Source(s)

azurerm_kubernetes_cluster_node_pool

Terraform Configuration Files

resource "azurerm_kubernetes_cluster" "aks" {
  for_each            = local.regions
  name                = "datatube-${local.env}-${each.value}-aks-v2"
  location            = each.value
  resource_group_name = azurerm_resource_group.aks[each.value].name
  dns_prefix          = "datatube${local.env}${each.value}aks-v2"

  default_node_pool {
    name                     = "system"
    node_count               = 2
    vm_size                  = "Standard_DS3_v2"
    os_disk_type             = "Ephemeral"
    enable_auto_scaling      = "true"
    max_count                = 2
    min_count                = 2
    type = "VirtualMachineScaleSets"
    zones                    = ["1", "2", "3"]
    vnet_subnet_id           = azurerm_subnet.backend[each.value].id
  }
  lifecycle  {
    ignore_changes = [
     "default_node_pool"
    ]
  }

  identity {type = "SystemAssigned"}

  ingress_application_gateway {
     gateway_id = azurerm_application_gateway.gateway[each.value].id
  }

  network_profile {
    network_plugin = "azure"
    network_policy = "azure"
  }

  tags = { Environment = "prod" }
}

resource "azurerm_kubernetes_cluster_node_pool" "apppool" {
  for_each              = local.regions
  name                  = "apppool"
  kubernetes_cluster_id = azurerm_kubernetes_cluster.aks[each.value].id
  vm_size               = "Standard_D8s_v3"
  node_count            = 3
  os_disk_type = "Ephemeral"
  enable_auto_scaling = "true"
  max_count = 5
  min_count = 3
  node_labels = { app = "proxy-app" }
  zones               = ["1", "2", "3"]
  tags = { Environment = "Production" }
  mode = "User"

#  lifecycle {
#    ignore_changes = [
#      "apppool"
#    ]
#  }
}

Debug Output/Panic Output

2-06-07T15:29:18.003+0300 [TRACE] DiffTransformer: azurerm_kubernetes_cluster_node_pool.apppool["southeastasia"] will be represented by azurerm_kubernetes_cluster_node_pool.apppool["southeastasia"]
2022-06-07T15:29:18.003+0300 [TRACE] DiffTransformer: azurerm_kubernetes_cluster_node_pool.apppool["southeastasia"] will be represented for destruction by azurerm_kubernetes_cluster_node_pool.apppool["southeastasia"] (destroy)

Expected Behaviour

With no change to the terraform manifest or the app node pool (user mode) - the resource would not effected by replace or delete

Actual Behaviour

after newly deployed manifest with 6 clusters in 6 regions. running again terraform plan, without any changes, will delete / replace all the app node pools.

Steps to Reproduce

Create AKS cluster add apppool node user mode 3 nodes add Application getaway / AKS ingress configure AGIC Apply

run again terraform plan

Important Factoids

No response

References

no

[NiklasRosenstein]

I'm currently facing the same issue. It says "forces replacement" for the kubernetes_cluster_id even thought it didn't change.

      ~ kubernetes_cluster_id  = "/subscriptions/33aaa451-2be4-4e1d-b677-29de9102e582/resourceGroups/kubernetes-dev/providers/Microsoft.ContainerService/managedClusters/REDACTED" -> "/subscriptions/33aaa451-2be4-4e1d-b677-29de9102e582/resourcegroups/kubernetes-dev/providers/Microsoft.ContainerService/managedClusters/REDACTED" # forces replacement

[ssrahul96]

+1 on the same,

no changes done, still says force replacement everytime i run terraform plan or terraform apply

[ssrahul96]

found a solution,

not sure whether its kept that way for a purpose,

adding vnet_subnet_id resolves this, i.e. its not replacing the nodepool

[rcskosir]

@oferbd9 Thank you for opening this issue. Was @ssrahul96's solution able to resolve your issue?

[ivica3730k]

@oferbd9 Thank you for opening this issue. Was @ssrahul96's solution able to resolve your issue?

thank you very much @ssrahul96 , @rcskosir it did work in my case

[ivica3730k]

reading the documentation, the vnet_subnet_id could be really set as mandatory parameter?

[ivica3730k]

In my case

resource "azurerm_kubernetes_cluster_node_pool" "extra-node-pool-1" { 
   name                  = "extrapool1" 
   kubernetes_cluster_id = azurerm_kubernetes_cluster.aks-cluster.id 
   #   kubernetes_cluster_id   = replace(azurerm_kubernetes_cluster.aks-cluster.id, "resourceGroups", "resourcegroups") 
   vm_size    = "Standard_D2_v2" 
   node_count = 2 
   max_pods   = 250 
   depends_on = [azurerm_kubernetes_cluster.aks-cluster] 

   vnet_subnet_id = azurerm_kubernetes_cluster.aks-cluster.default_node_pool[0].vnet_subnet_id 
   # check out https://stackoverflow.com/questions/67825862/terraform-forces-aks-node-pool-replacement-without-any-changes 
   #   lifecycle { 
   #     ignore_changes = [ 
   #       kubernetes_cluster_id 
   #     ] 
   #   } 
 }