hashicorp / terraform-provider-azurerm

Terraform provider for Azure Resource Manager
https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs
Mozilla Public License 2.0
4.62k stars 4.65k forks source link

azurerm_kubernetes_cluster property api_server_access_profile changes on every terraform run #27119

Open fplantinga-guida opened 3 months ago

fplantinga-guida commented 3 months ago

Is there an existing issue for this?

Community Note

Terraform Version

1.9.5

AzureRM Provider Version

3.116.0

Affected Resource(s)/Data Source(s)

azurerm_kubernetes_cluster

Terraform Configuration Files

resource "azurerm_resource_group" "example" {
  name     = "example-resources"
  location = "West Europe"
}

variable "api_server_authorized_ip_ranges" {
  description = "(Optional) The IP ranges to whitelist for incoming traffic to the masters. API server authorized IP address ranges are not supported for private clusters"
  type        = list(string)
  default     = []
}

resource "azurerm_kubernetes_cluster" "example" {
  name                    = "example-aks1"
  location                = azurerm_resource_group.example.location
  resource_group_name     = azurerm_resource_group.example.name
  dns_prefix              = "exampleaks1"
  private_cluster_enabled = true
  api_server_access_profile {
    authorized_ip_ranges = length(var.api_server_authorized_ip_ranges) > 0 ? var.api_server_authorized_ip_ranges : []
  }

  default_node_pool {
    name       = "default"
    node_count = 1
    vm_size    = "Standard_D2_v2"
  }

  identity {
    type = "SystemAssigned"
  }

  tags = {
    Environment = "Production"
  }
}

Debug Output/Panic Output

# module.aks.azurerm_kubernetes_cluster.main will be updated in-place
  ~ resource "azurerm_kubernetes_cluster" "main" {
        id                                  = ""
        name                                = "example-aks1"
        tags                                = {}
        # (40 unchanged attributes hidden)

      + api_server_access_profile {}

        # (8 unchanged blocks hidden)
    }

Expected Behaviour

When private_cluster_enabled == true and api_server_authorized_ip_ranges variable is set to [], there should be no changes for the api_server_access_profile after the initial run.

Actual Behaviour

The property changes on every terraform run. We can workaround this by removing the entire api_server_access_profile block or using a lifecycle ignore for api_server_access_profile. We use a terraform module to deploy public and private AKS clusters so it would be nice if we can keep the property in the module. I noticed this behaviour changed in version 3.114.0

Steps to Reproduce

  1. terraform apply
  2. terraform plan and notice the changes.

Important Factoids

No response

References

No response

MarkKharitonov commented 3 months ago

I observe the same behavior.

superff commented 2 months ago

why the subnet_id and vnet_integration_enabled properties removed in 3.116.0? ` subnet_id - (Optional) The ID of the Subnet where the API server endpoint is delegated to.

vnet_integration_enabled - (Optional) Should API Server VNet Integration be enabled? For more details please visit Use API Server VNet Integration. `

ana-cozma commented 2 weeks ago

Any update on this one?

sm-gravid-day commented 1 week ago

Still broken in azurerm 4.10.0

jemag commented 1 week ago

This post mentions it being removed for version 4+ of the provider due to the stable API use requirement: https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/guides/4.0-upgrade-guide#aks-migration-to-stable-api

Although not sure why it would be removed in any 3.x

Still think that moving to a stable only version of the AKS API makes using Terraform with AKS much less practical, and don't think this is the right choice moving forward. Explicit mention of preview features should be good enough for users while still allowing them to play around with preview features using their existing Terraform code.