hashicorp / terraform-provider-azurerm

Terraform provider for Azure Resource Manager
https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs
Mozilla Public License 2.0
4.61k stars 4.65k forks source link

`azurerm_resources` randomly fails to find an existing resource using the `required_tags` argument. #11810

Open arienkock opened 3 years ago

arienkock commented 3 years ago

Terraform (and AzureRM Provider) Version

0.15.3

Affected Resource(s)

Terraform Configuration Files

terraform {
  required_providers {
    azurerm = {
      source  = "hashicorp/azurerm"
      version = "=2.59.0"
    }
    random = {
      source  = "hashicorp/random"
      version = "3.1.0"
    }
    kubernetes = {
      source  = "hashicorp/kubernetes"
      version = "2.2.0"
    }
  }
  backend "azurerm" {
    container_name = "tfstate"
  }
}
provider "azurerm" {
  features {}
}
provider "kubernetes" {
  config_path = "~/.kube/config"
}
provider "random" {
}

variable "project_id" {
  type = string
}
variable "domain" {
  type = string
}
variable "tier" {
  type = string
}

resource "random_password" "mysql_admin_password" {
  length  = 16
  special = true
  keepers = {
    "version" = "3"
  }
}

resource "azurerm_mysql_server" "mysql_server" {
  name                = "${var.project_id}${var.tier}49gksh2fg"
  location            = data.azurerm_resource_group.rg.location
  resource_group_name = data.azurerm_resource_group.rg.name

  administrator_login          = "mysqladminun"
  administrator_login_password = random_password.mysql_admin_password.result

  sku_name   = "B_Gen5_1"
  storage_mb = 5120
  version    = "5.7"

  backup_retention_days            = 7
  ssl_enforcement_enabled          = true
  ssl_minimal_tls_version_enforced = "TLS1_2"
  auto_grow_enabled                = true
}

resource "azurerm_mysql_firewall_rule" "kubernetes_cluster_access" {
  name                = "${var.tier}-k8s-cluster"
  resource_group_name = data.azurerm_resource_group.rg.name
  server_name         = azurerm_mysql_server.mysql_server.name
  start_ip_address    = data.azurerm_public_ip.outbound_ip.ip_address
  end_ip_address      = data.azurerm_public_ip.outbound_ip.ip_address
}
data "azurerm_resources" "outbound_ip_info" {
  type                = "Microsoft.Network/publicIPAddresses"
  resource_group_name = "${var.project_id}-${var.tier}-aksnode-rg"
  required_tags = {
    type = "aks-slb-managed-outbound-ip"
  }
}
data "azurerm_public_ip" "outbound_ip" {
  depends_on = [
    data.azurerm_resources.outbound_ip_info
  ]
  name                = data.azurerm_resources.outbound_ip_info.resources[0].name
  resource_group_name = "${var.project_id}-${var.tier}-aksnode-rg"
}

resource "kubernetes_secret" "mysql_server_secrets" {
  metadata {
    name = "mysqlserversecrets"
  }

  data = {
    administrator_login          = azurerm_mysql_server.mysql_server.administrator_login
    administrator_login_password = random_password.mysql_admin_password.result
    fqdn                         = azurerm_mysql_server.mysql_server.fqdn
  }
}

data "azurerm_resource_group" "rg" {
  name = "${var.project_id}-${var.tier}-rg"
}

Debug Output

Expected Behaviour

The azurerm_resources should query and find (an existing) public IP resource using a tag key value pair. Subsequently the azurerm_public_ip resource should be able to reference the first item of the resources list like this: data.azurerm_resources.outbound_ip_info.resources[0].name.

Actual Behaviour

The vast majority of time this works fine. It will randomly fail with this error:

╷
│ Error: Invalid index
│ 
│   on main.tf line 99, in data "azurerm_public_ip" "outbound_ip":
│   99:   name                = data.azurerm_resources.outbound_ip_info.resources[0].name
│     ├────────────────
│     │ data.azurerm_resources.outbound_ip_info.resources is empty list of object
│ 
│ The given key does not identify an element in this collection value.
╵

This code runs on Azure DevOps cloud agent with vmImage: ubuntu-latest. I've never had this issue running the same terraform config locally. Locally I'm using the same Terraform version, except it was built for MacOS:

terraform -v
Terraform v0.15.3
on darwin_amd64

Steps to Reproduce

I simply tun terraform apply with the required variables. Since the failure is random (almost 50/50), I don't know how to reproduce.

Important Factoids

the fact that it runs in an ADO pipeline on an Ubuntu image.

favoretti commented 3 years ago

Hi there and thank you for reporting this. From my experience with Azure this is a bug in their tags API. Databricks clusters that clean themselves up use tags as well and that fails randomly for us too. Might help if you log a support issue with MSFT.