Deleting encrypted managed disk fails when attached to VM with encrypted OS disk

[mark-strasser-nnl]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform (and AzureRM Provider) Version

$ terraform -version Terraform v0.14.5

• provider registry.terraform.io/hashicorp/azurerm v2.44.0
• provider registry.terraform.io/hashicorp/local v2.0.0
• provider registry.terraform.io/hashicorp/random v3.0.1
• provider registry.terraform.io/hashicorp/tls v3.0.0

Affected Resource(s)

• azurerm_linux_virtual_machine
• azurerm_disk_encryption_set
• azurerm_managed_disk

Terraform Configuration Files

variable "key_vault_rg_name" {
  default = "rg-external-infra-test"
}

variable "key_vault_name" {
  default = "kv-external-infra-test"
}

variable "key_vault_key_name" {
  default = "key-external-infra"
}

variable "azure_location" {
  default = "USGovVirginia"
}

variable "prefix" {
  default = "example"
}

provider "azurerm" {
  environment = "usgovernment"
  features {}
}

data "azurerm_key_vault" "main" {
  name                = var.key_vault_name
  resource_group_name = var.key_vault_rg_name
}

data "azurerm_key_vault_key" "generated" {
  name         = var.key_vault_key_name
  key_vault_id = data.azurerm_key_vault.main.id
}

resource "azurerm_resource_group" "main" {
  name     = "rg-${var.prefix}-${terraform.workspace}"
  location = var.azure_location
}

resource "azurerm_disk_encryption_set" "main" {
  name                = "des-ghes-${terraform.workspace}"
  resource_group_name = azurerm_resource_group.main.name
  location            = azurerm_resource_group.main.location
  key_vault_key_id    = data.azurerm_key_vault_key.generated.id

  identity {
    type = "SystemAssigned"
  }
}

resource "azurerm_key_vault_access_policy" "disk" {
  key_vault_id = data.azurerm_key_vault.main.id
  tenant_id    = azurerm_disk_encryption_set.main.identity.0.tenant_id
  object_id    = azurerm_disk_encryption_set.main.identity.0.principal_id

  key_permissions = [
    "get",
    "unwrapKey",
    "wrapKey",
    "decrypt",
    "encrypt",
    "sign",
    "verify",
  ]
}

resource "azurerm_virtual_network" "vnet" {
  name                = "vnet-${var.prefix}-${terraform.workspace}"
  location            = azurerm_resource_group.main.location
  resource_group_name = azurerm_resource_group.main.name
  address_space       = ["10.0.0.0/16"]
}

resource "azurerm_subnet" "internal" {
  name                 = "snet-${var.prefix}-${terraform.workspace}"
  resource_group_name  = azurerm_resource_group.main.name
  virtual_network_name = azurerm_virtual_network.vnet.name
  address_prefixes     = ["10.0.1.0/24"]
}

resource "azurerm_public_ip" "pip" {
  name                = "pip-${var.prefix}-${terraform.workspace}"
  resource_group_name = azurerm_resource_group.main.name
  location            = azurerm_resource_group.main.location
  allocation_method   = "Dynamic"
  domain_name_label   = "${var.prefix}-${terraform.workspace}"
}

resource "azurerm_network_interface" "main" {
  name                = "nic-${var.prefix}-${terraform.workspace}"
  resource_group_name = azurerm_resource_group.main.name
  location            = azurerm_resource_group.main.location

  ip_configuration {
    name                          = "primary"
    subnet_id                     = azurerm_subnet.internal.id
    private_ip_address_allocation = "Dynamic"
    public_ip_address_id          = azurerm_public_ip.pip.id
  }
}

resource "tls_private_key" "example_ssh" {
  algorithm = "RSA"
  rsa_bits  = 4096
}

resource "azurerm_linux_virtual_machine" "main" {
  name                = "vm-${var.prefix}-${terraform.workspace}"
  computer_name       = "vm-${var.prefix}-${terraform.workspace}"
  resource_group_name = azurerm_resource_group.main.name
  location            = azurerm_resource_group.main.location
  size                = "Standard_D4s_v3"

  admin_username                  = "adminuser"
  disable_password_authentication = true

  network_interface_ids = [
    azurerm_network_interface.main.id
  ]

  admin_ssh_key {
    username   = "adminuser"
    public_key = tls_private_key.example_ssh.public_key_openssh
  }

  os_disk {
    caching              = "ReadWrite"
    storage_account_type = "StandardSSD_LRS"
    disk_encryption_set_id = azurerm_disk_encryption_set.main.id  # comment out to see "fixed" version
  }

  source_image_reference {
    publisher = "Canonical"
    offer     = "UbuntuServer"
    sku       = "18.04-LTS"
    version   = "latest"
  }
}

resource "azurerm_managed_disk" "data" {
  name                   = "disk-${azurerm_linux_virtual_machine.main.name}"
  location               = azurerm_resource_group.main.location
  resource_group_name    = azurerm_resource_group.main.name
  storage_account_type   = "StandardSSD_LRS"
  create_option          = "Empty"
  disk_size_gb           = 50
  disk_encryption_set_id = azurerm_disk_encryption_set.main.id
}

resource "azurerm_virtual_machine_data_disk_attachment" "data_attach" {
  managed_disk_id    = azurerm_managed_disk.data.id
  virtual_machine_id = azurerm_linux_virtual_machine.main.id
  lun                = "1"
  caching            = "ReadWrite"
}

Debug Output

https://gist.github.com/mark-strasser-nnl/deaaf45e2b9afee68ff888389f14a6e3

Panic Output

Expected Behaviour

• terraform destroy should have detached then deleted the managed disk properly

Actual Behaviour

• terraform destroy reported success when detaching the managed disk from the VM
• Then reported an error when deleting the managed disk

Error: Error deleting Managed Disk "disk-vm-example-default" (Resource Group "rg-example-default"): compute.DisksClient#Delete: Failure sending request: StatusCode=0 -- Original Error: autorest/azure: Service returned an error. Status=<nil> Code="OperationNotAllowed" Message="Disk disk-vm-example-default is attached
to VM /subscriptions/<id>/resourceGroups/rg-example-default/providers/Microsoft.Compute/virtualMachines/vm-example-default."

Steps to Reproduce

1. Create a resource group, key vault, and key, and specify them as variable names
2. terraform apply
3. terraform destroy

Important Factoids

Running in Azure Government (US).

References

None

[johnvanhienen]

Experiencing the same issue as described above. Disabling the os disk disk_encryption_set_id "resolves" the issue, but that's obviously not a solution. Any updates on this bug?

[MarkKharitonov]

This also happens when destroying an azurerm_kubernetes_cluster resource. We do not even create VMs manually, but we do set the disk_encryption_set_id property:

resource "azurerm_kubernetes_cluster" "aks" {
  name                            = var.name
  resource_group_name             = var.resource_group_name
  location                        = var.location
  disk_encryption_set_id          = var.cmk_enabled ? azurerm_disk_encryption_set.aks.id : null
  ...
}

The exact same behaviour is observed.