Error: Unable to determine the Resource ID for the Key Vault at URL

dkirrane commented 7 months ago

Is there an existing issue for this?

[X] I have searched the existing issues

Community Note

Please vote on this issue by adding a :thumbsup: reaction to the original issue to help the community and maintainers prioritize this request
Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
If you are interested in working on this issue or have submitted a pull request, please leave a comment and review the contribution guide to help.

Terraform Version

1.6.5

AzureRM Provider Version

3.96.0

Affected Resource(s)/Data Source(s)

azurerm_key_vault_secret

Terraform Configuration Files

# Just a bunch of secrets

terraform {
  required_version = "1.6.5"

  required_providers {
    azurerm = {
      source  = "hashicorp/azurerm"
      version = "3.96.0"
    }
    azuread = {
      source  = "hashicorp/azuread"
      version = "2.47.0"
    }
  }
}

resource "azurerm_key_vault_secret" "my_secret" {
  name         = "my-secret"
  value        = data.aiven_service_component.kafka_sasl.host
  key_vault_id = var.key_vault_id
  tags         = {}
}

Debug Output/Panic Output

# Terraform destroy fails with and bunch of these errors when destroying secrets in the KV.

Error: Unable to determine the Resource ID for the Key Vault at URL "https://test-1234-kv.vault.azure.net/"

Expected Behaviour

destroy completes successfully

Actual Behaviour

destroy fails with

Error: Unable to determine the Resource ID for the Key Vault at URL "https://test-1234-kv.vault.azure.net/"

Steps to Reproduce

terraform destroy -auto-approve -input=false -no-color -lock=false

Important Factoids

No response

References

This ticket is closed but it looks like it's still a problem with latest azurerm provider https://github.com/hashicorp/terraform-provider-azurerm/issues/19322

wuxu92 commented 7 months ago

Hi @dkirrane , Thanks for rasing this issue, may I ask how many key vault instances exists under the same subscription? as I cannot reproduce the issue in my subs.

dkirrane commented 7 months ago

@wuxu92 there are only 3 in the Sub I'm using. The Key Vault has 25 secrets and all still present after destroy fails

tombuildsstuff commented 5 months ago

hey @dkirrane

Thanks for opening this issue.

Digging into this one I've discovered there's a caching issue within the Azure API where incomplete/stale data can be returned from the Key Vault List API that we're using - more details can be found in https://github.com/hashicorp/terraform-provider-azurerm/pull/26070 - which I believe will fix this issue.

You should be able to validate if this issue is affecting your subscription by using the Azure CLI - first by running:

az rest --method get --url "https://management.azure.com/subscriptions/XXX/providers/Microsoft.KeyVault/vaults?api-version=2023-07-01"

Which will return a list of Key Vaults using the Key Vault List API (which I suspect in your case will return ~3/25, per your comment above) - and whilst technically this endpoint is paginated, from what I'm seeing the pagination doesn't surface anymore data.

By contrast - if you run either az keyvault list --resource-type vault (which'll cross page boundaries) - or:

az rest --method get --url "https://management.azure.com/subscriptions/XXX/resources?api-version=2015-11-01&\$filter=resourceType eq 'Microsoft.KeyVault/vaults'"

(note: I'm escaping $ there)

Which will return the list of Key Vaults via the Resources endpoint the Azure CLI is using - which I suspect will be 25 (per above) - or at least the first page, when hitting the API directly, rather than using the az command.

Would you be able to confirm if that's the case in your subscription? If so, I believe that #26070 will fix this - if not could you provide some (sanitised) output of what you're seeing from those APIs?

Thanks!

github-actions[bot] commented 4 months ago

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

hashicorp / terraform-provider-azurerm