Error: unable to locate "Storage Account appears to be region based

stumueller commented 4 months ago

Is there an existing issue for this?

[X] I have searched the existing issues

Community Note

Please vote on this issue by adding a :thumbsup: reaction to the original issue to help the community and maintainers prioritize this request
Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
If you are interested in working on this issue or have submitted a pull request, please leave a comment and review the contribution guide to help.

Terraform Version

1.8.2

AzureRM Provider Version

3.101.0

Affected Resource(s)/Data Source(s)

azurerm_storage_account

Terraform Configuration Files

resource "azurerm_storage_account" "data_lake_storage" {
  name                     = join("", [var.prefix, var.env, "dlakestr"])
  resource_group_name      = azurerm_resource_group.databricks.name
  location                 = azurerm_resource_group.databricks.location
  account_tier             = var.datalake_storage_sku
  account_replication_type = var.datalake_storage_redundancy
  is_hns_enabled           = true
  tags                     = local.tags
  blob_properties {
    delete_retention_policy {
      days = 15
    }
    container_delete_retention_policy {
      days = 15
    }
  }

  network_rules {
    default_action = "Deny"
    ip_rules       = var.local == "local" ? [var.client_ip_address] : null
    virtual_network_subnet_ids = [
      var.shared_vnet_id,
      var.dbr_private_subnet_id,
      var.dbr_public_subnet_id,
      var.k8s_subnet_id
    ]
    bypass = ["AzureServices"]
  }
}

Debug Output/Panic Output

{"value":[{"sku":{"name":"Standard_LRS","tier":"Standard"},"kind":"Storage","id":"/subscriptions/74a57460-013e-4db0-850b-86c4d083c797/resourceGroups/essmats-md99-backend-databricks-rg/providers/Microsoft.Storage/storageAccounts/essmatsmd99backenddb5fb","name":"essmatsmd99backenddb5fb","type":"Microsoft.Storage/storageAccounts","location":"eastus","tags":{"BusinessUnit":"ESS-NG","CostCenter":"SDPG","Env":"Dev","Owner":"mat-devs@educationsoftwaresolutions.co.uk","Project":"essmats","Region":"UK South"},"properties":{"defaultToOAuthAuthentication":true,"keyCreationTime":{"key1":"2024-02-29T14:11:25.3854137Z","key2":"2024-02-29T14:11:25.3854137Z"},"allowCrossTenantReplication":false,"privateEndpointConnections":[],"minimumTlsVersion":"TLS1_2","allowBlobPublicAccess":false,"networkAcls":{"ipv6Rules":[],"bypass":"AzureServices","virtualNetworkRules":[],"ipRules":[],"defaultAction":"Allow"},"supportsHttpsTrafficOnly":true,"encryption":{"services":{"file":{"keyType":"Account","enabled":true,"lastEnabledTime":"2024-02-29T14:11:26.0104161Z"},"blob":{"keyType":"Account","enabled":true,"lastEnabledTime":"2024-02-29T14:11:26.0104161Z"}},"keySource":"Microsoft.Storage"},"provisioningState":"Succeeded","creationTime":"2024-02-29T14:11:25.1197853Z","primaryEndpoints":{"blob":"https://essmatsmd99backenddb5fb.blob.core.windows.net/","queue":"https://essmatsmd99backenddb5fb.queue.core.windows.net/","table":"https://essmatsmd99backenddb5fb.table.core.windows.net/","file":"https://essmatsmd99backenddb5fb.file.core.windows.net/"},"primaryLocation":"eastus","statusOfPrimary":"available"}}],"nextLink":"https://management.azure.com/subscriptions/74a57460-013e-4db0-850b-86c4d083c797/providers/Microsoft.Storage/storageAccounts?api-version=2023-01-01&%24skiptoken=redacted"}: timestamp=2024-04-30T15:38:30.166Z
2024-04-30T15:38:30.166Z [WARN]  Provider "registry.terraform.io/hashicorp/azurerm" produced an invalid plan for azurerm_monitor_metric_alert.failed_logicapp_tr_build_gold_from_silver_run, but we are tolerating it because it is using the legacy plugin SDK.
    The following problems may be the cause of any confusing errors from downstream operations:
      - .auto_mitigate: planned value cty.True for a non-computed attribute
2024-04-30T15:38:30.168Z [ERROR] provider.terraform-provider-azurerm_v3.101.0_x5: Response contains error diagnostic: diagnostic_severity=ERROR tf_req_id=33e6e96e-ef7e-b038-f3e6-4e98798e86bf tf_rpc=ReadResource diagnostic_detail="" diagnostic_summary="unable to locate \"Storage Account (Subscription: \\"74a57460-013e-4db0-850b-86c4d083c797\\"\nResource Group Name: \\"essmats-dev-backend-databricks-rg\\"\nStorage Account Name: \\"essmatsdevdlakestr\\")\"" tf_proto_version=5.4 tf_provider_addr=provider tf_resource_type=azurerm_storage_account @caller=github.com/hashicorp/terraform-plugin-go@v0.19.0/tfprotov5/internal/diag/diagnostics.go:58 @module=sdk.proto timestamp=2024-04-30T15:38:30.167Z
2024-04-30T15:38:30.168Z [ERROR] vertex "azurerm_storage_account.data_lake_storage" error: unable to locate "Storage Account (Subscription: \"74a57460-013e-4db0-850b-86c4d083c797\"\nResource Group Name: \"essmats-dev-backend-databricks-rg\"\nStorage Account Name: \"essmatsdevdlakestr\")"
2024-04-30T15:38:30.168Z [ERROR] vertex "azurerm_storage_account.data_lake_storage (expand)" error: unable to locate "Storage Account (Subscription: \"74a57460-013e-4db0-850b-86c4d083c797\"\nResource Group Name: \"essmats-dev-backend-databricks-rg\"\nStorage Account Name: \"essmatsdevdlakestr\")"
2024-04-30T15:38:30.239Z [DEBUG]

Expected Behaviour

I have a storage account called essmatsdevdlakestr, in UK South. that it should be finding

Actual Behaviour

It appears that it is only finding the one and only storage account in East US, and not the storage accounts in Uk South.

This issue appears intermittent. If we re-run the pipeline it will then find the resource correctly.

Steps to Reproduce

No response

Important Factoids

No response

References

No response

andyr8939 commented 2 months ago

We have been experiencing this same issue across several regions but mainly West Europe and Australia East.

Quick workaround seems to be to add a random tag to the storage account, seems to refresh the API cache or something and then Terraform picks it up again. Its not a fix as it happens again, but something has changed. I have around 300 pipelines running daily and probably 20% fail intermittently with this and only started the last couple of weeks.

flcdrg commented 2 months ago

Been seeing this randomly since mid to late May. Storage accounts are in Australia East. Have also raised a support ticket with Microsoft.

RemiHG commented 2 months ago

Any progress on this topic ? I have the same behavior on West Europe Storage account. Trying to create a new FileShare on an existing Storage account. The plan is completed but the apply is failing with "Unable to locate Storage Account..."

hashicorp / terraform-provider-azurerm