Same issue with latest terraform provider. Reference issue closed without resolution Error reading queue properties for AzureRM Storage Account #3997

nirengithub commented 1 year ago

Is there an existing issue for this?

[X] I have searched the existing issues

Community Note

Please vote on this issue by adding a :thumbsup: reaction to the original issue to help the community and maintainers prioritize this request
Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform Version

1.3.7

AzureRM Provider Version

3.39.1

Affected Resource(s)/Data Source(s)

azurerm_storage_account

Terraform Configuration Files

Trying to create a synapse workspace with datalake storage as per documentation. This was working previously, but then started giving error , tried the resolution given in other forums and links but no luck.

 Error: reading queue properties for AzureRM Storage Account "powepjlbgour": queues.Client#GetServiceProperties: Failure responding to request: StatusCode=403 -- Original Error: autorest/azure: Service returned an error. 
Status=403 Code="AuthenticationFailed" Message="Server failed to authenticate the request. Make sure the value 
of Authorization header is formed correctly including the signature.\nRequestId:f75358b5-f003-0016-7881-53652d000000\nTime:2023-03-10T18:51:53.1331606Z"
│
│   with module.synapse-workspace.azurerm_storage_account.synapse_storage_account,
│   on module-synapse-workspace\synapse-ws\synapse-workspace.tf line 3, in resource "azurerm_storage_account" "synapse_storage_account":
│    3: resource "azurerm_storage_account" "synapse_storage_account" {

1. Tried with updating the version
2. Tried adding queue properties

Reference suggestion tried 

https://github.com/hashicorp/terraform-provider-azurerm/pull/16557

https://github.com/hashicorp/terraform-provider-azurerm/pull/4099

For debugging sharing the template I am using :

resource "azurerm_storage_account" "synapse_storage_account" {
  name                = var.synapse_storage_account
  location            = var.location
  resource_group_name = var.resource_group_name

  account_tier             = "Standard"
  account_replication_type = "LRS"
  account_kind             = "StorageV2"
  is_hns_enabled           = "true"
  tags = var.taggs

}

resource "azurerm_storage_data_lake_gen2_filesystem" "synapse_filesystem" {
  name               = "synapse-filesystem"
  storage_account_id = azurerm_storage_account.synapse_storage_account.id

    depends_on = [

              azurerm_storage_account.synapse_storage_account
    ]

}

resource "azurerm_synapse_workspace" "synapse_workspace" {
  name                = var.synapse_workspace_name
  location            = var.location
  resource_group_name = var.resource_group_name
  storage_data_lake_gen2_filesystem_id = azurerm_storage_data_lake_gen2_filesystem.synapse_filesystem.id
  sql_administrator_login              = "sqladminuser"
  sql_administrator_login_password     = "H@Sh1CoR3!"

  aad_admin {
    login     = "AzureAD Admin"
    object_id = "5dfc07db-f24d-44cd-9c86-78ac0c55c80a"
    tenant_id = "8c4850c4-d96e-409c-a09e-3446a7629232"
  }

  identity {
    type = "SystemAssigned"
  }

  tags = var.taggs

  depends_on = [

              azurerm_storage_account.synapse_storage_account
    ]

}

Debug Output/Panic Output

Error: reading queue properties for AzureRM Storage Account "powepjlbgour": queues.Client#GetServiceProperties: Failure responding to request: StatusCode=403 -- Original Error: autorest/azure: Service returned an error. 
Status=403 Code="AuthenticationFailed" Message="Server failed to authenticate the request. Make sure the value 
of Authorization header is formed correctly including the signature.\nRequestId:f75358b5-f003-0016-7881-53652d000000\nTime:2023-03-10T18:51:53.1331606Z"
│
│   with module.synapse-workspace.azurerm_storage_account.synapse_storage_account,
│   on module-synapse-workspace\synapse-ws\synapse-workspace.tf line 3, in resource "azurerm_storage_account" "synapse_storage_account":
│    3: resource "azurerm_storage_account" "synapse_storage_account" {

Expected Behaviour

It should create the storage account and the file system, it does create the store account only with no filesystem but the message shows error creating storage account.

Actual Behaviour

it creates the store account only with no filesystem and the message shows error creating storage account.

Steps to Reproduce

terraform apply

Important Factoids

NA

References

https://github.com/hashicorp/terraform-provider-azurerm/pull/4099 https://github.com/hashicorp/terraform-provider-azurerm/pull/16557 https://github.com/hashicorp/terraform-provider-azurerm/issues/4081

magodo commented 1 year ago

@nirengithub Is this systematic happening? Since the error occurs when reacing to the queue data plane API, it might be due to some service side issue. Alternatively, you can try setting storage_use_azuread = true.

SteveDevOps commented 1 year ago

This is happening to me as well, intermittently (not every time) on plans and applies. Continues to happen with the above provider setting @magodo recommends. I my case it's post creation phase, the queues exist, and intermittent when interrogating metadata surrounding the queues. During the plan/apply run we communicate to the storage account privately through its private endpoint.
Error: retrieving Queue "xxx" (Account "xxx-queue"): queues.Client#GetMetaData: Failure responding to request: StatusCode=403 -- Original Error: autorest/azure: Service returned an error. Status=403 Code="AuthorizationFailure" Message="This request is not authorized to perform this operation.\nRequestId:7b9f3ff5-b003-0081-0f37-fed107000000\nTime:2023-10-14T00:42:11.0594659Z"

srivastavaprashant commented 7 months ago

Downgrading to 3.97.1 didn't help.

│ Error: retrieving queue properties for Storage Account (Subscription: "xxx" │ Resource Group Name: "xxx" │ Storage Account Name: "openaistdev"): unmarshalling response: could not parse response body │ │ with module.storage_account.azurerm_storage_account.storage_account, │ on ../../modules/storage_account/main.tf line 13, in resource "azurerm_storage_account" "storage_account": │ 13: resource "azurerm_storage_account" "storage_account" {

michasacuer commented 6 months ago

Same error with private Storage Account

Kiaren-Kaizen commented 3 months ago

Same issue- using v3.115.0 using self-hosted agents (traffic is okay) tried a myiad of things.

setting storage_use_azuread ( when this is true i get issues with the below, when false - i get issues with the same but blob )
Adding Data plane perms to the spn
http only bool tried on and off -Storage account creates fine, theres nothing inside by design

[ERROR] provider.terraform-provider-azurerm_v3.115.0_x5: GET https://MASKED.queue.core.windows.net/?comp=properties&restype=service request failed: Get "https://MASKED.queue.core.windows.net/?comp=properties&restype=service": net/http: TLS handshake timeout: timestamp=2024-08-23

mpjtaylor commented 1 month ago

Also stuck on this issue with queues, storage_use_azuread does not help

hashicorp / terraform-provider-azurerm