RyanChen22 commented 1 week ago

Is there an existing issue for this?

[X] I have searched the existing issues

Community Note

Please vote on this issue by adding a :thumbsup: reaction to the original issue to help the community and maintainers prioritize this request
Please do not leave comments along the lines of "+1", "me too" or "any updates", they generate extra noise for issue followers and do not help prioritize the request
If you are interested in working on this issue or have submitted a pull request, please leave a comment and review the contribution guide to help.

Terraform Version

1.9.7

AzureRM Provider Version

4.0.1

Affected Resource(s)/Data Source(s)

Containers of Azure Storage account

Terraform Configuration Files

required_providers{
 azurerm = {
  source = "hasicorp/azurerm"
  version = "4.0.1"

}

}

Debug Output/Panic Output

024-10-10T06:40:54.883Z [ERROR] provider.terraform-provider-azurerm_v4.0.1_x5: Response contains error diagnostic: diagnostic_severity=ERROR
  diagnostic_summary=
  | retrieving static website properties for Storage Account (Subscription: "xxx"
  | Resource Group Name: "xx"
  | Storage Account Name: "xx"): executing request: Get "https://xx.blob.core.windows.net/?comp=properties&restype=service": context deadline exceeded

Expected Behaviour

the containers should be created successfully after provisioning the storage account

Actual Behaviour

storage account was provisioned successfully but containers were failed to be created with the error in the debug log above. It keeps retrieving static website properties which was not used at all in the client terraform.

We have also tested successfully with similar terraform whereby if the provisioned storage account has public access enabled without PE + network acl, both account and containers are successfully provisioned without issues.

Steps to Reproduce

use azurerm version "4.0.1" in the provider file
have 2 modules - one to provisioning storage account with PE + network acl
the other module to provisioning containers after account was created successfully.

Important Factoids

N.A.

References

Found similar Github issue but failed to succeed it by using azapi: https://github.com/hashicorp/terraform-provider-azurerm/issues/2977

mybayern1974 commented 1 week ago

@RyanChen22 thank you for filing this issue. Till now I'm aware there are two main types of Storage resource provisioning issues in TF:

Storage Account (SA) could be provisioned successfully but its sub resources failed say SA Container, as reported by #2977
SA could NOT be provisioned successfully due to failure in provisioning its sub resources, say static website., as reported by #20257, #27310, #24982.

Firstly. I suggest do not regard the above two types as the same problem. Treat them separately. I speculate your issue belong to #[2]. You may not agree by providing justification "but my SA gets provisioned successfully", given there is no full log provided here, I just speculate that success was only from observation from say Azure portal but TF shows SA provisioned failed? Please correct me if my speculation is wrong.

[2] is well aware by Hashicorp and Microsoft and we are actively looking into it. Meanwhile I'd like to suggest you can subscribe to #27310 and close this one as a duplicate, if you are fine with doing that.

Cc @WodansSon , @jackofallops , @magodo

RyanChen22 commented 1 week ago

@mybayern1974 - thanks for your response. The situation is not falling under point 2. Cos even with PE+network ACL, storage account could be provisioning successfully without issues. The concern is on the creation of container after account was created. If you look at the shared debug error, it seems always retrieving static website properties before attempting to create the container. However, static website was not used at all in the terraform file.

Also, please do not close the thread as the this has been affecting users and required an immediate solution asap.

mybayern1974 commented 1 week ago

@RyanChen22 sure no problem to keep this issue open. Could you please provide a minimum reproducible TF config describing your SA, SA Container, and PE?

RyanChen22 commented 1 week ago

sa&blob_storage_modules.zip @mybayern1974 - I am attaching the used files for your review.

mybayern1974 commented 1 week ago

@RyanChen22, in the current AzureRM implementation, it's SA that calls static website rather than the Storage Container, ending up in case any issues happen during accessing the static website in SA, the following provisioning to Storage Container would be impacted. Based on that, I think I can understand your saying that SA has already been fully successfully provisioned, then one possibility could be when executing terraform plan / apply to your terraform solution, accessing SA would always happen, which has no direct relationship with managing Storage Container, and accessing SA would trigger the access to static website.

With that, as I mentioned above, investigation on either decoupling the static website from SA or be tolerant to issues happening during accessing the static website is actively ongoing by owners of this project.

RyanChen22 commented 1 week ago

@mybayern1974 - Thanks for the reply. Tolerating the issue is definitely not acceptable as the container is not being provisioned after running the TF files since the error of static website will be thrown. Is there any solution or workaround on this? Cos if PE and network ACL is not configured in the TF, the provisioning of both account and containers are working successfully, which conflicts with your saying that accessing SA will trigger the access to static website.

mybayern1974 commented 1 week ago

@RyanChen22 I would suggest to stay tuned to the final solution that has not been finalized yet, so it's out of my current knowledge scope now to share insights what that solution would be like. Re your two questions:

Re "....the container is not being provisioned...since the error of static website will be thrown..." that is what I mean to internally tolerant the access-static-website error thus the follow up actions say provisioning the container could continue. Again this is only one solution option, which is still under evaluation.
Re "conflict with saying that..." that accessing SA would trigger access static website is an independent operation from setting PE + ACL or just using public network. A relation between the two operations could be setting PE sometimes (again I skip explaining in detail what "sometimes" means here, instead, stay tuned to the final solution) would block accessing static website but using public network won't block that.

RyanChen22 commented 1 week ago

@mybayern1974 - Appreciate your response and information so far. I would check with customers to see if the error could be ignored /skipped when deploying those TF files through pipeline. Meanwhile, will be happy to know if there is any ETA on the final solution. Thanks.

mybayern1974 commented 1 week ago

@mybayern1974 - Appreciate your response and information so far. I would check with customers to see if the error could be ignored /skipped when deploying those TF files through pipeline. Meanwhile, will be happy to know if there is any ETA on the final solution. Thanks.

Re "check with customers...error could be ignored...", I believe not. Customers are innocent and they are unable to swallow this error reported here at their layer (unless they use Terraform AzAPI to manage the SA), instead, the duty is at this provider.

RyanChen22 commented 1 week ago

In such case, when could we expect to have a solution on this?

mybayern1974 commented 1 week ago

In such case, when could we expect to have a solution on this?

Sorry this is out of my current information scope, instead, what I can tell is Hashicorp and Microsoft is actively looking into this issue.

RyanChen22 commented 1 week ago

@mybayern1974 - Appreciate your response and information so far. I would check with customers to see if the error could be ignored /skipped when deploying those TF files through pipeline. Meanwhile, will be happy to know if there is any ETA on the final solution. Thanks.

Re "check with customers...error could be ignored...", I believe not. Customers are innocent and they are unable to swallow this error reported here at their layer (unless they use Terraform AzAPI to manage the SA), instead, the duty is at this provider.

To speed up the process, customer has also tried AzAPI as well but encountering the below exception. Would we have some guidance on how to address the issue?

Error: Failed to query available provider packages │ │ Could not retrieve the list of available versions for provider hashicorp/azapi: provider registry registry.terraform.io does not have │ a provider named registry.terraform.io/hashicorp/azapi │ │ Did you intend to use azure/azapi? If so, you must specify that source address in each module which requires that provider. To see
│ which modules are currently depending on hashicorp/azapi, run the following command: │ terraform providers

mybayern1974 commented 1 week ago

@mybayern1974 - Appreciate your response and information so far. I would check with customers to see if the error could be ignored /skipped when deploying those TF files through pipeline. Meanwhile, will be happy to know if there is any ETA on the final solution. Thanks.

Re "check with customers...error could be ignored...", I believe not. Customers are innocent and they are unable to swallow this error reported here at their layer (unless they use Terraform AzAPI to manage the SA), instead, the duty is at this provider.

To speed up the process, customer has also tried AzAPI as well but encountering the below exception. Would we have some guidance on how to address the issue?

Error: Failed to query available provider packages │ │ Could not retrieve the list of available versions for provider hashicorp/azapi: provider registry registry.terraform.io does not have │ a provider named registry.terraform.io/hashicorp/azapi │ │ Did you intend to use azure/azapi? If so, you must specify that source address in each module which requires that provider. To see │ which modules are currently depending on hashicorp/azapi, run the following command: │ terraform providers

This looks like an AzAPI usage question. To keep this thread focused, please file the AzAPI question here to have discussion there.

RyanChen22 commented 5 days ago

@mybayern1974 - I have helped my customers to raise another GH link for azapi workaround. Will appreciate if you could help to further advice the mitigation. Thanks

hashicorp / terraform-provider-azurerm

Failed to create containers when provisioned azure storage account has configured PE and network acl #27607

Is there an existing issue for this?

Community Note

Terraform Version

AzureRM Provider Version

Affected Resource(s)/Data Source(s)

Terraform Configuration Files

Debug Output/Panic Output

Expected Behaviour

Actual Behaviour

Steps to Reproduce

Important Factoids

References

[2] is well aware by Hashicorp and Microsoft and we are actively looking into it. Meanwhile I'd like to suggest you can subscribe to #27310 and close this one as a duplicate, if you are fine with doing that.