hashicorp / terraform

Terraform enables you to safely and predictably create, change, and improve infrastructure. It is a source-available tool that codifies APIs into declarative configuration files that can be shared amongst team members, treated as code, edited, reviewed, and versioned.
https://www.terraform.io
Other
43.15k stars 9.58k forks source link

Terraform azurerm backend not listing all workspaces #35703

Open ljalonen opened 2 months ago

ljalonen commented 2 months ago

Terraform Version

Terraform v1.9.5
on linux_amd64

Terraform Configuration Files

terraform {
  backend "azurerm" {
    resource_group_name  = "xxx"
    storage_account_name = "yyy"
    container_name       = "zzz"
    key                  = "ddd"
  }
  required_providers {
    azurerm = {
      source  = "hashicorp/azurerm"
      version = "=4.1.0"
    }
  }
}

Debug Output

Debugs shows that terraform receives a XML response from Azure Storage which contains states for listed workspaces. There is NextMarker in response which indicates that there should be followup request happening to Azure Storage but Terraform does not request next page which has the most of the states. This can be verified with trace debugging where there is no second request generated and only workspaces from page 1 get listed.

Sanitized version of XML returned.

<?xml version="1.0" encoding="utf-8"?>
<EnumerationResults ServiceEndpoint="https://xxx.blob.core.windows.net/"
  ContainerName="yyy">
  <Prefix>aaa</Prefix>
  <Blobs>
    <Blob>
      <Name>bbb</Name>
      <Properties>
        ...
      </Properties>
    </Blob>
    <Blob>
      <Name>ccc</Name>
      <Properties>
        ...
      </Properties>
    </Blob>
    <Blob>
      <Name>ddd</Name>
      <Properties>
        ...
      </Properties>
    </Blob>
    ...
  </Blobs>
  <NextMarker>example-Zu8p6ve2dXYV8wMEGzFsgWSlRSSzh0v1-1-23-a!</NextMarker>
</EnumerationResults>

Expected Behavior

All the workspaces from all pages get listed and terraform follows NextMarker until there is not one defined in response (last page).

Actual Behavior

Only states listed in page 1 get listed by terraform and NextMarker provided in response is not followed.

Steps to Reproduce

  1. Requires condition from Azure storage where blob listing spans multiple response pages - this could happen due to partitioning etc. on Azure side.
  2. terraform init
  3. terrafowm workspace list / select XXX

Additional Context

No response

References

crw commented 2 months ago

Thanks for filing this issue! Azure backend issues are triaged by the Azure Provider team at HashiCorp, and they have been notified. Thanks again!