"Error retrieving keys for Storage Account" when using Management group scoped service principal using workload identity federation

[captainhook]

Describe the bug When I run any of the tasks such as init, it fails saying it is unable to access the storage account.

To Reproduce Steps to reproduce the behavior:

1. Setup service connection using 'workload identity federation' auth method
2. Grant Service Principal relevant access at Mgmt group level in Azure
3. main.tf:

   
   # Azure Provider source and version being used
   terraform {
   required_providers {
   azurerm = {
     source  = "hashicorp/azurerm"
     version = "=3.86.0"
   }
   }
   backend "azurerm" {
   # use_azuread_auth = true
   }
   }

Configure the Microsoft Azure Provider

provider "azurerm" { skip_provider_registration = false features { key_vault { purge_soft_delete_on_destroy = false } resource_group { prevent_deletion_if_contains_resources = true } subscription { prevent_cancellation_on_destroy = true } }

subscription_id = "34b7d951-xxxx-xxxx-xxxx-xxxxxxxxxxxx" }

provider "azurerm" { alias = "Connectivity-prod" skip_provider_registration = false features { key_vault { purge_soft_delete_on_destroy = false } resource_group { prevent_deletion_if_contains_resources = true } subscription { prevent_cancellation_on_destroy = true } }

subscription_id = azurerm_subscription.Connectivity-prod-001.subscription_id }

4. Configure release pipeline with Terraform init task and configure it to use the Management group scoped service connection.

steps:

• task: ms-devlabs.custom-terraform-tasks.custom-terraform-release-task.TerraformTaskV4@4 displayName: 'Terraform : init' inputs: workingDirectory: '$(System.DefaultWorkingDirectory)/_AzureLandingZones/azl-tf' commandOptions: '-upgrade' backendServiceArm: 'Terraform Central Mgmt' backendAzureRmResourceGroupName: 'rg-TerraformMgmt-Store-prod-001' backendAzureRmStorageAccountName: redactedterraformstorage backendAzureRmContainerName: tfstate backendAzureRmKey: terraform.tfstate

The json I wrote and imported for this release pipeline had "ensureBackend" and "backendAzureRmSubscriptionId" defined for this task but it does not appear to be showing anymore in the DevOps portal yaml viewer...

**Expected behavior**
Terraform init should find the correct existing subscription.

**Screenshots**
N/A

**Pipeline Logs**

2024-01-10T15:46:20.4009700Z ##[section]Starting: Terraform : init 2024-01-10T15:46:20.4014826Z ============================================================================== 2024-01-10T15:46:20.4014983Z Task : Terraform 2024-01-10T15:46:20.4015042Z Description : Execute terraform commands to manage resources on AzureRM, Amazon Web Services(AWS) and Google Cloud Platform(GCP) 2024-01-10T15:46:20.4015186Z Version : 4.227.24 2024-01-10T15:46:20.4015245Z Author : Microsoft Corporation 2024-01-10T15:46:20.4015331Z Help : Learn more about this task 2024-01-10T15:46:20.4015420Z ============================================================================== 2024-01-10T15:46:21.2373862Z [command]/opt/hostedtoolcache/terraform/1.6.6/x64/terraform init -upgrade -backend-config=storage_account_name=redactedterraformstorage -backend-config=container_name=tfstate -backend-config=key=terraform.tfstate -backend-config=resource_group_name=rg-TerraformMgmt-Store-prod-001 -backend-config=tenant_id=3f9427d9-xxxx-xxxx-xxxx-xxxxxxxxxxxx -backend-config=client_id= -backend-config=oidc_token= -backend-config=use_oidc=true 2024-01-10T15:46:21.2782288Z 2024-01-10T15:46:21.2785116Z Initializing the backend... 2024-01-10T15:46:21.2822122Z ╷ 2024-01-10T15:46:21.2823037Z │ Error: Failed to get existing workspaces: Error retrieving keys for Storage Account "redactedterraformstorage": storage.AccountsClient#ListKeys: Invalid input: autorest/validation: validation failed: parameter=client.SubscriptionID constraint=MinLength value="" details: value length must be greater than or equal to 1 2024-01-10T15:46:21.2823827Z │  2024-01-10T15:46:21.2824087Z │  2024-01-10T15:46:21.2824340Z ╵ 2024-01-10T15:46:21.2824574Z  2024-01-10T15:46:21.2915737Z ##[error]Error: The process '/opt/hostedtoolcache/terraform/1.6.6/x64/terraform' failed with exit code 1 2024-01-10T15:46:21.2963068Z ##[section]Finishing: Terraform : init

**Agent Configuration**
 - OS: Ubuntu lastest
 - Hosted
 - Terraform version used: 3.86.0
 - AzureCLI version used: Default

**Additional context**
When we use a subscription-scoped service connection, it appears to work fine. Once we switch to a mgmt group scoped service connection, it cannot find a subscription ID according to the error log. We have included subscription IDs in the main.tf under each provider.

[captainhook]

I think I just realised I am using the extension by Microsoft DevLabs... this is what happens when you're glued to an issue for too long without breaks.