Terraform Init is not compatible with Azure ARM Federated Identity service connection.

[futojin]

We have used a recommended service connection: Azure Resource Manager using Workload Identity federation with OpenID Connect (automatic)

However the Terraform Init seems having an issue connecting to azurerm storage backend:

2023-10-26T01:46:51.9604087Z ##[section]Starting: Terraform init
2023-10-26T01:46:51.9613132Z ==============================================================================
2023-10-26T01:46:51.9613529Z Task         : Terraform
2023-10-26T01:46:51.9614015Z Description  : Execute terraform commands to manage resources on AzureRM, Amazon Web Services(AWS) and Google Cloud Platform(GCP)
2023-10-26T01:46:51.9614504Z Version      : 4.227.24
2023-10-26T01:46:51.9614781Z Author       : Microsoft Corporation
2023-10-26T01:46:51.9615155Z Help         : [Learn more about this task](https://aka.ms/AAf0uqr)
2023-10-26T01:46:51.9615593Z ==============================================================================
2023-10-26T01:46:56.3786653Z [command]/azp/_work/_tool/terraform/1.2.9/x64/terraform init -backend-config=storage_account_name=its***ops -backend-config=container_name=devops -backend-config=key=tfstate -backend-config=resource_group_name=its***rg -backend-config=subscription_id=2d0***43e -backend-config=tenant_id=e3c***73c -backend-config=client_id=*** -backend-config=oidc_token=*** -backend-config=use_oidc=true
2023-10-26T01:46:56.4365717Z 2023-10-26T01:46:56.435Z [INFO]  Terraform version: 1.2.9
2023-10-26T01:46:56.4371822Z 2023-10-26T01:46:56.436Z [DEBUG] using github.com/hashicorp/go-tfe v1.0.0
2023-10-26T01:46:56.4375564Z 2023-10-26T01:46:56.437Z [DEBUG] using github.com/hashicorp/hcl/v2 v2.12.0
2023-10-26T01:46:56.4381161Z 2023-10-26T01:46:56.437Z [DEBUG] using github.com/hashicorp/terraform-config-inspect v0.0.0-20210209133302-4fd17a0faac2
2023-10-26T01:46:56.4384770Z 2023-10-26T01:46:56.438Z [DEBUG] using github.com/hashicorp/terraform-svchost v0.0.0-20200729002733-f050f53b9734
2023-10-26T01:46:56.4393160Z 2023-10-26T01:46:56.438Z [DEBUG] using github.com/zclconf/go-cty v1.11.0
2023-10-26T01:46:56.4405235Z 2023-10-26T01:46:56.439Z [INFO]  Go runtime version: go1.18.1
2023-10-26T01:46:56.4420941Z 2023-10-26T01:46:56.440Z [INFO]  CLI args: []string{"/azp/_work/_tool/terraform/1.2.9/x64/terraform", "init", "-backend-config=storage_account_name=its***ops", "-backend-config=container_name=devops", "-backend-config=key=tfstate", "-backend-config=resource_group_name=its***rg", "-backend-config=subscription_id=2d0***43e", "-backend-config=tenant_id=e3c***73c", "-backend-config=client_id=***", "-backend-config=oidc_token=***", "-backend-config=use_oidc=true"}
2023-10-26T01:46:56.4423536Z 2023-10-26T01:46:56.440Z [DEBUG] Attempting to open CLI config file: /root/.terraformrc
2023-10-26T01:46:56.4424493Z 2023-10-26T01:46:56.440Z [DEBUG] File doesn't exist, but doesn't need to. Ignoring.
2023-10-26T01:46:56.4433618Z 2023-10-26T01:46:56.442Z [DEBUG] ignoring non-existing provider search directory terraform.d/plugins
2023-10-26T01:46:56.4434720Z 2023-10-26T01:46:56.442Z [DEBUG] ignoring non-existing provider search directory /root/.terraform.d/plugins
2023-10-26T01:46:56.4436793Z 2023-10-26T01:46:56.442Z [DEBUG] ignoring non-existing provider search directory /root/.local/share/terraform/plugins
2023-10-26T01:46:56.4437921Z 2023-10-26T01:46:56.442Z [DEBUG] ignoring non-existing provider search directory /usr/local/share/terraform/plugins
2023-10-26T01:46:56.4439369Z 2023-10-26T01:46:56.442Z [DEBUG] ignoring non-existing provider search directory /usr/share/terraform/plugins
2023-10-26T01:46:56.4486816Z 2023-10-26T01:46:56.444Z [INFO]  CLI command args: []string{"init", "-backend-config=storage_account_name=its***ops", "-backend-config=container_name=devops", "-backend-config=key=tfstate", "-backend-config=resource_group_name=its***rg", "-backend-config=subscription_id=2d0***43e", "-backend-config=tenant_id=e3c***73c", "-backend-config=client_id=***", "-backend-config=oidc_token=***", "-backend-config=use_oidc=true"}
2023-10-26T01:46:56.4489162Z 2023-10-26T01:46:56.445Z [DEBUG] Module installer: begin itsi-clients-core
2023-10-26T01:46:56.4492095Z 2023-10-26T01:46:56.446Z [DEBUG] Module installer: itsi-clients-core installed at ../../../modules/itsi-clients-core
2023-10-26T01:46:56.4493044Z Initializing modules...
2023-10-26T01:46:56.4494827Z - itsi-clients-core in ../../../modules/itsi-clients-core
2023-10-26T01:46:56.4510750Z 
2023-10-26T01:46:56.4511655Z Initializing the backend...
2023-10-26T01:46:56.4512239Z ╷
2023-10-26T01:46:56.4512957Z │ Error: Invalid backend configuration argument
2023-10-26T01:46:56.4513586Z │ 
2023-10-26T01:46:56.4514374Z │ The backend configuration argument "oidc_token" given on the command line
2023-10-26T01:46:56.4515173Z │ is not expected for the selected backend type.
2023-10-26T01:46:56.4515739Z ╵
2023-10-26T01:46:56.4516201Z 
2023-10-26T01:46:56.4728996Z ##[error]Error: The process '/azp/_work/_tool/terraform/1.2.9/x64/terraform' failed with exit code 1
2023-10-26T01:46:56.4747687Z ##[section]Finishing: Terraform init

[mericstam]

Hi, thanks for reporting. could you please provide the Azure DevOps pipeline script ( YAML) or screenshots of you build steps if you use classic pipelines .

Br, Manuel

[futojin]

Thanks for getting back to me.

Here's the some YAML snippets extracted from the web browser context:

steps:
- task: ms-devlabs.custom-terraform-tasks.custom-terraform-installer-task.TerraformInstaller@0
  displayName: 'Install Terraform 1.2.9'
  inputs:
    terraformVersion: 1.2.9

steps:
- task: ms-devlabs.custom-terraform-tasks.custom-terraform-release-task.TerraformTaskV4@4
  displayName: 'Terraform init'
  inputs:
    workingDirectory: '$(System.DefaultWorkingDirectory)/environments/$(prefix)/$(project_name)'
    backendServiceArm: '$(devops_azure_service)'
    backendAzureRmResourceGroupName: '$(devops_resouce_group)'
    backendAzureRmStorageAccountName: '$(devops_storage_account)'
    backendAzureRmContainerName: '$(devops_container)'
    backendAzureRmKey: '$(tfstate_key)'

Note that the pipelines works when we are using Service principal (manual) service connection.

[mericstam]

Is this something you can take a look at, @jaredfholgate ?

[jaredfholgate]

Hi @futojin. I think you need to use a newer version of the Terraform CLI. 1.2.9 is quite old and may not have the OIDC support. I think it was added around 1.3.4. Also check the version of the azurerm provider you are targeting supports OIDC auth.

[mericstam]

@futojin , did upgrading to a later version resolve the issue?

[futojin]

@mericstam @jaredfholgate Thank you for the hint. I've upgraded the CLI to latest at the time of writing (1.6.3) and terraform init and other steps are working perfectly, with OIDC auth performed successfully.

Many thanks again for the help and closing this issue.