Update 0.1.22 Azure DevOps extension is not providing the -backend-config=client_id=*** -backend-config=client_secret=***

[jrbosch]

Last automatic update of the extension in Azure DevOps is not providing the -backend-config=client_id=*** -backend-config=client_secret=*** options for the init command. Failing with the error:

Error: Error building ARM Config: obtain subscription(xxx) from Azure CLI: parsing json result from the Azure CLI: waiting for the Azure CLI: exit status 1: ERROR: Please run 'az login' to setup account.

Task configuration:

  - task: TerraformTaskV4@4
    displayName: Initialize Terraform
    inputs:
      provider: 'azurerm'
      command: 'init'
      workingDirectory: '$(System.DefaultWorkingDirectory)'
      backendServiceArm: '${{parameters.serviceConnection}}'
      backendAzureRmResourceGroupName: '$(backendAzureRmResourceGroupName)'
      backendAzureRmStorageAccountName: '$(backendAzureRmStorageAccountName)'
      backendAzureRmContainerName: '$(backendAzureRmContainerName)'
      backendAzureRmKey: '${{parameters.env}}.tfstate'

Sensitive values are replaced by xxx

Before (version 0.1.21) it was working fine! Pipeline execution log:

Starting: Initialize Terraform
==============================================================================
Task         : Terraform
Description  : Execute terraform commands to manage resources on AzureRM, Amazon Web Services(AWS) and Google Cloud Platform(GCP)
Version      : 4.218.21
Author       : Microsoft Corporation
Help         : [Learn more about this task](https://aka.ms/AAf0uqr)
==============================================================================
/opt/hostedtoolcache/terraform/1.5.5/x64/terraform init -backend-config=storage_account_name=xxx -backend-config=container_name=xxx -backend-config=key=Development.tfstate -backend-config=resource_group_name=xxx -backend-config=subscription_id=xxx -backend-config=tenant_id=xxx -backend-config=client_id=*** -backend-config=client_secret=***

Initializing the backend...

Successfully configured the backend "azurerm"! Terraform will automatically
use this backend unless the backend configuration changes.
Initializing modules...
- application_insights in modules/application_insights
- function_app_linux in modules/function_app_linux
- function_app_windows in modules/function_app_windows
- resource_group in modules/resource_group
- service_bus_namespace in modules/service_bus_namespace
- storage_account in modules/storage_account

Initializing provider plugins...
- Finding latest version of hashicorp/random...
- Finding hashicorp/azurerm versions matching ">= 3.68.0"...
- Installing hashicorp/random v3.5.1...
- Installed hashicorp/random v3.5.1 (signed by HashiCorp)
- Installing hashicorp/azurerm v3.70.0...
- Installed hashicorp/azurerm v3.70.0 (signed by HashiCorp)

Terraform has created a lock file .terraform.lock.hcl to record the provider
selections it made above. Include this file in your version control repository
so that Terraform can guarantee to make the same selections by default when
you run "terraform init" in the future.

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.
Finishing: Initialize Terraform

Now (version 0.1.22) is not working. Pipeline execution log:

Starting: Initialize Terraform
==============================================================================
Task         : Terraform
Description  : Execute terraform commands to manage resources on AzureRM, Amazon Web Services(AWS) and Google Cloud Platform(GCP)
Version      : 4.227.22
Author       : Microsoft Corporation
Help         : [Learn more about this task](https://aka.ms/AAf0uqr)
==============================================================================
/opt/hostedtoolcache/terraform/1.5.5/x64/terraform init -backend-config=storage_account_name=xxx -backend-config=container_name=xxx -backend-config=key=Development.tfstate -backend-config=resource_group_name=xxx -backend-config=subscription_id=xxx -backend-config=tenant_id=xxx

Initializing the backend...
Initializing modules...
- application_insights in modules/application_insights
- function_app_linux in modules/function_app_linux
- function_app_windows in modules/function_app_windows
- resource_group in modules/resource_group
- service_bus_namespace in modules/service_bus_namespace
- storage_account in modules/storage_account
╷
│ Error: Error building ARM Config: obtain subscription(xxx) from Azure CLI: parsing json result from the Azure CLI: waiting for the Azure CLI: exit status 1: ERROR: Please run 'az login' to setup account.
│ 
│ 
╵

##[error]Error: The process '/opt/hostedtoolcache/terraform/1.5.5/x64/terraform' failed with exit code 1
Finishing: Initialize Terraform

Both execution log headers provide the task version and you can see the second log doesn't have the options -backend-config=client_id=*** -backend-config=client_secret=*** in the command line.

I was searching but there is no way to rollback those Azure DevOps updates, even so I could fix it using TerraformTaskV3 instead of TerraformTaskV4.

[mericstam]

@jaredfholgate Can you take a look if anything is related to last release?

[jaredfholgate]

@jrbosch Please could you run the task with diagnostics turned on and look for debug output lines like this:

##[debug]813272eb-ad8d-496f-93f6-5ebea98107b8 auth scheme = ServicePrincipal
##[debug]Setting up backend for authorization scheme: serviceprincipal.

If you could reply here with those two lines that would really help. Thanks

[jaredfholgate]

Quick update. We think we have identified the issue and have made a fix will which be rolled out ASAP.

It looks like there was a change to the casing of the authorization scheme attribute some time ago, so older service connections may be serviceprincipal (lower case), but newer service connections are ServicePrincipal (pascal case). The code was not handling the old casing, so the fix will handle any casing and some other edge cases.

[jaredfholgate]

@jrbosch Just to confirm, the fix is now deployed in version 0.1.23. Please could you confirm it works in your environment.

It would still be very useful for me to see those two debug log output lines if you could share them with me. Thanks.

[jrbosch]

@jaredfholgate Awesome! Please see the two lines from my debug log and yes, it is working now

##[debug]02acb8d0-ddab-46d0-a776-e51675d66758 auth scheme = serviceprincipal
##[debug]Setting up backend for authorization scheme: serviceprincipal.

Many thanks !!

[jaredfholgate]

@jaredfholgate Jared Holgate FTE Awesome! Please see the two lines from my debug log and yes, it is working now

##[debug]02acb8d0-ddab-46d0-a776-e51675d66758 auth scheme = serviceprincipal
##[debug]Setting up backend for authorization scheme: serviceprincipal.

Many thanks !!

@jrbosch Thanks for confirming and thanks for the logs. They confirm the issue was as expected and is now resolved.