User Managed Identity connection didn't work into azure devops with AzureCLI@2

diegolagospagopa commented 10 months ago

Community Note

Please vote on this issue by adding a :thumbsup: reaction to the original issue to help the community and maintainers prioritize this request
Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
If you are interested in working on this issue or have submitted a pull request, please leave a comment and review the contribution guide to help.

Terraform Version

1.6.6

AzureRM Provider Version

3.85.0

Affected Resource(s)/Data Source(s)

backend_state

Terraform Configuration Files

- task: AzureCLI@2
    displayName: "🔦 Plan Terraform"
    inputs:
      connectedServiceNameARM: '${{ parameters.AZURE_SERVICE_CONNECTION_NAME }}'
      addSpnToEnvironment: true
      scriptType: 'bash'
      scriptLocation: 'inlineScript'
      failOnStandardError: true
      workingDirectory: '${{ parameters.WORKINGDIR }}'
      inlineScript: |
        echo "##[section] 💈 Start terraform plan for tf-env-folder=${{ parameters.TF_ENVIRONMENT_FOLDER }}"

        export cliendId=$(az ad sp show --id ${servicePrincipalId} --query "appId" -o tsv) 
        echo "servicePrincipalId: ${servicePrincipalId}"
        echo "cliendId: ${cliendId}"

        export ARM_USE_MSI=true
        export ARM_CLIENT_ID=${cliendId}
        export ARM_SUBSCRIPTION_ID=$(az account show --query id --output tsv)
        export ARM_TENANT_ID=$(az account show --query tenantId --output tsv)

        generate_state_output=""

        if [[ "${{ parameters.TF_SUMMARIZE }}" == "True" ]]; then
          echo "[info] Added option to generate tfplan"
          generate_state_output="-out=tfplan"
        else
          echo "📢 Summary not enabled: ${{ parameters.TF_SUMMARIZE }}"
        fi

        if [[ "${{ parameters.AKS_NAME }}" != "" ]]; then
          echo "[INFO] 🚀 Run terraform plan + kubernetes"
          ./terraform.sh plan ${{ parameters.TF_ENVIRONMENT_FOLDER }} -var k8s_kube_config_path_prefix="$(pwd)" -lock-timeout=300s -lock=false ${generate_state_output}
        else
          echo "[INFO] 🚀 Run terraform plan"
          ./terraform.sh plan ${{ parameters.TF_ENVIRONMENT_FOLDER }} -lock-timeout=300s -lock=false ${generate_state_output}
        fi

`

terraform.sh do:
1. az subscription set
2. terraform init
3. terraform apply

Debug Output/Panic Output

Expected Behaviour

The terraform is able to load the backend configuration, and continue with terraform init

Actual Behaviour

the terraform is unable to load the backend config, and can't start terraform init

Error: Failed to get existing workspaces: Error retrieving keys for Storage Account "pagopainfraterraformdev": azure.BearerAuthorizer#WithAuthorization: Failed to refresh the Token for request to https://management.azure.com/subscriptions/xyz/resourceGroups/my-infra-rg/providers/Microsoft.Storage/storageAccounts/pagopainfraterraformdev/listKeys?api-version=2021-01-01: StatusCode=400 -- Original Error: adal: Refresh request failed. Status Code = '400'. Response body: {"error":"invalid_request","error_description":"Identity not found"} Endpoint http://169.254.169.254/metadata/identity/oauth2/token?api-version=2018-02-01&resource=https%3A%2F%2Fmanagement.azure.com%2F

Error: Backend initialization required, please run "terraform init"

Reason: Initial configuration of the requested backend "azurerm"

The "backend" is the interface that Terraform uses to store state, perform operations, etc. If this message is showing up, it means that the Terraform configuration you're using is using a custom configuration for the Terraform backend.

Changes to backend configurations require reinitialization. This allows Terraform to set up the new configuration, copy existing state, etc. Please run "terraform init" with either the "-reconfigure" or "-migrate-state" flags to use the current configuration.

If the change reason above is incorrect, please verify your configuration hasn't changed and try again. At this point, no changes to your existing configuration or state have been made.

Steps to Reproduce

user managed identity:

create a managed identity with Contributor permission on the subscription

terraform:

setup the backend into a storage account

setup the env variables like this:

export ARM_USE_MSI=true
export ARM_CLIENT_ID=${cliendId}
export ARM_SUBSCRIPTION_ID=$(az account show --query id --output tsv)
export ARM_TENANT_ID=$(az account show --query tenantId --output tsv)

that allow to load the managed identity

run terraform init

Important Factoids

No response

References

The project iac can be found here:

https://github.com/pagopa/pagopa-azure-devops/tree/iac-core-migration/azure-devops/iac

rcskosir commented 10 months ago

Thanks for taking the time to open this issue. Issues related to the Azure backend live in the hashicorp/terraform repo, so I am going to move this issue there.

diegolagospagopa commented 9 months ago

Azurerm service connections with federated managed identity manual create a user managed identity that federates with azdo via oidc (Configure an identity managed by an external OpenID Connect Provider to access Azure resources as this application)

Currently therefore you need to use this code snippet to allow the connection correctly

  - task: AzureCLI@2
    displayName: "🔦 Plan Terraform"
    inputs:
      connectedServiceNameARM: '${{ parameters.AZURE_SERVICE_CONNECTION_NAME }}'
      addSpnToEnvironment: true
      scriptType: 'bash'
      scriptLocation: 'inlineScript'
      failOnStandardError: true
      workingDirectory: '${{ parameters.WORKINGDIR }}'
      inlineScript: |
        echo "##[section] 💈 Start terraform plan for tf-env-folder=${{ parameters.TF_ENVIRONMENT_FOLDER }}"

        export cliendId=$(az ad sp show --id ${servicePrincipalId} --query "appId" -o tsv)

        echo "[INFO] Terraform setup variables"
        export ARM_USE_OIDC=true
        export ARM_OIDC_TOKEN=${idToken}
        export ARM_CLIENT_ID=${cliendId}
        export ARM_SUBSCRIPTION_ID=$(az account show --query id --output tsv)
        export ARM_TENANT_ID=$(az account show --query tenantId --output tsv)

        generate_state_output=""

        if [[ "${{ parameters.TF_SUMMARIZE }}" == "True" ]]; then
          echo "[info] Added option to generate tfplan"
          generate_state_output="-out=tfplan"
        else
          echo "📢 Summary not enabled: ${{ parameters.TF_SUMMARIZE }}"
        fi

        if [[ "${{ parameters.AKS_NAME }}" != "" ]]; then
          echo "[INFO] 🚀 Run terraform plan + kubernetes"
          ./terraform.sh plan ${{ parameters.TF_ENVIRONMENT_FOLDER }} -var k8s_kube_config_path_prefix="$(pwd)" -lock-timeout=300s -lock=false ${generate_state_output}
        else
          echo "[INFO] 🚀 Run terraform plan"
          ./terraform.sh plan ${{ parameters.TF_ENVIRONMENT_FOLDER }} -lock-timeout=300s -lock=false ${generate_state_output}
        fi

hashicorp / terraform