Support for authentication through az cli with service principal

hegerdes commented 1 year ago

Is there an existing issue for this?

[X] I have searched the existing issues

Community Note

Please vote on this issue by adding a :thumbsup: reaction to the original issue to help the community and maintainers prioritize this request
Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
If you are interested in working on this issue or have submitted a pull request, please leave a comment

Description

When using a basic terraform setup with the azurerm provider:

terraform {
  required_providers {
    azurerm = {
      source  = "hashicorp/azurerm"
      version = "=3.58.0"
    }
  }
    backend "azurerm" {
        resource_group_name  = "default"
        storage_account_name = "mysa"
        container_name       = "tf-state-test"
        key                  = "k8s-tf-test.tfstate"
    }
}

you get:

Initializing the backend...

Error: Error building ARM Config: Authenticating using the Azure CLI is only supported as a User (not a Service Principal).

To authenticate to Azure using a Service Principal, you can use the separate 'Authenticate using a Service Principal' auth method - instructions for which can be found here: https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/guides/service_principal_client_secret

Alternatively you can authenticate using the Azure CLI by using a User Account.

You can not authenticate via the az cli using service principals. This is espacilly a needed feature when running terraform in the CI like GitHub Actions. This is also documented in the azurerm docs but there is no Explanation why this feature is not implemented.

This means you can NOT reuse setups like below with GitHub Actions and con NOT implement GitHubs best practices for CI security to not use static client credentials but instead use openid-connect with Federated Identities

      - name: Log in with Azure
        uses: azure/login@v1
        with:
          client-id: ${{ secrets.AZURE_CLIENT_ID }}
          tenant-id: ${{ secrets.AZURE_TENANT_ID }}
          subscription-id: ${{ secrets.AZURE_SUBSCRIPTION_ID }}

Please support login with the az cli like most other tools, since static credentials can pose a serious security threat and are a nightmare to manage in any larger setup. User accounts are also not a alternative in the CI since non interaktive auth with a proper secured account (2FA) is almost impossible and just hacky.

New or Affected Resource(s)/Data Source(s)

azurerm_3.58.0

Potential Terraform Configuration

No response

References

This might be related to #21593 but I'm not sure. Al least it shows how impotent it is to have a more flexible auth setup.

magodo commented 1 year ago

@hegerdes In order to follow CI best practice, you'd probably want to auth with a SP against OpenID Connect: https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/guides/service_principal_oidc

hegerdes commented 1 year ago

@magodo Thanks for that link. This works indeed 👍. But does require to add these vars to the provider setup:

terraform {
  required_providers {
    azurerm = {
      source  = "hashicorp/azurerm"
      version = "=3.59.0"
    }
  }
    backend "azurerm" {
        resource_group_name  = "default"
        storage_account_name = "mysa"
        container_name       = "tf-state-test"
        key                  = "k8s-tf-test.tfstate"
        use_oidc             = true
    }
}
provider "azurerm" {
  use_oidc = true
}

Which should not required and may interfere with local az setups. Is there any reason why the OIDC method over the az cli with SPs is not supported? Since I debugged a similar problem recently with cosign I found that supporting the cli auth is quiet easy. Since the azurerm provider also uses Go like cosign you might want to look into this cosign/azure/client to see their implementation

magodo commented 1 year ago

@tombuildsstuff Do you know why do we constraint az cli auth to only user accounts?

hegerdes commented 1 year ago

Any updates on this? Do you consider adding OIDC login for SPs via the AZ method?

cc: @tombuildsstuff

HTSchoenfelder commented 11 months ago

I'm a bit confused regarding the documentation. It's mentioned that "Authenticating via the Azure CLI is only supported when using a User Account", but when I run a Terraform deployment via a GitHub Workflow and authenticate using az login with a Managed Identity through Federated Identity, everything seems to work perfectly. Could someone provide clarity on this discrepancy? Why does it work in this specific scenario when the documentation suggests otherwise?

HTSchoenfelder commented 11 months ago

I'm a bit confused regarding the documentation. It's mentioned that "Authenticating via the Azure CLI is only supported when using a User Account", but when I run a Terraform deployment via a GitHub Workflow and authenticate using az login with a Managed Identity through Federated Identity, everything seems to work perfectly. Could someone provide clarity on this discrepancy? Why does it work in this specific scenario when the documentation suggests otherwise?

Apologies for the confusion in my previous comment. I need to clarify: my initial observation was based on a quick test without specifying a backend. As soon as I include the backend in my configuration, I encounter the error: "Error building ARM Config: Authenticating using the Azure CLI is only supported as a User (not a Service Principal)." This is truly frustrating, especially since the preferred approach would be to use Managed Identity with Federated Credentials.

tanerjn commented 9 months ago

As of October 6, 2023 this problem persists. As someone worked with AWS and OpenStack before, Azure's Identity Management looks very odd tbh. I've tried to check my root profile's identity service, this is what I get : Selected user account does not exist in tenant 'Microsoft Services' and cannot access the application 'blah' in that tenant. The account needs to be added as an external user in the tenant first. Please use a different account. Besides, this problem may not be Hashicorp related, az cli is not a mature product, according to microsfoties.

bcline760 commented 8 months ago

@hegerdes In order to follow CI best practice, you'd probably want to auth with a SP against OpenID Connect: https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/guides/service_principal_oidc

Aye, this'll work, but it's not terribly suitable for CI/CD platforms such as GitHub Actions. As the OP mentioned, we cannot use the reusable GH action to authenticate to Azure for service principals. Instead, people have to stuff environment variables or code the credentials/tokens into the provider block itself, that of which poses some high security risks. I'm just wondering what is so terribly different between a user account and a service principal that would negate authenticating using az cli for one. Furthermore, requiring a use_oidc parameter in the provider block feels like a hack.

hegerdes commented 8 months ago

Any updates on this? Is it on the roadmap or any other feedback regarding this?
Seems like quiet some people are intrested in this

Also why is it labeled duplicate? Is there another issue for this, can it be linked?

cc: @magodo @tombuildsstuff

gregoryca commented 6 months ago

Hi Any updates or info on this issue ? It's 3 januari 2024 and it's pretty quite while the feature is highly needed !

gregoryca commented 6 months ago

as a workaround i have added a file to my github repo which creates a federated credential linked to an github repo. I was busy restructing my infrastructure and this workaround made it so i could continue my work and deploy my pipelines. After fighting with the config it seems like it's alot easier than using a service principal.

msterin commented 4 months ago

bump. Would be nice to have SOME answer

AidanStrong commented 4 months ago

Bump again. Though, the Terraform documentation for the azurerm provider states the following:

Prior to version 3.44, authenticating via the Azure CLI was only supported when using a User Account. For example az login --service-principal was not supported and you had to use either a Client Secret or a Client Certificate. From 3.44 upwards, authenticating via the Azure CLI is supported when using a Service Principal or Managed Identity.

This documentation seems inaccurate and doesn't correspond with either my experience of the eperience of users on this thread.

hegerdes commented 4 months ago

Update Nevermind, did the same mistake as HTSchoenfelder and forgot to push my backend.tf so it used a lokal file instead of azure storage. Lost my state and now have to delete/import everything by hand.

Please fix this that the backand also can use the az login method for SPs. 🙏 Even if the necessary changes need to be done in terraform itself. I found a issue on that repo (https://github.com/hashicorp/terraform/issues/34456) dealing with the same problem. I looked into the code for the azure backend auth. It is intresting. It does not use a the standard azure go sdk insted of a own helper package that has not been updated since almost two years. I would really like this to be as near to the azure upstream interface as possible - like with the aws sdk.

Original Post:

This documentation seems inaccurate and doesn't correspond with either my experience of the eperience of users on this thread.

Thanks for mentioning that @AidanStrong

I retested my setup with Terraform v1.7.5 and azurerm version 3.97.1 and it worked. I have the following config for my provider:

provider "azurerm" {
  use_oidc = true
  features {}
}

My gh action looks like this:

name: infra

on:
  workflow_dispatch:
    inputs:
      tf-extra-args:
        type: string
        default: ""
        description: Extra args for terraform
  push:
    tags: ["v*"]

permissions:
  contents: read
  id-token: write

jobs:
  k8s-infra:
    runs-on: ubuntu-latest
    environment: default
    steps:
      - name: Checkout repository
        uses: actions/checkout@v4

      - name: Set up Terraform
        uses: hashicorp/setup-terraform@v3

      - name: Log in with Azure
        uses: azure/login@v2
        with:
          client-id: ${{ secrets.AZURE_CLIENT_ID }}
          tenant-id: ${{ secrets.AZURE_TENANT_ID }}
          subscription-id: ${{ secrets.AZURE_SUBSCRIPTION_ID }}

      - name: Run Terraform
        working-directory: terraform
        run: |
          terraform --version
          terraform init
          terraform plan ${{ inputs.tf-extra-args || '' }} -out plan.infra
          terraform apply -auto-approve plan.infra

Azure is configured to allow requests from that repo on the main branch using federated credentials.

It is extremely frustrating that HashiCorp/Azure never really commented on this issue but for me it seems to be fixed even though they only put a note on the docs as mentioned by @AidanStrong.

Can others confirm my experience?

bcline760 commented 4 months ago

Yea, if I use OIDC as well, it works using a service principal. Though, Microsoft's implementation of OIDC in Azure is lacking, though, that's for another thread.

On Sat, Mar 23, 2024 at 5:33 AM Henrik Gerdes @.***> wrote:

This documentation seems inaccurate and doesn't correspond with either my experience of the eperience of users on this thread.

Thanks for mentioning that @AidanStrong https://github.com/AidanStrong

I retested my setup with Terraform v1.7.5 and azurerm version 3.97.1 and it worked. I have the following config for my provider:

provider "azurerm" { use_oidc = true features {} }

My gh action looks like this:

name: infra on: workflow_dispatch: inputs: tf-extra-args: type: string default: "" description: Extra args for terraform push: tags: ["v*"] permissions: contents: read id-token: write jobs: k8s-infra: runs-on: ubuntu-latest environment: default steps:

name: Checkout repository uses: @.***

name: Set up Terraform uses: @.***

name: Log in with Azure uses: @.*** with: client-id: ${{ secrets.AZURE_CLIENT_ID }} tenant-id: ${{ secrets.AZURE_TENANT_ID }} subscription-id: ${{ secrets.AZURE_SUBSCRIPTION_ID }}

name: Run Terraform working-directory: terraform run: | terraform --version terraform init terraform plan ${{ inputs.tf-extra-args || '' }} -out plan.infra terraform apply -auto-approve plan.infra

Azure is configured to allow requests from that repo on the main branch using federated credentials.

It is extremely frustrating that HashiCorp/Azure never really commented on this issue but for me it seems to be fixed even though they only put a note on the docs as mentioned by @AidanStrong https://github.com/AidanStrong.

Can others confirm my experience?

— Reply to this email directly, view it on GitHub https://github.com/hashicorp/terraform-provider-azurerm/issues/22034#issuecomment-2016480742, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEVVV3WND736E7YQL75ER2LYZVZCRAVCNFSM6AAAAAAY2EM2H2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMJWGQ4DANZUGI . You are receiving this because you commented.Message ID: @.***>

alexkunde commented 3 months ago

For me it wasnt working until additionally adding ARM_USE_OIDC: true even though i added the provider feature.

jobs:
  terraform:
    runs-on: ubuntu-latest
    env:
      ARM_SUBSCRIPTION_ID: ${{ secrets.AZURE_SUBSCRIPTION_ID }}
      ARM_CLIENT_ID: ${{ secrets.AZURE_CLIENT_ID }}
      ARM_TENANT_ID: ${{ secrets.AZURE_TENANT_ID }}
      ARM_USE_OIDC: true
    steps:
      - uses: actions/checkout@v4
      - uses: hashicorp/setup-terraform@v3
        with:
          terraform_version: 1.6.5
      - name: Azure login
        uses: azure/login@v2.0.0
        with:
          client-id: ${{ secrets.AZURE_CLIENT_ID }}
          tenant-id: ${{ secrets.AZURE_TENANT_ID }}
          subscription-id: ${{ secrets.AZURE_SUBSCRIPTION_ID }}
      - name: Terraform fmt
        id: fmt
        run: terraform fmt -check
      - name: Terraform Init
        id: init
        run: terraform init -lockfile=readonly

ms-henglu commented 1 month ago

Hi all,

The azurerm provider and azure backend use different library/codes to authenticate with Azure. And this error comes from the azure backend.

Here's how to reproduce on a local machine:

Use the azure backend like the following:


terraform {
backend "azurerm" {
resource_group_name  = "henglubackend"  # Can be passed via `-backend-config=`"resource_group_name=<resource group name>"` in the `init` command.
storage_account_name = "henglubackend"                      # Can be passed via `-backend-config=`"storage_account_name=<storage account name>"` in the `init` command.
container_name       = "henglubackend"                       # Can be passed via `-backend-config=`"container_name=<container name>"` in the `init` command.
key                  = "prod.terraform.tfstate"        # Can be passed via `-backend-config=`"key=<blob key name>"` in the `init` command.
subscription_id = "your subscription id"
}
}

data "azurerm_resource_group" "example" { name = "henglubackend" }

2. Use `az login --service-principal -u <client-id> -p <client-secret> --tenant <tenant-id>` to login.
3. Run `terraform plan`, and will encounter the following error:

│ Error: Error building ARM Config: Authenticating using the Azure CLI is only supported as a User (not a Service Principal). │ │ To authenticate to Azure using a Service Principal, you can use the separate 'Authenticate using a Service Principal' │ auth method - instructions for which can be found here: https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/guides/service_principal_client_secret │ │ Alternatively you can authenticate using the Azure CLI by using a User Account.



Workaround
The azure backend supports auth with service principal: https://developer.hashicorp.com/terraform/language/settings/backends/azurerm#backend-azure-ad-service-principal-via-client-secret

Root cause:
The `github.com/hashicorp/go-azure-helpers` that azure backend uses doesn't support authenticating with az login with service principal. But this issue was already fixed in the later version.

So I opened this PR to fix it in the `terraform` repo: https://github.com/hashicorp/terraform/pull/35381 which updates the `github.com/hashicorp/go-azure-helpers` to `v0.69.0`.

bpedman commented 6 days ago

@alexkunde has the best workaround, you just need to set the 4 environment variables:

ARM_SUBSCRIPTION_ID
ARM_CLIENT_ID
ARM_TENANT_ID
ARM_USE_OIDC

This works for both the backend and the provider. Also note that you do NOT need to run the azure/login action since the terraform provider and backend will authenticate themselves with OIDC from the environment variables.

This also allows users to work from their local environment to run a terraform plan since the use_oidc argument is not set in the provider configuration.

Updated example:

jobs:
  terraform:
    runs-on: ubuntu-latest
    env:
      ARM_SUBSCRIPTION_ID: ${{ secrets.AZURE_SUBSCRIPTION_ID }}
      ARM_CLIENT_ID: ${{ secrets.AZURE_CLIENT_ID }}
      ARM_TENANT_ID: ${{ secrets.AZURE_TENANT_ID }}
      ARM_USE_OIDC: true
    steps:
      - uses: actions/checkout@v4
      - uses: hashicorp/setup-terraform@v3
        with:
          terraform_version: 1.6.5
      - name: Terraform Init
        id: init
        run: terraform init
      - name: Terraform Plan
        id: plan
        run: terraform plan

And the terraform provider is just a simple:

provider "azurerm" {
  features {}
}

Which works locally and in the github action

hashicorp / terraform-provider-azurerm