microsoft / terraform-provider-azuredevops

Terraform Azure DevOps provider
https://www.terraform.io/docs/providers/azuredevops/
MIT License
372 stars 260 forks source link

Sourcing from AZDO_PERSONAL_ACCESS_TOKEN environment does not work anymore for microsoft/azuredevops v1.0.0 #990

Closed sebDK closed 3 months ago

sebDK commented 3 months ago

AZDO_PERSONAL_ACCESS_TOKEN is delivered as environment variable but I get the following error from today:

│ Error: Invalid provider configuration │ │ Provider "registry.terraform.io/microsoft/azuredevops" requires explicit │ configuration. Add a provider block to the root module and configure the │ provider's required arguments as described in the provider documentation. │ ╵ ╷ │ Error: Invalid combination of arguments │ │ with provider["registry.terraform.io/microsoft/azuredevops"], │ on line 0: │ (source code not available) │ │ "use_oidc": only one of │ client_certificate,client_certificate_path,client_secret,client_secret_path,personal_access_token,use_msi,use_oidc │ can be specified, but client_secret,personal_access_token were specified. ╵ ╷ │ Error: Invalid combination of arguments │ │ with provider["registry.terraform.io/microsoft/azuredevops"], │ on line 0: │ (source code not available) │ │ "client_secret": only one of │ client_certificate,client_certificate_path,client_secret,client_secret_path,personal_access_token,use_msi,use_oidc │ can be specified, but client_secret,personal_access_token were specified. ╵ ╷ │ Error: Invalid combination of arguments │ │ with provider["registry.terraform.io/microsoft/azuredevops"], │ on line 0: │ (source code not available) │ │ "client_secret_path": only one of │ client_certificate,client_certificate_path,client_secret,client_secret_path,personal_access_token,use_msi,use_oidc │ can be specified, but client_secret,personal_access_token were specified. ╵ ╷ │ Error: Invalid combination of arguments │ │ with provider["registry.terraform.io/microsoft/azuredevops"], │ on line 0: │ (source code not available) │ │ "use_msi": only one of │ client_certificate,client_certificate_path,client_secret,client_secret_path,personal_access_token,use_msi,use_oidc │ can be specified, but client_secret,personal_access_token were specified.

basti304 commented 3 months ago

As fas as I can tell this issue occurs when you export client secret and PAT both as an environment variable. We are having the same issue because we need to export both in our build pipeline, i.e. export ARM_CLIENT_SECRET="$(secret)" export AZDO_PERSONAL_ACCESS_TOKEN=$(pat) results in the issue above.

xjhofernandes commented 3 months ago

Same here... In my case, I had to go back to provider version 0.11.0.

AlexanderRijnbeek commented 3 months ago

We also had to pin the provider back to version 0.11.0, waiting to be fixed or waiting till our platform team allows service principals in Azure DevOps

jonbamber commented 3 months ago

This broke all my pipelines that use OIDC for Azure resources and PAT for Azure DevOps resources.

It appears to be because of a deliberate choice to pick up authentication variables from the environment using ARM_ prefixed variables already used by the azurerm provider. I'm not sure why this was done rather than requiring AZDO_ prefixed ones.

Have rolled back to 0.11.0 for now.

nyanhp commented 3 months ago

Very questionable choice. I would have at least like to use use_oidc = false to keep my pipelines running with a combination of OIDC for the azapi provider and PAT for this one.

But even this is not possible:

Error: Missing required argument
│
│   with provider["registry.terraform.io/microsoft/azuredevops"],
│   on provider.tf line 61, in provider "azuredevops":
│   61:   use_oidc = false
│
│ "use_oidc": all of `client_id,tenant_id,use_oidc` must be specified
Ihar-Ratner commented 3 months ago

The same like for others, it broke all my pipelines, reverted to previous version.

drdamour commented 3 months ago

same here, we use the azure backend using service principal auth as defined at bottom of https://developer.hashicorp.com/terraform/language/settings/backends/azurerm

client_secret - (Optional) The Client Secret of the Service Principal. This can also be sourced from the ARM_CLIENT_SECRET environment variable.

so we set ARM_CLIENT_SECRET environment variable...and that's making the new release think we want to use service principal auth to AzDO...but we don't...we want to use PAT for AzDO. the provider shouldn't be so opinionated about environment variables being set IMO and use_oidc = false should be enough to tell it to ignore those environment vairables related to oidc IMO.

pinning to older version, hoping for fix soon

lahiruperamune commented 3 months ago

Same happened and all the pipelines were broken. Looking for a fix soon.

jgroves71 commented 3 months ago

Also broken when using AZDO_PERSONAL_ACCESS_TOKEN in conjunction with Workload Identity Federation for azure service connection / service principal. I don't have any ARM environment variables set.

ChrisSidebotham commented 3 months ago

+1 - Version 0.11 worked fine

jubr commented 3 months ago

I added a https://github.com/microsoft/terraform-provider-azuredevops/pull/747#discussion_r1515814519 in another thread, will paste the (slightly edited) content here so people don't need to go look:


@davidcorrigan714 @jaredfholgate @xuzhang3 whatever happened to not using ARM_ vars? This is now breaking pipelines that have (a subset) of those vars set for the other azure providers, but have no relevance for azuredevops.

I understand the — let's make this easy for people so it "just works" — sentiment, but in practice this will "just break" things for people in an unsuspecting way if they (or their pipeline) use a combination of the azure{rm,ad,devops} providers. Which will not be an empty groups me thinks. $0.02 given.

Thinking through on this, how about when this (for some people unintended) OIDC authentication is attempted and it fails, the provider shows a hint in how this new behavior can be disabled. Perhaps with an elegant way like for instance ignore_arm_env_vars = true. #OpenForExtensionClosedForModification

Pinning to version = "~> 0.11" until this is resolved or until we have time to refactor our authentication configuration.

mateustanaka commented 3 months ago

The same issue here. We have rolled back to 0.11.0 for now.

davidcorrigan714 commented 3 months ago

You all seem to be more familiar with the other Azure providers than I am, it seems like this would be a problem for them too. The problem being multiple Azure connections with some using OIDC and some not. I think my intention was for oidc_token = false (as @nyanhp tried) to be the escape hatch there and I botched that. Seems like it'd solve the problem though right?

davidcorrigan714 commented 3 months ago

The other way out would seem to be to not use the environment variables for the other providers. The azurerm provider at least seems to support them as either environment variables or parameters on the provider block.

davidcorrigan714 commented 3 months ago

@jgroves71 Got the specific error message you're seeing?

jgroves71 commented 3 months ago

@davidcorrigan714 Hi David, this was the error:

client_certificate,client_certificate_path,client_secret,client_secret_path,personal_access_token,use_msi,use_oidc │ can be specified, but personal_access_token,use_oidc were specified. ╵ ╷ │ Error: Invalid combination of arguments │ │ with provider["registry.terraform.io/microsoft/azuredevops"], │ on line 0: │ (source code not available) │ │ "client_secret": only one of │ client_certificate,client_certificate_path,client_secret,client_secret_path,personal_access_token,use_msi,use_oidc │ can be specified, but personal_access_token,use_oidc were specified. ╵ ╷ │ Error: Invalid combination of arguments │ │ with provider["registry.terraform.io/microsoft/azuredevops"], │ on line 0: │ (source code not available) │ │ "use_oidc": only one of │ client_certificate,client_certificate_path,client_secret,client_secret_path,personal_access_token,use_msi,use_oidc │ can be specified, but personal_access_token,use_oidc were specified. ╵ ╷ │ Error: Invalid combination of arguments │ │ with provider["registry.terraform.io/microsoft/azuredevops"], │ on line 0: │ (source code not available) │ │ "client_secret_path": only one of │ client_certificate,client_certificate_path,client_secret,client_secret_path,personal_access_token,use_msi,use_oidc │ can be specified, but personal_access_token,use_oidc were specified. ╵ #

davidcorrigan714 commented 3 months ago

Rereading the code it really looks like the validation for the inputs ended up being over-ambitious and overly restrictive and just needs to be wiped and a line added to the docs about the variable precedence. The code that actually fetches the token looks like it'll handle everything fine and has reasonable error messages for most things already. I'll put up a PR in a bit.