VenelinMartinov
commented
2 months ago Thanks for the report! This is a bit unexpected, could you please add your ESC environment here? (deleting any secrets there)
Open stooj opened 2 months ago
VenelinMartinov
commented
2 months ago Thanks for the report! This is a bit unexpected, could you please add your ESC environment here? (deleting any secrets there)
stooj
commented
2 months ago Sure. It's actually two environments:
pulumi-support:
imports:
- pulumi-support-oidc-gcp
values:
pulumiConfig:
gcp:project: pulumi-ce-teampulumi-support-oidc-gcp:
values:
gcp:
login:
fn::open::gcp-login:
project: 438338752289
oidc:
workloadPoolId: pulumi-oidc-identity-pool-stooj
providerId: pulumi-oidc-provider-stooj
serviceAccount: pulumi-oidc-service-acct-stooj@pulumi-ce-team.iam.gserviceaccount.com
environmentVariables:
GOOGLE_PROJECT: ${gcp.login.project}
CLOUDSDK_AUTH_ACCESS_TOKEN: ${gcp.login.accessToken}
GOOGLE_OAUTH_ACCESS_TOKEN: ${gcp.login.accessToken}
GOOGLE_REGION: europe-central2Don't think there are any visible secrets in there to remove, so that's the whole env.
VenelinMartinov
commented
2 months ago Thanks for the info, we'll take a look - it's quite unexpected that ESC vs manually specified env vars make a difference.
Are you sure there isn't some ambient login taking effect when you don't use ESC? Perhaps your gcloud CLI login? You can log that out with gcloud auth logout and gcloud auth application-default logout
Or are there maybe other env vars?
Could you check that the program fails in the manual case if you don't specify the token as an env var? Trying to make sure the issue is indeed specific to ESC or maybe it is a general issue with the GCP provider.
stooj
commented
2 months ago Running the repo again to make sure there are no local logins:
mkdir region-failure && cd region-failure
pulumi new gcp-python -y
pulumi config set gcp:project pulumi-ce-team
gcloud auth revoke # Returns "ERROR: (gcloud.auth.revoke) Invalid value for [accounts]: No credentials available to revoke."
gcloud auth application-default revoke # Returns "Application Default Credentials have not been set up, nothing to revoke."
pulumi config env add pulumi-support
pulumi previewI can confirm that I'm definitely not logged in locally, and the issue is still there.
Other env vars - I had a look through env and didn't see anything likely. I am setting PULUMI_HOME to something other than the default so I don't clutter my main pulumi workspace.
Did you have any other env vars in mind? The env vars declared in the esc are not available to my shell, only inside pulumi when it's running, and I've checked that none of them are set externally (env | grep GOOGLE_OAUTH_ACCESS_TOKEN etc).
Could you check that the program fails in the manual case if you don't specify the token as an env var? Trying to make sure the issue is indeed specific to ESC or maybe it is a general issue with the GCP provider.
Is there a way to use OIDC in an ESC env without exposing them as env vars? Sorry, don't know how to do this :disappointed:
If it helps with the reproduction, I used this code to generate the OIDC credentials and create the ESC env.
VenelinMartinov
commented
2 months ago Very much appreciated! Thanks for taking the time to look into this.
Is there a way to use OIDC in an ESC env without exposing them as env vars?
No, this is certainly what happens and is correct. I was mostly trying to understand how could the behaviour under ESC be different given that the only thing ESC does is change the environment.
I think what's happening is that when you run with ESC the provider gets the GOOGLE_REGION environment variable - this is actually what determines if the region check should happen.
When running manually, because the provider doesn't have a region configured it never does the region check, so never prints the warning.
Given this I am fairly sure the issue is NOT specific to ESC but is a general GCP provider issue when a region is configured and auth is given via GOOGLE_OAUTH_ACCESS_TOKEN.
I hope you don't mind if I edit the issue title to reflect this and add a short note to the description!
Describe what happened
When running with GCP and OIDC configured in an environment, the provider is unable to retrieve a regions list. Fortunately it's just a warning rather than an error, so it's not blocking anything.
The warning goes away if I set
PULUMI_GCP_SKIP_REGION_VALIDATION=true.This is not quite the same as #1641, which has been fixed. This only happens when the OIDC credentials are loaded via a pulumi environment.
EDIT: this happens without ESC too, as long as a region is configured for the provider.
Sample program
Running
pulumi previewwith "local" OIDC ~works fine~ (this only works fine if no region is configured as the region check will be skipped):Running
pulumi previewwith esc-loaded env throws the warning:Output of
pulumi aboutContributing
Vote on this issue by adding a 👍 reaction. To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).