search

aws / aws-cdk

The AWS Cloud Development Kit is a framework for defining cloud infrastructure in code
https://aws.amazon.com/cdk
Apache License 2.0
11.36k stars 3.77k forks source link

cli: unable to resolve AWS account to use with CLI with SSO #24744

Open astef opened 1 year ago

astef commented 1 year ago

Describe the bug

Approximately in 25% of npx cdk deploy --all --require-approval never runs, I'm getting this error:

~/rep/myproject/cdk> npx cdk deploy --all --require-approval never
Bundling asset MyData/keyGenerator/Code/Stage...
Bundling asset MyApi/statusFunction/Code/Stage...
Bundling asset MyApi/importFunction/Code/Stage...
Bundling asset MyApi/getReportFunction/Code/Stage...
Bundling asset MyApi/getMetricsFunction/Code/Stage...
[WARNING] aws-cdk-lib.aws_apigateway.RestApiProps#minimumCompressionSize is deprecated.
  - superseded by `minCompressionSize`
  This API will be removed in the next major release.

✨  Synthesis time: 7.83s

MyData: building assets...

MyApi: building assets...

 ❌ Building assets failed: Error: Building Assets Failed: Error: Unable to resolve AWS account to use. It must be either configured when you define your CDK Stack, or through the environment, Error: Unable to resolve AWS account to use. It must be either configured when you define your CDK Stack, or through the environment
    at buildAllStackAssets (/Users/me/rep/myproject/cdk/node_modules/.pnpm/aws-cdk@2.69.0/node_modules/aws-cdk/lib/index.js:374:115279)
    at async CdkToolkit.deploy (/Users/me/rep/myproject/cdk/node_modules/.pnpm/aws-cdk@2.69.0/node_modules/aws-cdk/lib/index.js:374:143496)
    at async exec4 (/Users/me/rep/myproject/cdk/node_modules/.pnpm/aws-cdk@2.69.0/node_modules/aws-cdk/lib/index.js:429:51795)

Building Assets Failed: Error: Unable to resolve AWS account to use. It must be either configured when you define your CDK Stack, or through the environment, Error: Unable to resolve AWS account to use. It must be either configured when you define your CDK Stack, or through the environment

I've started to experience this after changing my env to the following:

~/.aws/config

[profile sandbox]
sso_session = sandbox
sso_account_id = 111111111111111
sso_role_name = AdministratorAccess
region = us-east-1
[sso-session sandbox]
sso_start_url = https://example.com/start
sso_region = eu-central-1
sso_registration_scopes = sso:account:access
[profile staging]
sso_session = staging
sso_account_id = 222222222222
sso_role_name = AdministratorAccess
region = us-east-1
[sso-session staging]
sso_start_url = https://example.com/start
sso_region = eu-central-1
sso_registration_scopes = sso:account:access

~/.aws/credentials - empty file

~/.zshrc

export AWS_DEFAULT_PROFILE=sandbox

This setup works perfectly for me, because I can switch profiles with only AWS_DEFAULT_PROFILE env variable. AWS SDK understands it and works perfectly.

For whatever reason, CDK tool sometimes fails in the middle of the deployment. After repeating the same command immediately, it succeeds.

Expected Behavior

Not to fail.

Current Behavior

Failure (sometimes)

Reproduction Steps

Didn't do, but I'm seeing this in different totally independent projects on same machine.

Possible Solution

No response

Additional Information/Context

No response

CDK CLI Version

2.69.0 (build 60a5b2a)

Framework Version

No response

Node.js Version

v16.19.1

OS

macOS Ventura 13.2.1

Language

Typescript

Language Version

TypeScript (4.9.5)

Other information

No response

pahud commented 1 year ago

Thank you for your report. Yes I occasionally have this issue as well but I can't find the root cause. I am making it p1 bug here and we probably need more feedback before we can identify the root cause.

pahud commented 1 year ago

Hi

Does it work by running:

AWS_PROFILE=sandbox cdk synth

If it does not, can you share your log with -vv ?

For example: 

AWS_PROFILE=sandbox cdk synth -vv | head -n 10

You should see logs like this

[19:43:25] Determining if we're on an EC2 instance.
[19:43:25] Does not look like an EC2 instance.
[19:43:25] Toolkit stack: CDKToolkit
[19:43:25] Setting "CDK_DEFAULT_REGION" environment variable to us-east-1
[19:43:25] Resolving default credentials
[19:43:25] Reading cached notices from /Users/<deducted>/.cdk/cache/notices.json
[19:43:25] Looking up default account ID from STS
[19:43:26] [AWS sts 200 1.019s 0 retries] getCallerIdentity({})
[19:43:26] Default account ID: <deducted>
[19:43:26] Setting "CDK_DEFAULT_ACCOUNT" environment variable to <deducted>
[19:43:26] context: {
mfittko commented 11 months ago

Having the same issue on like 1/5th of all cdk deploys and it's driving me nuts. Re-run usually works though.

astef commented 11 months ago

I have the same symptom as mfittko mentioned. ~20% of time, and autoheals after rerun.

Error text:

 ❌ Deployment failed: Error: Unable to resolve AWS account to use. It must be either configured when you define your CDK Stack, or through the environment
    at SdkProvider.resolveEnvironment (/Users/astef/proj/cdk/node_modules/.pnpm/aws-cdk@2.81.0/node_modules/aws-cdk/lib/index.js:310:47706)
    at async Deployments.prepareSdkFor (/Users/astef/proj/cdk/node_modules/.pnpm/aws-cdk@2.81.0/node_modules/aws-cdk/lib/index.js:400:7608)
    at async Deployments.isSingleAssetPublished (/Users/astef/proj/cdk/node_modules/.pnpm/aws-cdk@2.81.0/node_modules/aws-cdk/lib/index.js:400:11801)
    at async WorkGraph.removeUnnecessaryAssets (/Users/astef/proj/cdk/node_modules/.pnpm/aws-cdk@2.81.0/node_modules/aws-cdk/lib/index.js:400:137364)
    at async CdkToolkit.removePublishedAssets (/Users/astef/proj/cdk/node_modules/.pnpm/aws-cdk@2.81.0/node_modules/aws-cdk/lib/index.js:402:5081)
    at async CdkToolkit.deploy (/Users/astef/proj/cdk/node_modules/.pnpm/aws-cdk@2.81.0/node_modules/aws-cdk/lib/index.js:400:152337)
    at async exec4 (/Users/astef/proj/cdk/node_modules/.pnpm/aws-cdk@2.81.0/node_modules/aws-cdk/lib/index.js:455:51970)

The environment is the same, as I posted, but CDK version is now 2.79.0 (build e4a2c01)

I think it was ok for at least one version between 2.69.0 and 2.79.0

SaravEasy commented 9 months ago

I have been trying to resolve this for a few hours now and almost tried everything said/suggested in different Github issues and Tickets.

  1. Combined the .aws/config and .aws/credentials file
  2. Matched the profile name between both files
  3. Removed the default profile entirely from both files
  4. added profile keyword into the cdk.json
  5. Reinstalled aws-cdk
  6. tried directly running cdk bootstrap ACCOUNTNO/REGION
  7. Tried setting NODE_TLS_REJECT_UNAUTHORIZED=0.

I don't remember spending this much time just for authentication so far in any of the tools.

awscli version - aws-cli/2.13.3 Python/3.11.4 Darwin/22.1.0 exe/x86_64 prompt/off cdk npm version - 2.96.2 (build 3edd240)

CDK was supposed to help me automate things and reduce my load but now am spending lot of time in setting up the CDK itself 👎 😒

m6a-UdS commented 9 months ago

I just wanted to add that in my case, it looks like it has something to do with the browser: When it happens (that my SSO session expired, but randomly seem not to be able to re-login), copy-pasting the URL in another browser and completing the login cycle there seems to solve the problem.

jwholme2 commented 9 months ago

I just wanted to add that in my case, it looks like it has something to do with the browser: When it happens (that my SSO session expired, but randomly seem not to be able to re-login), copy-pasting the URL in another browser and completing the login cycle there seems to solve the problem.

I have the same results. My SSO session has expired, so I login via the browser again, restart vs code, and everything seems to work.

object-Object commented 8 months ago

One possible workaround is to add a script or command to the build property in cdk.json which raises an error if the CDK_DEFAULT_ACCOUNT environment variable is not set. This doesn't fix the issue, but it causes the synth/deploy to fail much earlier than it otherwise would - I've found it very helpful.

calid commented 7 months ago

I also get the "Unable to resolve AWS account to use" error when my sso session has expired. I am able to fix by running aws sso login.

$ cdk deploy -v --profile foo
...
 ❌ Deployment failed: Error: Unable to resolve AWS account to use. It must be either configured when you define your CDK Stack, or through the environment
    at SdkProvider.resolveEnvironment (/home/dmc/.nvm/versions/node/v18.18.0/lib/node_modules/aws-cdk/lib/index.js:384:14622)
    at async Deployments.prepareSdkFor (/home/dmc/.nvm/versions/node/v18.18.0/lib/node_modules/aws-cdk/lib/index.js:424:7691)
    at async Deployments.isSingleAssetPublished (/home/dmc/.nvm/versions/node/v18.18.0/lib/node_modules/aws-cdk/lib/index.js:424:11602)
    at async /home/dmc/.nvm/versions/node/v18.18.0/lib/node_modules/aws-cdk/lib/index.js:424:165621
[11:20:33] Reading cached notices from /home/dmc/.cdk/cache/notices.json

Unable to resolve AWS account to use. It must be either configured when you define your CDK Stack, or through the environment
[11:20:33] Error: Unable to resolve AWS account to use. It must be either configured when you define your CDK Stack, or through the environment
    at SdkProvider.resolveEnvironment (/home/dmc/.nvm/versions/node/v18.18.0/lib/node_modules/aws-cdk/lib/index.js:384:14622)
    at async Deployments.prepareSdkFor (/home/dmc/.nvm/versions/node/v18.18.0/lib/node_modules/aws-cdk/lib/index.js:424:7691)
    at async Deployments.isSingleAssetPublished (/home/dmc/.nvm/versions/node/v18.18.0/lib/node_modules/aws-cdk/lib/index.js:424:11602)
    at async /home/dmc/.nvm/versions/node/v18.18.0/lib/node_modules/aws-cdk/lib/index.js:424:165621

After aws sso login:

$ cdk deploy -v --profile foo
...
Do you wish to deploy these changes (y/n)?

I can also repro and re-trigger the error simply by running aws sso logout

My ~/.aws/config is structured like:


[sso-session my_session]
sso_start_url = https://d-xxxxxxxxxx.awsapps.com/start
sso_region = us-east-1

[default]
sso_session = my_session

[profile profile_a]
sso_session = my_session
sso_account_id = 01234
sso_role_name = MyRole
region = us-east-1

[profile profile_b]
sso_session = my_session
sso_account_id = 56789
sso_role_name = MyRole
region = us-east-1
drewmullen commented 6 months ago

ive been using AWS sso temp creds for weeks. not really had an issue till tonight. same issue as described above

mfittko commented 6 months ago

I have been getting this issue on multiple systems with various cdk CLI versions and it never disappeared to date. It drove me so nuts, that I have created a command wrapper script that actually ensures that I am logged in as it is using aws cli to retrieve the current profile's account ID. I pass this profile into the cdk command and I am 100% sure that I even set the AWS_ACCOUNT_ID env variable and the AWS_PROFILE env variable, but still, I am sometimes getting this!

So I am also setting CDK_DEFAULT_ACCOUNT now when constructing/running the cdk command from my wrapper tool, which should hopefully "fix" the issue.

jedrekdomanski commented 5 months ago

I have the same problem. I have 3 different profiles of which 2 are configured with sso and one is my private account and I see this in the logs

Unable to determine the default AWS account (ProcessCredentialsProviderFailure): Profile jedrek did not include credential process

~/.aws/.config

[profile jedrek]
region = eu-central-1
output = json
cli_pager =

~/.aws/.credentials

[jedrek]
AWS_ACCESS_KEY_ID=xxxxxxx
AWS_SECRET_ACCESS_KEY=xxxxxx

I can list my files in my S3 bucket aws s3 ls and it gives me proper result so my account is 100% configured correctly.

I just want to mention that CDK used to work for me before I configured the two SSO accounts in AWS CLI.

Is there a fix for this? I simply cannot deploy my stack to AWS now at all :(

mfittko commented 5 months ago

Did you try setting the account id via CDK_DEFAULT_ACCOUNT? I know it's an ugly hack and something in cdk is broken, but that helped for me.

jedrekdomanski commented 5 months ago

Did you try setting the account id via CDK_DEFAULT_ACCOUNT? I know it's an ugly hack and something in cdk is broken, but that helped for me.

Of course I did and it still doesn't work. I even did export CDK_DEFAULT_ACCOUNT=xxxxx and export CDK_DEFAULT_REGION=eu-central-1 and then I get a different error:

❌ Deployment failed: Error: Need to perform AWS calls for account xxxxxx, but no credentials have been configured
mfittko commented 4 months ago

we're still all getting this intermittently when running cdk commands, even if setting CDK_DEFAULT_ACCOUNT and the aws account on the stacks. Any plans on tackling this?

jrbryo commented 2 months ago

I just saw this as well. I restarted my terminal session and everything worked again.

DimitriosKay commented 2 months ago

Having the same issues come out of nowhere, running for a week now.

Tried all the common suggestions: logging out and in (restarting terminals), using a different browser to authenticate, with/without default profile, with/without a .aws/credentials file

Still getting the: Unable to determine the default AWS account (ProcessCredentialsProviderFailure): Profile X did not include credential process

Here is my .aws/config:

[profile main]
sso_session = user
sso_account_id = XXX
sso_role_name = AdministratorAccess
region = eu-west-2
output = json
[profile support]
sso_session = user
sso_account_id = XXX
sso_role_name = SupportAccess
region = eu-west-2
output = json
[sso-session user]
sso_start_url = https://samp.awsapps.com/start/#
sso_region = eu-west-2
sso_registration_scopes = sso:account:access

Correct me if I'm wrong, but the idea behind this sso-session block above is so you don't input AWS_ACCESS_KEY_ID=xxxxxxx and AWS_SECRET_ACCESS_KEY=xxxxxx.

Should also add, I have replicated the exact same setup on a different machine and it works just fine. AWS CLI works fine even where CDK doesn't. What does CDK do under the hood that can cause it not to recognise the files under .aws/sso/cache ? (if that's at all what it pulls when you run an aws sso login)

Dev Environment: CDK: 2.137.0 CLI: 2.15.40

EDIT (fix for users with ESET Security on Windows):

Had the notion something must have changed on my work environment, things that have been working for a year don't just break. I'm on Windows and have ESET Security set up. Long story short ESET had an update exactly one week ago (17 April) and that took out my npm and cdk. Seems to be something to do with certificates.

To those who have been facing CDK ProcessCredentialsProviderFailure issues, have a dig at the below links. Fix is simple if you are a Windows user with an ESET Security install.

https://stackoverflow.com/a/78347970/11131550 https://forum.eset.com/topic/40702-eset-ssl-protection-produces-an-invalid-certificate-chain-for-nodejs-apps/

calid commented 2 months ago

@DimitriosKay try adding a default section with your sso-session set:

[sso-session user]
sso_start_url = https://samp.awsapps.com/start/#
sso_region = eu-west-2
sso_registration_scopes = sso:account:access

[default]
sso_session = user
DimitriosKay commented 2 months ago

@calid I have this in and out at times. Mostly it allows me not to specify a profile on sso login so far. I have updated my comment with my solution to the issue. Very specific case no doubt, but it may help someone.

kevin-donovan-zocdoc commented 2 months ago

Hi is there any update on this? None of the workarounds work for me. I don't see this issue all of the time, but once I hit it, I get it consistently.

mfittko commented 2 months ago

I built a small ruby based CLI tools and wrapped all my cdk calls in a retry method that calls aws sso login if the cdk call fails and retries 😅

oosawy commented 2 months ago

After this error occurs many times, with aws sso login then cdk with --profile instead of AWS_PROFILE env like cdk bootstrap --profile oosawy-dev worked for me.

And this error occurred again with cdk bootstrap even cdk bootstrap --profile $AWS_PROFILE works.

Does cdk not support even a basic AWS_PROFILE env?

gdratp commented 2 weeks ago

In my case unless I export it explicitly it won't work. AWS_PROFILE=profile-name npx cdk deploy doesn't work. But export AWS_PROFILE=profile-name; npx cdk deploy does work.