search

99designs / aws-vault

A vault for securely storing and accessing AWS credentials in development environments
MIT License
8.52k stars 820 forks source link

SSO broken in aws-vault 7.1.0 #1186

Closed brontolinux closed 1 year ago

brontolinux commented 1 year ago

I have a set-up of aws-vault with AWS SSO (IAM Identity Center). Things worked perfectly with v7.0.x and 7.0.1 in particular.

This morning I upgraded to 7.1.0. As soon as I tried to either access the console or a shell, I got a failure:

$ aws-vault login myprofile 
aws-vault: error: login: Failed to get credentials: operation error STS: GetSessionToken, https response error StatusCode: 403, RequestID: 06d17f46-6d61-4801-a0bf-9015c3ee5bf9, api error InvalidClientTokenId: The security token included in the request is invalid.

Same thing with exec:

$ aws-vault exec myprofile 
aws-vault: error: exec: Failed to get credentials for rikstvtest: operation error STS: GetSessionToken, https response error StatusCode: 403, RequestID: 7d4077d7-3f3e-4399-b6c1-cc07aa723478, api error InvalidClientTokenId: The security token included in the request is invalid.

The relevant part of config is (redacted):

[profile myprofile]
sso_start_url=https://mycompany.awsapps.com/start
sso_region=eu-west-1
sso_account_id=1234567890
sso_role_name=SRE-Admin

One thing that may be relevant is that, before using AWS SSO, the profile myprofile had "static" credentials, that we rotated every 24 hours and I added to my laptop's keyring with aws-vault add. It could be that aws-vault 7.1.0 is pointing to those expired credentials instead of running the SSO authentication, but I don't know if it's the case.

I am using Debian Linux 11 with the latest security updates. My desktop environment is GNOME.

HTH

Ciao -- bronto

mtibben commented 1 year ago

Please provide debug output

brontolinux commented 1 year ago

With 7.1.0 (doesn't work):

$ aws-vault-7.1.0 --debug login myprofile
2023/03/15 16:01:36 aws-vault v7.1.0
2023/03/15 16:01:36 Using prompt driver: terminal
2023/03/15 16:01:36 [keyring] Considering backends: [secret-service]
2023/03/15 16:01:36 Loading config file /home/bronto/.aws/config
2023/03/15 16:01:36 Parsing config file /home/bronto/.aws/config
2023/03/15 16:01:36 Using region "eu-west-1" from AWS_DEFAULT_REGION
2023/03/15 16:01:36 Profile 'default' missing in config file
2023/03/15 16:01:36 profile myprofile: using stored credentials
2023/03/15 16:01:36 profile myprofile: using GetSessionToken 
2023/03/15 16:01:36 Looking up keyring for 'myprofile'
aws-vault: error: login: Failed to get credentials: operation error STS: GetSessionToken, https response error StatusCode: 403, RequestID: c5e349d3-8075-4eaa-941d-d44c6ff005dd, api error InvalidClientTokenId: The security token included in the request is invalid.
$  /var/cfengine/cache/bin/aws-vault-7.1.0 --debug exec myprofile
2023/03/15 16:02:43 aws-vault v7.1.0
2023/03/15 16:02:43 Using prompt driver: terminal
2023/03/15 16:02:43 Loading config file /home/bronto/.aws/config
2023/03/15 16:02:43 Parsing config file /home/bronto/.aws/config
2023/03/15 16:02:43 [keyring] Considering backends: [secret-service]
2023/03/15 16:02:43 Using region "eu-west-1" from AWS_DEFAULT_REGION
2023/03/15 16:02:43 Profile 'default' missing in config file
2023/03/15 16:02:43 profile myprofile: using stored credentials
2023/03/15 16:02:43 profile myprofile: using GetSessionToken 
2023/03/15 16:02:43 Setting subprocess env: AWS_REGION=eu-west-1, AWS_DEFAULT_REGION=eu-west-1
2023/03/15 16:02:43 Looking up keyring for 'myprofile'
aws-vault: error: exec: Failed to get credentials for myprofile: operation error STS: GetSessionToken, https response error StatusCode: 403, RequestID: 278ed1fb-b1ef-41a5-8cc6-329fa111b540, api error InvalidClientTokenId: The security token included in the request is invalid.

With 7.0.1 (works):

$ aws-vault --debug exec myprofile
2023/03/15 16:04:26 aws-vault v7.0.1
2023/03/15 16:04:26 Using prompt driver: terminal
2023/03/15 16:04:26 Loading config file /home/bronto/.aws/config
2023/03/15 16:04:26 Parsing config file /home/bronto/.aws/config
2023/03/15 16:04:26 [keyring] Considering backends: [secret-service]
2023/03/15 16:04:26 Using region "eu-west-1" from AWS_DEFAULT_REGION
2023/03/15 16:04:26 Profile 'default' missing in config file
2023/03/15 16:04:26 profile myprofile: using SSO role credentials
2023/03/15 16:04:26 Setting subprocess env: AWS_REGION=eu-west-1, AWS_DEFAULT_REGION=eu-west-1
2023/03/15 16:04:26 Created new OIDC client (expires at: 2023-06-13 17:04:26 +0200 CEST)
2023/03/15 16:04:26 Created OIDC device code for https://mycompany.awsapps.com/start (expires in: 600s)
2023/03/15 16:04:26 Opening SSO authorization page in browser
Opening the SSO authorization page in your default browser (use Ctrl-C to abort)
https://device.sso.eu-west-1.amazonaws.com/?user_code=XXXX-XXXX
2023/03/15 16:04:33 Created new OIDC access token for https://mycompany.awsapps.com/start (expires in: 11428s)
2023/03/15 16:04:33 Got credentials ****************6QKT for SSO role SRE-Admin (account: 1234567890), expires in 7h59m58.628982653s
2023/03/15 16:04:33 Setting subprocess env: AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY
2023/03/15 16:04:33 Setting subprocess env: AWS_SESSION_TOKEN
2023/03/15 16:04:33 Setting subprocess env: AWS_CREDENTIAL_EXPIRATION
2023/03/15 16:04:33 Starting subshell /bin/bash, use `exit` to exit the subshell
2023/03/15 16:04:33 Exec command /bin/bash 
2023/03/15 16:04:33 Found executable /bin/bash

So it definitely looks like a regression to me. 7.1.0 gets to the old, expired credentials from the secret-service, tries to use those, and fails. 7.0.1 considers secret-service, too, but then finds the SSO configuration in myprofile and uses that one instead.

FernandoMiguel commented 1 year ago

as a data point, i'm not seeing this issue on v7.1.0 and using SSO to tens of accounts

brontolinux commented 1 year ago

as a data point, i'm not seeing this issue on v7.1.0 and using SSO to tens of accounts

Hi @FernandoMiguel

What makes a difference in my case is that, for that profile, I have some old static credentials in the keyring that were in use before I switched that profile to SSO. If I didn't have those, 7.1.0 would work just fine: it would not find credentials in secret-service that match the requested profile and would use config instead, so SSO. But 7.1.0 finds the old static credentials and tries to use them instead of relying on the authoritative information in config, that indicates SSO as the source of credentials.

7.0.1 does the right thing there, in that it finds credentials through both secret-service and config, and uses the latter. That's why I think this is a regression.

mtibben commented 1 year ago

Yes it looks like your myprofile is using stored credentials instead of the sso config. You can aws-vault rm myprofile to remove the old credentials. v7.0.0 and v7.0.1 preferenced the SSO config incorrectly, fixed in v7.0.2.

brontolinux commented 1 year ago

Yes it looks like your myprofile is using stored credentials instead of the sso config. You can aws-vault rm myprofile to remove the old credentials. v7.0.0 and v7.0.1 preferenced the SSO config incorrectly, fixed in v7.0.2.

Hi @mtibben I can confirm that everything worked in 7.1.0 after removing the old static credentials. I feel that the "wrong" behaviour in v7.0.1 was more consistent in that it considered config to be authoritative over the rest, but... aws-vault is yours, and so the decision 😉 Ciao -- bronto

mtibben commented 1 year ago

I don't disagree @brontolinux.... I also thought this behaviour would be more correct but ended up with many side-effects.... see https://github.com/99designs/aws-vault/pull/1183