search

xeger / awsume-1password-plugin

Automates awsume MFA entry via 1Password CLI.
MIT License
9 stars 5 forks source link

Awsume error: Invalid profile [default] Missing keys aws_access_key_id, aws_secret_access_key, or credential_source, or credential_process #8

Closed deanhtid closed 6 months ago

deanhtid commented 7 months ago

Describe the bug when i try to invoke awsume in the terminal i get the following error:

Awsume error: Invalid profile [default] Missing keys aws_access_key_id, aws_secret_access_key, or credential_source, or credential_process

Expected behavior i would expect it to automatically assume my default profile

Output awsume --debug

[2024-03-19 14:41:55,080] main.py:main : [DEBUG] Debug logs are visible
[2024-03-19 14:41:55,080] main.py:main : [DEBUG] Executing awsume
[2024-03-19 14:41:55,080] app.py:__init__ : [DEBUG] Initalizing app
[2024-03-19 14:41:55,081] app.py:get_plugin_manager : [DEBUG] Creating plugin manager
[2024-03-19 14:41:55,081] app.py:get_plugin_manager : [DEBUG] Loading plugins
[2024-03-19 14:41:55,094] app.py:parse_args : [DEBUG] Gathering arguments
[2024-03-19 14:41:55,095] default_plugins.py:add_arguments : [INFO] Adding arguments
[2024-03-19 14:41:55,095] app.py:parse_args : [DEBUG] Parsing arguments
[2024-03-19 14:41:55,096] app.py:parse_args : [DEBUG] Handling arguments
[2024-03-19 14:41:55,096] default_plugins.py:post_add_arguments : [DEBUG] Post add arguments
[2024-03-19 14:41:55,096] default_plugins.py:post_add_arguments : [DEBUG] {"version": false, "output_profile": null, "clean": false, "profile_name": null, "force_refresh": false, "show_commands": false, "unset_variables": false, "auto_refresh": false, "kill": false, "list_profiles": null, "session_tags": null, "refresh_autocomplete": false, "role_arn": null, "principal_arn": null, "source_profile": null, "external_id": null, "mfa_token": null, "region": null, "session_name": null, "role_duration": null, "with_saml": false, "with_web_identity": false, "who": false, "json": null, "credentials_file": null, "config_file": null, "config": null, "list_plugins": false, "info": false, "debug": true}
[2024-03-19 14:41:55,096] default_plugins.py:post_add_arguments : [DEBUG] No profile name passed, target profile name will be "default"
[2024-03-19 14:41:55,096] app.py:get_profiles : [DEBUG] Gathering profiles
[2024-03-19 14:41:55,096] default_plugins.py:collect_aws_profiles : [INFO] Collecting AWS profiles
[2024-03-19 14:41:55,099] default_plugins.py:collect_aws_profiles : [DEBUG] Collected 6 profiles
[2024-03-19 14:41:55,099] default_plugins.py:post_collect_aws_profiles : [INFO] Post collect AWS profiles
[2024-03-19 14:41:55,099] app.py:get_credentials : [DEBUG] Getting credentials
[2024-03-19 14:41:55,099] profile.py:get_role_chain : [DEBUG] Getting role chain for [default]
Error invoking 1Password plugin; please file a bug report:
  https://github.com/xeger/awsume-1password-plugin/issues/new/choose
Traceback (most recent call last):
  File "/Users/dean/.pyenv/versions/3.12.2/lib/python3.12/site-packages/1password.py", line 102, in pre_get_credentials
    cache_file_name = 'aws-credentials-' + source_credentials.get('AccessKeyId')
                      ~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
TypeError: can only concatenate str (not "NoneType") to str
[2024-03-19 14:41:55,100] app.py:get_credentials : [DEBUG] Pulling credentials from default awsume flow
[2024-03-19 14:41:55,100] profile.py:get_role_chain : [DEBUG] Getting role chain for [default]
[2024-03-19 14:41:55,100] default_plugins.py:get_credentials : [DEBUG] Role chain: ['default']
[2024-03-19 14:41:55,100] default_plugins.py:get_credentials_handler : [INFO] Getting credentials: default
[2024-03-19 14:41:55,100] profile.py:validate_profile : [DEBUG] Validating profile
[2024-03-19 14:41:55,100] app.py:run : [DEBUG]
Traceback (most recent call last):
  File "/Users/dean/.pyenv/versions/3.12.2/lib/python3.12/site-packages/awsume/awsumepy/app.py", line 265, in run
    credentials = self.get_credentials(args, profiles)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/dean/.pyenv/versions/3.12.2/lib/python3.12/site-packages/awsume/awsumepy/app.py", line 204, in get_credentials
    credentials = self.plugin_manager.hook.get_credentials(config=self.config, arguments=args, profiles=profiles)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/dean/.pyenv/versions/3.12.2/lib/python3.12/site-packages/pluggy/_hooks.py", line 501, in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/dean/.pyenv/versions/3.12.2/lib/python3.12/site-packages/pluggy/_manager.py", line 119, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/dean/.pyenv/versions/3.12.2/lib/python3.12/site-packages/pluggy/_callers.py", line 138, in _multicall
    raise exception.with_traceback(exception.__traceback__)
  File "/Users/dean/.pyenv/versions/3.12.2/lib/python3.12/site-packages/pluggy/_callers.py", line 102, in _multicall
    res = hook_impl.function(*args)
          ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/dean/.pyenv/versions/3.12.2/lib/python3.12/site-packages/awsume/awsumepy/default_plugins.py", line 626, in get_credentials
    credentials = get_credentials_handler(config=config, arguments=arguments, profiles=profiles, profile_name=profile_name, credentials=credentials)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/dean/.pyenv/versions/3.12.2/lib/python3.12/site-packages/awsume/awsumepy/default_plugins.py", line 577, in get_credentials_handler
    profile_lib.validate_profile(config, arguments, profiles, profile_name)
  File "/Users/dean/.pyenv/versions/3.12.2/lib/python3.12/site-packages/awsume/awsumepy/lib/profile.py", line 94, in validate_profile
    raise exceptions.InvalidProfileError(user_profile_name, message='Missing keys {}, or credential_source, or credential_process'.format(', '.join(missing_keys)))
awsume.awsumepy.lib.exceptions.InvalidProfileError: Invalid profile [default] Missing keys aws_access_key_id, aws_secret_access_key, or credential_source, or credential_process
Awsume error: Invalid profile [default] Missing keys aws_access_key_id, aws_secret_access_key, or credential_source, or credential_process

Configuration If relevant, please share your AWSume configuration file below. This file should be free of secrets, but please verify before submitting your report.

colors: true
1password: AWS Prod (Company)

Additional context This works fine and returns the mfa

op item get --otp "AWS Access Key (Company)"
808596
xeger commented 7 months ago

Thank you for the bug report. Can you share your ~/.aws/config file with me? The issue seems to be with identifying the default profile in that file; your awsume configuration file looks fine.

A common use case for AWSume is to have static credentials for the default profile and to assume other roles via STS. So, config files tend to look like:

[default]
region = us-east-1

[profile assumed]
source_profile = default
mfa_serial = arn:aws:iam::1234:mfa/some-iam-user
role_arn = arn:aws:iam::1234:role/PowerUserAccess

So assuming the default is a no-op (it has hardcoded credentials) and you'd assume other profiles using MFA.

xeger commented 7 months ago

I tried fixing the crash in my plugin, but AWSume itself still crashes if the AWS config has only a single profile.

Feel free to uninstall my plugin and give it a try on your own machine, or use the development branch of my plugin, and you'll likely see AWSume crash as well. (See CONTRIBUTING.md for info on how to run this plugin in development mode inside a pipenv.)

I'm happy to release my fix, but suspect it's not a solution.

Given the AWS config below:

[default]
mfa_serial = arn:aws:iam::204005782057:mfa/REDACTED@xeger.net
role_arn = arn:aws:iam::204005782057:role/PowerUserAccess

And an AWSume config:

colors: true
1password: AWS (Xeger, readonly)

AWSume crashes if I awsume default:

Traceback (most recent call last):
  File "/PREFIX/bin/awsumepy", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/PREFIX/lib/python3.11/site-packages/awsume/awsumepy/main.py", line 29, in main
    run_awsume(sys.argv[1:])
  File "/PREFIX/lib/python3.11/site-packages/awsume/awsumepy/main.py", line 17, in run_awsume
    awsume.run(argument_list)
  File "/PREFIX/lib/python3.11/site-packages/awsume/awsumepy/app.py", line 263, in run
    credentials = self.get_credentials(args, profiles)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/PREFIX/lib/python3.11/site-packages/awsume/awsumepy/app.py", line 202, in get_credentials
    credentials = self.plugin_manager.hook.get_credentials(config=self.config, arguments=args, profiles=profiles)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/PREFIX/lib/python3.11/site-packages/pluggy/_hooks.py", line 493, in __call__
    return self._hookexec(self.name, self._hookimpls, kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/PREFIX/lib/python3.11/site-packages/pluggy/_manager.py", line 115, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/PREFIX/lib/python3.11/site-packages/pluggy/_callers.py", line 113, in _multicall
    raise exception.with_traceback(exception.__traceback__)
  File "/PREFIX/lib/python3.11/site-packages/pluggy/_callers.py", line 77, in _multicall
    res = hook_impl.function(*args)
          ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/PREFIX/lib/python3.11/site-packages/awsume/awsumepy/default_plugins.py", line 583, in get_credentials
    role_chain = get_role_chain(config, arguments, profiles, target_profile_name)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/PREFIX/lib/python3.11/site-packages/awsume/awsumepy/lib/profile.py", line 132, in get_role_chain
    profiles[target_profile_name]['mfa_serial'] = target_profile['mfa_serial']
    ~~~~~~~~^^^^^^^^^^^^^^^^^^^^^
KeyError: None
deanhtid commented 7 months ago

sorry i've just seen this now, i'll send you my config

i can get it to work if i put my access keys in the credentials file but that makes it kinda pointless. I've pasted my file below but removed an PII

deanhtid commented 7 months ago

[default] region = eu-west-1 mfa_serial = arn:aws:iam::123456789912:mfa/dean.whittaker@company.com output = json cli_pager=

[profile poc] region = eu-west-1 role_arn = arn:aws:iam::123456789912:role/CrossAccountAdminAccess source_profile = default mfa_serial = arn:aws:iam::123456789912:mfa/dean.whittaker@company.com output = json cli_pager=

[profile corenet] region = eu-west-1 role_arn = arn:aws:iam::123456789912:role/OrganizationAccountAccessRole source_profile = default mfa_serial = arn:aws:iam::123456789912:mfa/dean.whittaker@company.com output = json cli_pager=

[profile enablement] region = eu-west-1 role_arn = arn:aws:iam::123456789912:role/OrganizationAccountAccessRole source_profile = default mfa_serial = arn:aws:iam::123456789912:mfa/dean.whittaker@company.com output = json cli_pager=

[profile circleci] region = eu-west-1 role_arn = arn:aws:iam::123456789912:role/company/Automation/ci-deploy-Role-1234567 source_profile = circle-ci output = json cli_pager=

xeger commented 6 months ago

In the organizations where I've used AWSume, we typically create an AWS account that is designated solely for login operations; my IAM user and its permanent credentials exist in that account, and my [default] profile does record those credentials on disk -- but the credentials aren't good for anything; my user's only permissions are to assume roles in other accounts.

Then, I invoke awsume to assume into other proflies, and AWSume uses those profiles' role_arn plus my source profile's mfa_serial to prompt me for an MFA token -- which is where this plugin comes in. It figures out which MFA token will be needed, grabs it from 1Password, and sends it to AWSume so that I don't need to type anything.

It sounds like you're trying to accomplish something different, however: you have powerful permanent credentials that you don't want to put on disk, so I think you might be asking for this feature.

I will reproduce tonight and, if I'm crashing and AWSume is not, I'll make sure not to crash and instead print some useful information. I suspect AWSume might not be a good fit for your use case, however. As far as I know, it "assumes" (no pun intended) that the credentials needed to peform an STS role-assume operation are resident in ~/.aws/credentials

xeger commented 6 months ago

Here are my results with a config similar to yours and vanilla awsume (no 1password plugin).

~ awsume
Awsume error: Invalid profile [default] Missing keys aws_access_key_id, aws_secret_access_key, or credential_source, or credential_process
~ awsume xeger-test
Awsume error: Invalid profile [default] Missing keys aws_access_key_id, aws_secret_access_key, or credential_source, or credential_process

As you can see, awsume cannot assume default, nor any profile that uses default as a source.

My config is:

[default]
mfa_serial = arn:aws:iam::204005782057:mfa/readonly@xeger.net
output = json
profile= xeger
region = us-west-2

[profile xeger-test]
source_profile = default
mfa_serial = arn:aws:iam::204005782057:mfa/readonly@xeger.net
role_arn = arn:aws:iam::204005782057:role/PowerUserAccess

And my credentials is:

[default]

Which is to say I am not storing the credentials of either account.

The 1password plugin shouldn't crash the process, of course! But I don't think awsume itself supports what you're trying to do.

If your default user has very limited access in its own account (i.e. a super restrictive role), it should be safe to create static credentials and put them in credentials under default profile. The policy that allows the user to assume roles into should require MFA, and awsume + 1password will make MFA effortless whenever you assume into other roles/accounts.

xeger commented 6 months ago

I pushed v1.2.3 which fixes the crash, @deanhtid, but that simply falls through to AWSume's default behavior of complaining about missing credentials.

This plugin was developed to retrieve MFA codes from 1Password, and not credentials. I see the value of storing credentials in 1P as well, but it's a divergence from the requirements, which pushes awsume itself beyond its design intent, and I'd want to think about how to accomplish it cleanly. See #6 for more discussion.

Is the workaround (of using puny low-risk creds which your store in your filesystem) sufficient?