search

awslabs / awsprocesscreds

Process credential providers for AWS SDKs and Tools
Apache License 2.0
132 stars 40 forks source link

awsprocesscreds.saml.SAMLError: Unable to choose role #15

Closed mbokman closed 6 years ago

mbokman commented 6 years ago

If I use an --role-arn which is different than my "initial" role but is valid for me to switch to, I get the following error:

Error when retrieving credentials from custom-process: Traceback (most recent call last):
  File "/usr/local/bin/awsprocesscreds-saml", line 11, in <module>
    sys.exit(saml())
  File "/Library/Python/2.7/site-packages/awsprocesscreds/cli.py", line 81, in saml
    creds = fetcher.fetch_credentials()
  File "/Library/Python/2.7/site-packages/awsprocesscreds/saml.py", line 348, in fetch_credentials
    creds = super(SAMLCredentialFetcher, self).fetch_credentials()
  File "/Library/Python/2.7/site-packages/botocore/credentials.py", line 555, in fetch_credentials
    return self._get_cached_credentials()
  File "/Library/Python/2.7/site-packages/botocore/credentials.py", line 565, in _get_cached_credentials
    response = self._get_credentials()
  File "/Library/Python/2.7/site-packages/awsprocesscreds/saml.py", line 357, in _get_credentials
    kwargs = self._get_assume_role_kwargs()
  File "/Library/Python/2.7/site-packages/awsprocesscreds/saml.py", line 398, in _get_assume_role_kwargs
    arns = self._get_role_and_principal_arn(assertion)
  File "/Library/Python/2.7/site-packages/awsprocesscreds/saml.py", line 379, in _get_role_and_principal_arn
    self._config.get('role_arn'), role_arns
awsprocesscreds.saml.SAMLError: Unable to choose role "arn:aws:iam::1234567890:role/RoleInDifferentAccount" from ['arn:aws:iam::0987654321:role/InitialRole']

I have obscured the actual account numbers and role names above but the important thing is that they are different roles in different accounts (it is a sub accounts versus a master account in our organization) and I have no problem switching between them in the AWS Console.

I do not get the error if I use the "InitialRole" as the argument to --role-arn but obviously then I am not in the account I want to be, nor do I have the correct role.

This is when using the ADFS provider and I had to make manual changes to be able to log in as described in issue #11

hoegertn commented 6 years ago

You can only log in to roles, that are configured for your ADFS. Directly switching to another role is not possible as it has to be included in the SAML response.

But it should be possible to define another profile in your .aws/config that uses your SAML profile as a source and does the switching afterward.

mbokman commented 6 years ago

Hi @hoegertn thanks for your response!

I am a bit of a ADFS / SAML noob so I am probably doing something very wrong, I can not get this to work.

I have the following set up:

$ cat ~/.aws/config
[profile InitialRole]
region=us-east-1
[profile SwitchRole]
region=us-east-1
role_arn=arn:aws:iam::0987654321:role/SwitchRole
source_profile=InitialRole
$ cat  ~/.aws/credentials
[InitialRole]
credential_process = awsprocesscreds-saml --endpoint 'https://fs.company.com/adfs/ls/idpinitiatedsignon?loginToRp=urn:amazon:webservices' --username 'me@company.com' --provider adfs --role-arn 'arn:aws:iam::1234567890:role/InitialRole' --verbose

Again, using the InitialRole is not an issue:

$ aws ec2 --profile InitialRole describe-availability-zones --query "AvailabilityZones[0].RegionName"
"us-east-1"

However, if I try to switch role:

$ aws ec2 --profile SwitchRole describe-availability-zones --query "AvailabilityZones[0].RegionName"

'role_arn'
hoegertn commented 6 years ago

This looks ok but I configure the credential_process in the .aws/config file. Not sure this is the issue but it is worth a try. Are you sure the InitialRole is allowed to assume the SwitchRole?

mbokman commented 6 years ago

Moving the credential_process to the .aws/config file does not help.

I am pretty sure that assuming the role is allowed. Normally I use a tool called aws_auth.py that will retrieve a token and sets it for a profile called saml. If I set source_profile=saml for the SwitchRole it works and I login as the InitialRole by default so in my view it is doing the same.

mbokman commented 6 years ago

I believe I am running into this issue:

aws ec2 --profile SwitchRole --region us-east-1 describe-availability-zones --query "AvailabilityZones[0].RegionName" --debug
2017-12-23 14:49:43,387 - MainThread - awscli.clidriver - DEBUG - CLI version: aws-cli/1.14.10 Python/3.6.4 Darwin/17.4.0 botocore/1.8.14
2017-12-23 14:49:43,388 - MainThread - awscli.clidriver - DEBUG - Arguments entered to CLI: ['ec2', '--profile', 'SwitchRole', '--region', 'us-east-1', 'describe-availability-zones', '--query', 'AvailabilityZones[0].RegionName', '--debug']
2017-12-23 14:49:43,388 - MainThread - botocore.hooks - DEBUG - Event session-initialized: calling handler <function add_scalar_parsers at 0x10b35e7b8>
2017-12-23 14:49:43,388 - MainThread - botocore.session - DEBUG - Loading variable profile from instance vars with value 'SwitchRole'.
2017-12-23 14:49:43,388 - MainThread - botocore.hooks - DEBUG - Event session-initialized: calling handler <function inject_assume_role_provider_cache at 0x10ae599d8>
2017-12-23 14:49:43,389 - MainThread - botocore.session - DEBUG - Loading variable profile from instance vars with value 'SwitchRole'.
2017-12-23 14:49:43,389 - MainThread - botocore.session - DEBUG - Loading variable credentials_file from defaults.
2017-12-23 14:49:43,389 - MainThread - botocore.session - DEBUG - Loading variable config_file from defaults.
2017-12-23 14:49:43,389 - MainThread - botocore.session - DEBUG - Loading variable profile from instance vars with value 'SwitchRole'.
2017-12-23 14:49:43,389 - MainThread - botocore.session - DEBUG - Loading variable metadata_service_timeout from defaults.
2017-12-23 14:49:43,389 - MainThread - botocore.session - DEBUG - Loading variable profile from instance vars with value 'SwitchRole'.
2017-12-23 14:49:43,389 - MainThread - botocore.session - DEBUG - Loading variable metadata_service_num_attempts from defaults.
2017-12-23 14:49:43,390 - MainThread - botocore.session - DEBUG - Loading variable profile from instance vars with value 'SwitchRole'.
2017-12-23 14:49:43,390 - MainThread - botocore.credentials - DEBUG - Skipping environment variable credential check because profile name was explicitly set.
2017-12-23 14:49:43,390 - MainThread - botocore.hooks - DEBUG - Event session-initialized: calling handler <function attach_history_handler at 0x10b104d90>
2017-12-23 14:49:43,390 - MainThread - botocore.session - DEBUG - Loading variable profile from instance vars with value 'SwitchRole'.
2017-12-23 14:49:43,391 - MainThread - botocore.session - DEBUG - Loading variable profile from instance vars with value 'SwitchRole'.
2017-12-23 14:49:43,391 - MainThread - botocore.session - DEBUG - Loading variable api_versions from defaults.
2017-12-23 14:49:43,395 - MainThread - botocore.loaders - DEBUG - Loading JSON file: /usr/local/Cellar/awscli/1.14.10/libexec/lib/python3.6/site-packages/botocore/data/ec2/2016-11-15/service-2.json
2017-12-23 14:49:43,429 - MainThread - botocore.hooks - DEBUG - Event service-data-loaded.ec2: calling handler <function register_retries_for_service at 0x10aa03268>
2017-12-23 14:49:43,429 - MainThread - botocore.handlers - DEBUG - Registering retry handlers for service: ec2
2017-12-23 14:49:43,440 - MainThread - botocore.hooks - DEBUG - Event building-command-table.ec2: calling handler functools.partial(<function _remove_commands at 0x10b2ed0d0>, commands_to_remove=['import-instance', 'import-volume'])
2017-12-23 14:49:43,440 - MainThread - awscli.customizations.removals - DEBUG - Removing operation: import-instance
2017-12-23 14:49:43,440 - MainThread - awscli.customizations.removals - DEBUG - Removing operation: import-volume
2017-12-23 14:49:43,440 - MainThread - botocore.hooks - DEBUG - Event building-command-table.ec2: calling handler <function add_waiters at 0x10b3649d8>
2017-12-23 14:49:43,448 - MainThread - botocore.loaders - DEBUG - Loading JSON file: /usr/local/Cellar/awscli/1.14.10/libexec/lib/python3.6/site-packages/botocore/data/ec2/2016-11-15/waiters-2.json
2017-12-23 14:49:43,450 - MainThread - awscli.clidriver - DEBUG - OrderedDict([('filters', <awscli.arguments.ListArgument object at 0x10baa4be0>), ('zone-names', <awscli.arguments.ListArgument object at 0x10baa4c18>), ('dry-run', <awscli.arguments.BooleanArgument object at 0x10baa4cf8>), ('no-dry-run', <awscli.arguments.BooleanArgument object at 0x10baa4cc0>)])
2017-12-23 14:49:43,450 - MainThread - botocore.hooks - DEBUG - Event building-argument-table.ec2.describe-availability-zones: calling handler <function add_streaming_output_arg at 0x10b35ea60>
2017-12-23 14:49:43,450 - MainThread - botocore.hooks - DEBUG - Event building-argument-table.ec2.describe-availability-zones: calling handler <function rename_arg.<locals>._rename_arg at 0x10b37a510>
2017-12-23 14:49:43,450 - MainThread - botocore.hooks - DEBUG - Event building-argument-table.ec2.describe-availability-zones: calling handler <function rename_arg.<locals>._rename_arg at 0x10b37a598>
2017-12-23 14:49:43,451 - MainThread - botocore.hooks - DEBUG - Event building-argument-table.ec2.describe-availability-zones: calling handler functools.partial(<function pull_up_bool at 0x10b364488>, event_handler=<botocore.hooks.HierarchicalEmitter object at 0x1098f3e48>)
2017-12-23 14:49:43,451 - MainThread - botocore.hooks - DEBUG - Event building-argument-table.ec2.describe-availability-zones: calling handler <function add_cli_input_json at 0x10ae59e18>
2017-12-23 14:49:43,451 - MainThread - botocore.hooks - DEBUG - Event building-argument-table.ec2.describe-availability-zones: calling handler <function unify_paging_params at 0x10b2db488>
2017-12-23 14:49:43,459 - MainThread - botocore.loaders - DEBUG - Loading JSON file: /usr/local/Cellar/awscli/1.14.10/libexec/lib/python3.6/site-packages/botocore/data/ec2/2016-11-15/paginators-1.json
2017-12-23 14:49:43,460 - MainThread - botocore.hooks - DEBUG - Event building-argument-table.ec2.describe-availability-zones: calling handler <function add_generate_skeleton at 0x10b2c1d08>
2017-12-23 14:49:43,460 - MainThread - botocore.hooks - DEBUG - Event before-building-argument-table-parser.ec2.describe-availability-zones: calling handler <bound method OverrideRequiredArgsArgument.override_required_args of <awscli.customizations.cliinputjson.CliInputJSONArgument object at 0x10baa4d68>>
2017-12-23 14:49:43,460 - MainThread - botocore.hooks - DEBUG - Event before-building-argument-table-parser.ec2.describe-availability-zones: calling handler <bound method GenerateCliSkeletonArgument.override_required_args of <awscli.customizations.generatecliskeleton.GenerateCliSkeletonArgument object at 0x10baa4da0>>
2017-12-23 14:49:43,461 - MainThread - botocore.hooks - DEBUG - Event operation-args-parsed.ec2.describe-availability-zones: calling handler functools.partial(<function validate_boolean_mutex_groups at 0x10b364510>, boolean_pairs=[])
2017-12-23 14:49:43,461 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.ec2.describe-availability-zones.filters: calling handler <function uri_param at 0x10ade99d8>
2017-12-23 14:49:43,462 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.ec2.describe-availability-zones.zone-names: calling handler <function uri_param at 0x10ade99d8>
2017-12-23 14:49:43,462 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.ec2.describe-availability-zones.dry-run: calling handler <function uri_param at 0x10ade99d8>
2017-12-23 14:49:43,462 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.ec2.describe-availability-zones.cli-input-json: calling handler <function uri_param at 0x10ade99d8>
2017-12-23 14:49:43,462 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.ec2.describe-availability-zones.generate-cli-skeleton: calling handler <function uri_param at 0x10ade99d8>
2017-12-23 14:49:43,462 - MainThread - botocore.hooks - DEBUG - Event calling-command.ec2.describe-availability-zones: calling handler <bound method CliInputJSONArgument.add_to_call_parameters of <awscli.customizations.cliinputjson.CliInputJSONArgument object at 0x10baa4d68>>
2017-12-23 14:49:43,462 - MainThread - botocore.hooks - DEBUG - Event calling-command.ec2.describe-availability-zones: calling handler <bound method GenerateCliSkeletonArgument.generate_json_skeleton of <awscli.customizations.generatecliskeleton.GenerateCliSkeletonArgument object at 0x10baa4da0>>
2017-12-23 14:49:43,462 - MainThread - botocore.session - DEBUG - Loading variable profile from instance vars with value 'SwitchRole'.
2017-12-23 14:49:43,463 - MainThread - botocore.session - DEBUG - Loading variable ca_bundle from defaults.
2017-12-23 14:49:43,463 - MainThread - botocore.session - DEBUG - Loading variable profile from instance vars with value 'SwitchRole'.
2017-12-23 14:49:43,463 - MainThread - botocore.session - DEBUG - Loading variable api_versions from defaults.
2017-12-23 14:49:43,463 - MainThread - botocore.credentials - DEBUG - Looking for credentials via: assume-role
2017-12-23 14:49:43,463 - MainThread - awscli.clidriver - DEBUG - Exception caught in main()
Traceback (most recent call last):
  File "/usr/local/Cellar/awscli/1.14.10/libexec/lib/python3.6/site-packages/awscli/clidriver.py", line 207, in main
    return command_table[parsed_args.command](remaining, parsed_args)
  File "/usr/local/Cellar/awscli/1.14.10/libexec/lib/python3.6/site-packages/awscli/clidriver.py", line 347, in __call__
    return command_table[parsed_args.operation](remaining, parsed_globals)
  File "/usr/local/Cellar/awscli/1.14.10/libexec/lib/python3.6/site-packages/awscli/clidriver.py", line 519, in __call__
    call_parameters, parsed_globals)
  File "/usr/local/Cellar/awscli/1.14.10/libexec/lib/python3.6/site-packages/awscli/clidriver.py", line 637, in invoke
    verify=parsed_globals.verify_ssl)
  File "/usr/local/Cellar/awscli/1.14.10/libexec/lib/python3.6/site-packages/botocore/session.py", line 850, in create_client
    credentials = self.get_credentials()
  File "/usr/local/Cellar/awscli/1.14.10/libexec/lib/python3.6/site-packages/botocore/session.py", line 474, in get_credentials
    'credential_provider').load_credentials()
  File "/usr/local/Cellar/awscli/1.14.10/libexec/lib/python3.6/site-packages/botocore/credentials.py", line 1642, in load_credentials
    creds = provider.load()
  File "/usr/local/Cellar/awscli/1.14.10/libexec/lib/python3.6/site-packages/botocore/credentials.py", line 1211, in load
    return self._load_creds_via_assume_role(self._profile_name)
  File "/usr/local/Cellar/awscli/1.14.10/libexec/lib/python3.6/site-packages/botocore/credentials.py", line 1220, in _load_creds_via_assume_role
    role_config, profile_name
  File "/usr/local/Cellar/awscli/1.14.10/libexec/lib/python3.6/site-packages/botocore/credentials.py", line 1361, in _resolve_source_credentials
    return self._resolve_credentials_from_profile(source_profile)
  File "/usr/local/Cellar/awscli/1.14.10/libexec/lib/python3.6/site-packages/botocore/credentials.py", line 1370, in _resolve_credentials_from_profile
    return self._load_creds_via_assume_role(profile_name)
  File "/usr/local/Cellar/awscli/1.14.10/libexec/lib/python3.6/site-packages/botocore/credentials.py", line 1218, in _load_creds_via_assume_role
    role_config = self._get_role_config(profile_name)
  File "/usr/local/Cellar/awscli/1.14.10/libexec/lib/python3.6/site-packages/botocore/credentials.py", line 1263, in _get_role_config
    role_arn = profile['role_arn']
KeyError: 'role_arn'
2017-12-23 14:49:43,466 - MainThread - awscli.clidriver - DEBUG - Exiting with rc 255

'role_arn'

But I do not understand the comment on that issue that says "a profile references a source profile that has no credentials".

mbokman commented 6 years ago

I do see however that when I use my saml profile as source_profile, the aws_auth.py tool seems to set a aws_access_key_id, aws_secret_access_key and aws_session_token for the profile, in the ~/.aws/credentials.

Is awsprocesscreds supposed to do something similar?

mbokman commented 6 years ago

I am at a loss how AWS CLI works with credentials caching but I got it working by doing the following:

$ diff saml.py saml.py.org
4d3
< import ConfigParser
21d19
< from os.path import expanduser
351d348
<         self.save_credentials(creds['access_key'], creds['secret_key'], creds['token'])
359,388d355
<     def save_credentials(self, access_key, secret_key, token):
<         # awsconfigfile: The file where this script will store the temp
<         # credentials under the saml profile
<         awsconfigfile = '/.aws/credentials'
<
<         # Write the AWS STS token into the AWS credential file
<         filename = expanduser("~/" + awsconfigfile)
<
<         # Read in the existing config file
<         config = ConfigParser.RawConfigParser()
<         config.read(filename)
<
<         # Put the credentials into a saml specific section instead of clobbering
<         # the default credentials
<         if not config.has_section('saml'):
<             config.add_section('saml')
<
<         if access_key:
<             config.set('saml', 'aws_access_key_id', access_key)
<         if secret_key:
<             config.set('saml', 'aws_secret_access_key', secret_key)
<         if token:
<             config.set('saml', 'aws_session_token', token)
<
<         # Write the updated config file
<         with open(filename, 'w+') as configfile:
<             config.write(configfile)
<
<         return
<

This saves the aws_access_key_id, aws_secret_access_key and aws_session_token in the ~/.aws/credentials file with the values obtained from SAML.

With ~/.aws/config like this:

$ more ~/.aws/config
[profile saml]
region=us-east-1
credential_process=/usr/local/bin/awsprocesscreds-saml --endpoint 'https://fs.company.com/adfs/ls/idpinitiatedsignon?loginToRp=urn:amazon:webservices' --username 'me@company.com' --provider adfs --role-arn 'arn:aws:iam::1234567890:role/InitialRole' --verbose

[profile SwitchRole]
region=eu-central-1
role_arn=arn:aws:iam::0987654321:role/SwitchRole
source_profile=saml

I can now execute AWS CLI command using different roles:

$ aws ec2 --profile saml describe-availability-zones --query "AvailabilityZones[0].RegionName"
"us-east-1"
$ aws ec2 --profile SwitchRole describe-availability-zones --query "AvailabilityZones[0].RegionName"
"eu-central-1"

Probably I totally do not understand how this all is supposed to work and thus I totally slaughtered the intent of the awsprocesscreds tool :)

JordonPhillips commented 6 years ago

Unfortunately process profiles are not supported as a source for assume role profiles currently, which is why this isn't working. There is an existing feature request for supporting that feature at boto/botocore#1329

The provider in this repo requires you to have a specific role, and that needs to be your final role. If you would like to do something more complicated in the mean time, you can always wrap the command. This would not be terribly difficult to do since all you need to do is provide a very simple json blob to make a valid process provider. See here for more details on how to do that.

Closing this out in favor of the botocore feature request.