search

aws / aws-cli

Universal Command Line Interface for Amazon Web Services
Other
15.57k stars 4.13k forks source link

Cannot access IAM via CLI in GovCloud with short-term credentials. #8918

Closed ranok closed 2 months ago

ranok commented 2 months ago

Describe the bug

I am trying to get an IAM role details for a role in GovCloud (specified region as us-gov-west-1) while using a cli that's configured with short-term credentials (ASIA...) and an aws_session_token set. I can perform API queries to other services (e.g., STS, S3, Lambda), but IAM throws the following error (also tried via boto3): botocore.exceptions.ClientError: An error occurred (InvalidClientTokenId) when calling the GetRole operation: The security token included in the request is invalid

I have verified that the endpoint is correct (iam.us-gov.amazonaws.com), and when I created the short term credentials with sts get-session-token I specified the region.

When I use permanent access credentials, this works fine, but the short term credentials fail for IAM specifically.

Expected Behavior

Like when aws cli is configured with static, long-term creds, I expect to see the role details for the specified role.

Current Behavior

aws iam get-role --role-name ROLE_NAME --debug --region us-gov-west-1
2024-09-12 14:08:40,953 - MainThread - awscli.clidriver - DEBUG - CLI version: aws-cli/2.15.25 Python/3.11.8 Darwin/23.6.0 exe/x86_64
2024-09-12 14:08:40,955 - MainThread - awscli.clidriver - DEBUG - Arguments entered to CLI: ['iam', 'get-role', '--role-name', 'ROLE_NAME, '--debug', '--region', 'us-gov-west-1']
2024-09-12 14:08:40,972 - MainThread - botocore.hooks - DEBUG - Event building-command-table.main: calling handler <function add_s3 at 0x11067eca0>
2024-09-12 14:08:40,972 - MainThread - botocore.hooks - DEBUG - Event building-command-table.main: calling handler <function add_ddb at 0x1104cac00>
2024-09-12 14:08:40,972 - MainThread - botocore.hooks - DEBUG - Event building-command-table.main: calling handler <bound method BasicCommand.add_command of <class 'awscli.customizations.configure.configure.ConfigureCommand'>>
2024-09-12 14:08:40,972 - MainThread - botocore.hooks - DEBUG - Event building-command-table.main: calling handler <function change_name at 0x110426840>
2024-09-12 14:08:40,972 - MainThread - botocore.hooks - DEBUG - Event building-command-table.main: calling handler <function change_name at 0x1104562a0>
2024-09-12 14:08:40,972 - MainThread - botocore.hooks - DEBUG - Event building-command-table.main: calling handler <function alias_opsworks_cm at 0x1106b1760>
2024-09-12 14:08:40,972 - MainThread - botocore.hooks - DEBUG - Event building-command-table.main: calling handler <function add_history_commands at 0x110531620>
2024-09-12 14:08:40,972 - MainThread - botocore.hooks - DEBUG - Event building-command-table.main: calling handler <bound method BasicCommand.add_command of <class 'awscli.customizations.devcommands.CLIDevCommand'>>
2024-09-12 14:08:40,973 - MainThread - botocore.hooks - DEBUG - Event building-command-table.main: calling handler <function add_waiters at 0x1106b1620>
2024-09-12 14:08:40,973 - MainThread - botocore.hooks - DEBUG - Event building-command-table.main: calling handler <bound method AliasSubCommandInjector.on_building_command_table of <awscli.alias.AliasSubCommandInjector object at 0x110774410>>
2024-09-12 14:08:40,973 - MainThread - botocore.loaders - DEBUG - Loading JSON file: /usr/local/aws-cli/awscli/data/cli.json
2024-09-12 14:08:40,974 - MainThread - botocore.hooks - DEBUG - Event top-level-args-parsed: calling handler <function resolve_types at 0x1105dafc0>
2024-09-12 14:08:40,974 - MainThread - botocore.hooks - DEBUG - Event top-level-args-parsed: calling handler <function no_sign_request at 0x1105db2e0>
2024-09-12 14:08:40,974 - MainThread - botocore.hooks - DEBUG - Event top-level-args-parsed: calling handler <function resolve_verify_ssl at 0x1105db240>
2024-09-12 14:08:40,974 - MainThread - botocore.hooks - DEBUG - Event top-level-args-parsed: calling handler <function resolve_cli_read_timeout at 0x1105db420>
2024-09-12 14:08:40,974 - MainThread - botocore.hooks - DEBUG - Event top-level-args-parsed: calling handler <function resolve_cli_connect_timeout at 0x1105db380>
2024-09-12 14:08:40,974 - MainThread - botocore.hooks - DEBUG - Event top-level-args-parsed: calling handler <built-in method update of dict object at 0x110771700>
2024-09-12 14:08:40,974 - MainThread - botocore.session - DEBUG - Setting config variable for region to 'us-gov-west-1'
2024-09-12 14:08:40,974 - MainThread - awscli.clidriver - DEBUG - CLI version: aws-cli/2.15.25 Python/3.11.8 Darwin/23.6.0 exe/x86_64 prompt/off
2024-09-12 14:08:40,974 - MainThread - awscli.clidriver - DEBUG - Arguments entered to CLI: ['iam', 'get-role', '--role-name', 'ROLE_NAME', '--debug', '--region', 'us-gov-west-1']
2024-09-12 14:08:40,974 - MainThread - botocore.hooks - DEBUG - Event session-initialized: calling handler <function add_timestamp_parser at 0x11067f600>
2024-09-12 14:08:40,974 - MainThread - botocore.hooks - DEBUG - Event session-initialized: calling handler <function register_uri_param_handler at 0x10fd3ed40>
2024-09-12 14:08:40,974 - MainThread - botocore.hooks - DEBUG - Event session-initialized: calling handler <function add_binary_formatter at 0x110732e80>
2024-09-12 14:08:40,974 - MainThread - botocore.hooks - DEBUG - Event session-initialized: calling handler <function no_pager_handler at 0x10fc30fe0>
2024-09-12 14:08:40,974 - MainThread - botocore.hooks - DEBUG - Event session-initialized: calling handler <function inject_assume_role_provider_cache at 0x10fd5a660>
2024-09-12 14:08:40,976 - MainThread - botocore.utils - DEBUG - IMDS ENDPOINT: http://169.254.169.254/
2024-09-12 14:08:40,984 - MainThread - botocore.hooks - DEBUG - Event session-initialized: calling handler <function attach_history_handler at 0x110509ee0>
2024-09-12 14:08:40,984 - MainThread - botocore.hooks - DEBUG - Event session-initialized: calling handler <function inject_json_file_cache at 0x1104b2e80>
2024-09-12 14:08:40,999 - MainThread - botocore.loaders - DEBUG - Loading JSON file: /usr/local/aws-cli/awscli/botocore/data/iam/2010-05-08/service-2.json
2024-09-12 14:08:41,007 - MainThread - botocore.hooks - DEBUG - Event building-command-table.iam: calling handler <function _add_wizard_command at 0x110731bc0>
2024-09-12 14:08:41,007 - MainThread - botocore.hooks - DEBUG - Event building-command-table.iam: calling handler <function add_waiters at 0x1106b1620>
2024-09-12 14:08:41,020 - MainThread - botocore.loaders - DEBUG - Loading JSON file: /usr/local/aws-cli/awscli/botocore/data/iam/2010-05-08/waiters-2.json
2024-09-12 14:08:41,020 - MainThread - botocore.hooks - DEBUG - Event building-command-table.iam: calling handler <bound method AliasSubCommandInjector.on_building_command_table of <awscli.alias.AliasSubCommandInjector object at 0x110774410>>
2024-09-12 14:08:41,020 - MainThread - awscli.clidriver - DEBUG - OrderedDict([('role-name', <awscli.arguments.CLIArgument object at 0x110a3fed0>)])
2024-09-12 14:08:41,020 - MainThread - botocore.hooks - DEBUG - Event building-argument-table.iam.get-role: calling handler <function add_streaming_output_arg at 0x11067fb00>
2024-09-12 14:08:41,020 - MainThread - botocore.hooks - DEBUG - Event building-argument-table.iam.get-role: calling handler <function add_cli_input_json at 0x10fd5afc0>
2024-09-12 14:08:41,020 - MainThread - botocore.hooks - DEBUG - Event building-argument-table.iam.get-role: calling handler <function add_cli_input_yaml at 0x10fd5b060>
2024-09-12 14:08:41,020 - MainThread - botocore.hooks - DEBUG - Event building-argument-table.iam.get-role: calling handler <function unify_paging_params at 0x1104cb240>
2024-09-12 14:08:41,034 - MainThread - botocore.loaders - DEBUG - Loading JSON file: /usr/local/aws-cli/awscli/botocore/data/iam/2010-05-08/paginators-1.json
2024-09-12 14:08:41,034 - MainThread - botocore.hooks - DEBUG - Event building-argument-table.iam.get-role: calling handler <function add_generate_skeleton at 0x1105d9800>
2024-09-12 14:08:41,034 - MainThread - botocore.hooks - DEBUG - Event before-building-argument-table-parser.iam.get-role: calling handler <bound method OverrideRequiredArgsArgument.override_required_args of <awscli.customizations.cliinput.CliInputJSONArgument object at 0x110a3c4d0>>
2024-09-12 14:08:41,034 - MainThread - botocore.hooks - DEBUG - Event before-building-argument-table-parser.iam.get-role: calling handler <bound method OverrideRequiredArgsArgument.override_required_args of <awscli.customizations.cliinput.CliInputYAMLArgument object at 0x110998c10>>
2024-09-12 14:08:41,034 - MainThread - botocore.hooks - DEBUG - Event before-building-argument-table-parser.iam.get-role: calling handler <bound method GenerateCliSkeletonArgument.override_required_args of <awscli.customizations.generatecliskeleton.GenerateCliSkeletonArgument object at 0x110a3cc90>>
2024-09-12 14:08:41,034 - MainThread - botocore.hooks - DEBUG - Event building-command-table.iam_get-role: calling handler <function add_waiters at 0x1106b1620>
2024-09-12 14:08:41,034 - MainThread - botocore.hooks - DEBUG - Event building-command-table.iam_get-role: calling handler <bound method AliasSubCommandInjector.on_building_command_table of <awscli.alias.AliasSubCommandInjector object at 0x110774410>>
2024-09-12 14:08:41,034 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.iam.get-role.role-name: calling handler <awscli.paramfile.URIArgumentHandler object at 0x1107a7750>
2024-09-12 14:08:41,034 - MainThread - botocore.hooks - DEBUG - Event process-cli-arg.iam.get-role: calling handler <awscli.argprocess.ParamShorthandParser object at 0x10fc5e350>
2024-09-12 14:08:41,034 - MainThread - awscli.arguments - DEBUG - Unpacked value of 'ROLE_NAME' for parameter "role_name": 'ROLE_NAME'
2024-09-12 14:08:41,034 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.iam.get-role.cli-input-json: calling handler <awscli.paramfile.URIArgumentHandler object at 0x1107a7750>
2024-09-12 14:08:41,034 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.iam.get-role.cli-input-yaml: calling handler <awscli.paramfile.URIArgumentHandler object at 0x1107a7750>
2024-09-12 14:08:41,034 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.iam.get-role.generate-cli-skeleton: calling handler <awscli.paramfile.URIArgumentHandler object at 0x1107a7750>
2024-09-12 14:08:41,034 - MainThread - botocore.hooks - DEBUG - Event calling-command.iam.get-role: calling handler <bound method CliInputArgument.add_to_call_parameters of <awscli.customizations.cliinput.CliInputJSONArgument object at 0x110a3c4d0>>
2024-09-12 14:08:41,034 - MainThread - botocore.hooks - DEBUG - Event calling-command.iam.get-role: calling handler <bound method CliInputArgument.add_to_call_parameters of <awscli.customizations.cliinput.CliInputYAMLArgument object at 0x110998c10>>
2024-09-12 14:08:41,034 - MainThread - botocore.hooks - DEBUG - Event calling-command.iam.get-role: calling handler <bound method GenerateCliSkeletonArgument.generate_skeleton of <awscli.customizations.generatecliskeleton.GenerateCliSkeletonArgument object at 0x110a3cc90>>
2024-09-12 14:08:41,035 - MainThread - botocore.credentials - DEBUG - Looking for credentials via: env
2024-09-12 14:08:41,035 - MainThread - botocore.credentials - DEBUG - Looking for credentials via: assume-role
2024-09-12 14:08:41,035 - MainThread - botocore.credentials - DEBUG - Looking for credentials via: assume-role-with-web-identity
2024-09-12 14:08:41,035 - MainThread - botocore.credentials - DEBUG - Looking for credentials via: sso
2024-09-12 14:08:41,035 - MainThread - botocore.credentials - DEBUG - Looking for credentials via: shared-credentials-file
2024-09-12 14:08:41,035 - MainThread - botocore.credentials - INFO - Found credentials in shared credentials file: ~/.aws/credentials
2024-09-12 14:08:41,036 - MainThread - botocore.loaders - DEBUG - Loading JSON file: /usr/local/aws-cli/awscli/botocore/data/endpoints.json
2024-09-12 14:08:41,044 - MainThread - botocore.hooks - DEBUG - Event choose-service-name: calling handler <function handle_service_name_alias at 0x10e8bea20>
2024-09-12 14:08:41,058 - MainThread - botocore.loaders - DEBUG - Loading JSON file: /usr/local/aws-cli/awscli/botocore/data/iam/2010-05-08/endpoint-rule-set-1.json
2024-09-12 14:08:41,059 - MainThread - botocore.loaders - DEBUG - Loading JSON file: /usr/local/aws-cli/awscli/botocore/data/partitions.json
2024-09-12 14:08:41,060 - MainThread - botocore.hooks - DEBUG - Event creating-client-class.iam: calling handler <function add_generate_presigned_url at 0x10e80ca40>
2024-09-12 14:08:41,060 - MainThread - botocore.configprovider - DEBUG - Looking for endpoint for iam via: environment_service
2024-09-12 14:08:41,060 - MainThread - botocore.configprovider - DEBUG - Looking for endpoint for iam via: environment_global
2024-09-12 14:08:41,060 - MainThread - botocore.configprovider - DEBUG - Looking for endpoint for iam via: config_service
2024-09-12 14:08:41,060 - MainThread - botocore.configprovider - DEBUG - Looking for endpoint for iam via: config_global
2024-09-12 14:08:41,060 - MainThread - botocore.configprovider - DEBUG - No configured endpoint found.
2024-09-12 14:08:41,061 - MainThread - botocore.regions - DEBUG - Using partition endpoint for iam, us-gov-west-1: aws-us-gov-global
2024-09-12 14:08:41,062 - MainThread - botocore.endpoint - DEBUG - Setting iam timeout as (60, 60)
2024-09-12 14:08:41,062 - MainThread - botocore.regions - DEBUG - Calling endpoint provider with parameters: {'Region': 'us-gov-west-1', 'UseDualStack': False, 'UseFIPS': False}
2024-09-12 14:08:41,063 - MainThread - botocore.regions - DEBUG - Endpoint provider result: https://iam.us-gov.amazonaws.com
2024-09-12 14:08:41,063 - MainThread - botocore.regions - DEBUG - Selecting from endpoint provider's list of auth schemes: "sigv4". User selected auth scheme is: "None"
2024-09-12 14:08:41,063 - MainThread - botocore.regions - DEBUG - Selected auth type "v4" as "v4" with signing context params: {'region': 'us-gov-west-1', 'signing_name': 'iam'}
2024-09-12 14:08:41,063 - MainThread - botocore.hooks - DEBUG - Event provide-client-params.iam.GetRole: calling handler <function base64_decode_input_blobs at 0x110732f20>
2024-09-12 14:08:41,063 - MainThread - botocore.hooks - DEBUG - Event before-parameter-build.iam.GetRole: calling handler <function generate_idempotent_uuid at 0x10e8e4d60>
2024-09-12 14:08:41,063 - MainThread - botocore.hooks - DEBUG - Event before-call.iam.GetRole: calling handler <function inject_api_version_header_if_needed at 0x10e8e6840>
2024-09-12 14:08:41,063 - MainThread - botocore.endpoint - DEBUG - Making request for OperationModel(name=GetRole) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'aws-cli/2.15.25 Python/3.11.8 Darwin/23.6.0 exe/x86_64 prompt/off command/iam.get-role'}, 'body': {'Action': 'GetRole', 'Version': '2010-05-08', 'RoleName': 'lambda-test-session-create-role'}, 'url': 'https://iam.us-gov.amazonaws.com/', 'context': {'client_region': 'aws-us-gov-global', 'client_config': <botocore.config.Config object at 0x110f506d0>, 'has_streaming_input': False, 'auth_type': 'v4', 'signing': {'region': 'us-gov-west-1', 'signing_name': 'iam'}, 'endpoint_properties': {'authSchemes': [{'name': 'sigv4', 'signingName': 'iam', 'signingRegion': 'us-gov-west-1'}]}}}
2024-09-12 14:08:41,063 - MainThread - botocore.hooks - DEBUG - Event request-created.iam.GetRole: calling handler <bound method RequestSigner.handler of <botocore.signers.RequestSigner object at 0x110e41990>>
2024-09-12 14:08:41,063 - MainThread - botocore.hooks - DEBUG - Event choose-signer.iam.GetRole: calling handler <function set_operation_specific_signer at 0x10e8e4c20>
2024-09-12 14:08:41,064 - MainThread - botocore.auth - DEBUG - Calculating signature using v4 auth.
2024-09-12 14:08:41,064 - MainThread - botocore.auth - DEBUG - CanonicalRequest:
POST
/

content-type:application/x-www-form-urlencoded; charset=utf-8
host:iam.us-gov.amazonaws.com
x-amz-date:20240912T200841Z
x-amz-security-token:REDACTED

content-type;host;x-amz-date;x-amz-security-token
74b72a3883b5c8a56c4470e82868fa93b8859579248627bf8dae9f2d63f3677c
2024-09-12 14:08:41,064 - MainThread - botocore.auth - DEBUG - StringToSign:
AWS4-HMAC-SHA256
20240912T200841Z
20240912/us-gov-west-1/iam/aws4_request
c55a81c0ff4e38601b5899b00a0aff945f0928a592270b5f4f983ab58af6a33d
2024-09-12 14:08:41,064 - MainThread - botocore.auth - DEBUG - Signature:
fec59f4db720406899d6b3844d15387136d527ff5bee49a38547c0d22893fc8b
2024-09-12 14:08:41,064 - MainThread - botocore.endpoint - DEBUG - Sending http request: <AWSPreparedRequest stream_output=False, method=POST, url=https://iam.us-gov.amazonaws.com/, headers={'Content-Type': b'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': b'aws-cli/2.15.25 Python/3.11.8 Darwin/23.6.0 exe/x86_64 prompt/off command/iam.get-role', 'X-Amz-Date': b'20240912T200841Z', 'X-Amz-Security-Token': b'REDACTED', 'Authorization': b'AWS4-HMAC-SHA256 Credential=ASIAREDACTED/20240912/us-gov-west-1/iam/aws4_request, SignedHeaders=content-type;host;x-amz-date;x-amz-security-token, Signature=fec59f4db720406899d6b3844d15387136d527ff5...', 'Content-Length': '74'}>
2024-09-12 14:08:41,065 - MainThread - botocore.httpsession - DEBUG - Certificate path: /usr/local/aws-cli/awscli/botocore/cacert.pem
2024-09-12 14:08:41,065 - MainThread - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): iam.us-gov.amazonaws.com:443
2024-09-12 14:08:41,484 - MainThread - urllib3.connectionpool - DEBUG - https://iam.us-gov.amazonaws.com:443 "POST / HTTP/1.1" 403 305
2024-09-12 14:08:41,484 - MainThread - botocore.parsers - DEBUG - Response headers: {'Date': 'Thu, 12 Sep 2024 20:08:40 GMT', 'x-amzn-RequestId': '6ca66a39-87b0-4d36-aab9-4bff707642a2', 'Content-Type': 'text/xml', 'Content-Length': '305'}
2024-09-12 14:08:41,484 - MainThread - botocore.parsers - DEBUG - Response body:
b'<ErrorResponse xmlns="https://iam.amazonaws.com/doc/2010-05-08/">\n  <Error>\n    <Type>Sender</Type>\n    <Code>InvalidClientTokenId</Code>\n    <Message>The security token included in the request is invalid</Message>\n  </Error>\n  <RequestId>6ca66a39-87b0-4d36-aab9-4bff707642a2</RequestId>\n</ErrorResponse>\n'
2024-09-12 14:08:41,489 - MainThread - botocore.hooks - DEBUG - Event needs-retry.iam.GetRole: calling handler <bound method RetryHandler.needs_retry of <botocore.retries.standard.RetryHandler object at 0x1109aa5d0>>
2024-09-12 14:08:41,490 - MainThread - botocore.retries.standard - DEBUG - Not retrying request.
2024-09-12 14:08:41,490 - MainThread - botocore.hooks - DEBUG - Event after-call.iam.GetRole: calling handler <function json_decode_policies at 0x10e8e59e0>
2024-09-12 14:08:41,490 - MainThread - botocore.hooks - DEBUG - Event after-call.iam.GetRole: calling handler <bound method RetryQuotaChecker.release_retry_quota of <botocore.retries.standard.RetryQuotaChecker object at 0x110f0b450>>
2024-09-12 14:08:41,490 - MainThread - awscli.clidriver - DEBUG - Exception caught in main()
Traceback (most recent call last):
  File "awscli/clidriver.py", line 460, in main
  File "awscli/clidriver.py", line 595, in __call__
  File "awscli/clidriver.py", line 798, in __call__
  File "awscli/clidriver.py", line 929, in invoke
  File "awscli/clidriver.py", line 941, in _make_client_call
  File "awscli/botocore/client.py", line 357, in _api_call
  File "awscli/botocore/client.py", line 724, in _make_api_call
botocore.exceptions.ClientError: An error occurred (InvalidClientTokenId) when calling the GetRole operation: The security token included in the request is invalid

Reproduction Steps

For a GovCloud account, get a temporary session token with aws sts get-session-token, then configure the AWS CLI with those values, use aws configure set aws_session_token TOKEN to set the session token. Then try to perform an IAM get-role call.

Possible Solution

No response

Additional Information/Context

No response

CLI version used

aws-cli/2.17.49

Environment details (OS name and version, etc.)

aws-cli/2.17.49 Python/3.11.9 Darwin/23.6.0 exe/x86_64

ranok commented 2 months ago

Apparently this is a limitation with GetSessionToken without MFA: https://docs.aws.amazon.com/STS/latest/APIReference/API_GetSessionToken.html

github-actions[bot] commented 2 months ago

This issue is now closed. Comments on closed issues are hard for our team to see. If you need more assistance, please open a new issue that references this one.