search

boto / botocore

The low-level, core functionality of boto3 and the AWS CLI.
Apache License 2.0
1.49k stars 1.09k forks source link

IMDS is queried when credential_process is set #3272

Open vincer opened 2 weeks ago

vincer commented 2 weeks ago

Describe the bug

In my ~/.aws/config I have 

[profile foo]
credential_process = foo

For some reason this seems to trigger the IMDS credential provider to try to get credentials from the metadata service which, since I'm not running this in EC2, adds several seconds of latency as it retries the connections a few times.

Regression Issue

Expected Behavior

Not calling the IMDS service when credential_process is set.

Current Behavior

It does call the IMDS service.

Reproduction Steps

~/.aws/config:

credential_process = foo

aws --debug --profile foo sts get-caller-identity

Note in the debug output that it is trying to GET http://169.254.169.254/latest/api/token.

Possible Solution

I'm not entirely sure why this behavior gets triggered, but it feels like a bug. It seems like if the user is setting credential_process, then the SDK should simply use that process and not check other credential providers for creds.

If there is a good reason for this behavior, it would be useful to have an option that is settable in ~/.aws/config to disable it, whether by profile, or globally.

Additional Information/Context

A relatively simple workaround is setting export AWS_EC2_METADATA_DISABLED=true, though not a very user friendly one as we'd have to explain to all our users to set this in their environment.

Rerunning with that set skips the IMDS checks.

SDK version used

aws-cli/2.17.64

Environment details (OS name and version, etc.)

macOS 14

adev-code commented 1 week ago

Hello @vincer, thanks for reaching out. The only way for IMDS to be disabled is by having AWS_EC2_METADATA_DISABLED=true as per the documentation [1] and what you have used. If you have any follow up questions, please do let me know. Thank you.

[1] https://docs.aws.amazon.com/sdkref/latest/guide/feature-imds-credentials.html#feature-imds-credentials-sdk-security

vincer commented 1 week ago

Hi @adev-code ,

In that doc link you posted the first line says:

By default, when the AWS SDK is not configured with valid credentials the SDK will attempt to use the Amazon EC2 Instance Metadata Service (IMDS) to retrieve credentials for an AWS role.

In this case, I'm setting credential_process, so this default behavior should not be triggered. So this seems like a bug.

adev-code commented 4 days ago

Hi @vincer, thanks for the update. For further look, please include the full debug response by adding boto3.set_stream_logger('') to your code and redacting any sensitive information. Thank you.