Add option to AWSConfigs to only use ec2 instance profile role for AWSCredentials

benfmiller commented 2 months ago

Describe the feature

Allow specifying which AWSCredentials source to use so that we can ensure our application only uses credentials from one source (instance profile).

Use Case

We are currently using InstanceProfileAWSCredentials to make sure we only get credentials from the instance profile. This has the downside that if the instance profile associated with the EC2 instance is changed, then the .NET application will still attempt to retrieve credentials for the old instance profile and fail. The new instance profile is only picked up after the application is restarted.

We want to prevent any other credential source from being used but still automatically refresh the instance profile after a different instance profile is associated with the EC2 instance. (Skip to the last entry (Amazon EC2 instance metadata) in the Credential and profile resolution)

Proposed Solution

Add an option to AWSConfigs like AWSProfileOnlyInstanceProfile so that we can use the default way to get/refresh credentials but still prevent credentials from different sources from accidentally being used.

Other Information

Error message trying to s3:PutObject using InstanceProfileAWSCredentials after a different instance profile is associated with the EC2 instance

Amazon.Runtime.AmazonClientException: The retrieved credentials have already expired: Now = 07/09/2024 05:37:58, Credentials expiration = 07/09/2024 05:37:55
   at Amazon.Runtime.RefreshingAWSCredentials.UpdateToGeneratedCredentials(CredentialsRefreshState state, TimeSpan preemptExpiryTime) in D:\JenkinsWorkspaces\trebuchet-stage-release\AWSDotNetPublic\sdk\src\Core\Amazon.Runtime\Credentials\RefreshingAWSCredentials.cs:line 185
   at Amazon.Runtime.RefreshingAWSCredentials.GetCredentials() in D:\JenkinsWorkspaces\trebuchet-stage-release\AWSDotNetPublic\sdk\src\Core\Amazon.Runtime\Credentials\RefreshingAWSCredentials.cs:line 133
   at Amazon.Runtime.Internal.CredentialsRetriever.PreInvoke(IExecutionContext executionContext) in D:\JenkinsWorkspaces\trebuchet-stage-release\AWSDotNetPublic\sdk\src\Core\Amazon.Runtime\Pipeline\Handlers\CredentialsRetriever.cs:line 54
   at Amazon.Runtime.Internal.CredentialsRetriever.InvokeSync(IExecutionContext executionContext) in D:\JenkinsWorkspaces\trebuchet-stage-release\AWSDotNetPublic\sdk\src\Core\Amazon.Runtime\Pipeline\Handlers\CredentialsRetriever.cs:line 72
   at Amazon.Runtime.Internal.RetryHandler.InvokeSync(IExecutionContext executionContext) in D:\JenkinsWorkspaces\trebuchet-stage-release\AWSDotNetPublic\sdk\src\Core\Amazon.Runtime\Pipeline\RetryHandler\RetryHandler.cs:line 91
   at Amazon.Runtime.Internal.CallbackHandler.InvokeSync(IExecutionContext executionContext) in D:\JenkinsWorkspaces\trebuchet-stage-release\AWSDotNetPublic\sdk\src\Core\Amazon.Runtime\Pipeline\Handlers\CallbackHandler.cs:line 46
   at Amazon.Runtime.Internal.CallbackHandler.InvokeSync(IExecutionContext executionContext) in D:\JenkinsWorkspaces\trebuchet-stage-release\AWSDotNetPublic\sdk\src\Core\Amazon.Runtime\Pipeline\Handlers\CallbackHandler.cs:line 46
   at Amazon.S3.Internal.AmazonS3ExceptionHandler.InvokeSync(IExecutionContext executionContext) in D:\JenkinsWorkspaces\trebuchet-stage-release\AWSDotNetPublic\sdk\src\Services\S3\Custom\Internal\AmazonS3ExceptionHandler.cs:line 48
   at Amazon.Runtime.Internal.ErrorCallbackHandler.InvokeSync(IExecutionContext executionContext) in D:\JenkinsWorkspaces\trebuchet-stage-release\AWSDotNetPublic\sdk\src\Core\Amazon.Runtime\Pipeline\Handlers\ErrorCallbackHandler.cs:line 44
   at Amazon.Runtime.Internal.MetricsHandler.InvokeSync(IExecutionContext executionContext) in D:\JenkinsWorkspaces\trebuchet-stage-release\AWSDotNetPublic\sdk\src\Core\Amazon.Runtime\Pipeline\Handlers\MetricsHandler.cs:line 40
   at Amazon.Runtime.Internal.RuntimePipeline.InvokeSync(IExecutionContext executionContext) in D:\JenkinsWorkspaces\trebuchet-stage-release\AWSDotNetPublic\sdk\src\Core\Amazon.Runtime\Pipeline\RuntimePipeline.cs:line 136
   at Amazon.Runtime.AmazonServiceClient.Invoke[TResponse](AmazonWebServiceRequest request, InvokeOptionsBase options) in D:\JenkinsWorkspaces\trebuchet-stage-release\AWSDotNetPublic\sdk\src\Core\Amazon.Runtime\AmazonServiceClient.cs:line 203

Acknowledgements

[ ] I may be able to implement this feature request
[ ] This feature might incur a breaking change

AWS .NET SDK and/or Package version used

AWSSDK.Core 3.7.0.17

Targeted .NET Platform

.NET framework 4.7.2

Operating System and version

Windows 10

ashishdhingra commented 2 months ago

@benfmiller Good afternoon. Are you referring to scenario where you replace instance profile for an EC2 instance, e.g. How do I attach or replace an instance profile on an Amazon EC2 instance??

benfmiller commented 2 months ago

@ashishdhingra Good afternoon. Yep, replacing the instance profile for an EC2 instance.

When we use the InstanceProfileAWSCredentials to make sure we only use the instance profile, it won't pick up the new instance profile after it is replaced.

If we use the default credentials (not explicitly create any credentials) it successfully picks up the new instance profile after it is replaced, but it could accidentally pick up different credentials from the Credential search order. We don't control the environment the application is running in, so we want to make sure it always uses instance profile role.

aws / aws-sdk-net