Proxy issue when using S3 SDK in ECS containers

[rui-ktei]

General summary

We're running docker image (microsoft/dotnet:2.1.0-runtime) on AWS ECS. We set up our own proxy for all outgoing requests. This means that whenever we want to use services like S3, the underneath requests will go through our proxy.

So we config S3Client instance to use our proxy, something like this:

var config = new AmazonS3Config
{
    RegionEndpoint = RegionEndpoint.APSoutheast2
};
config.SetWebProxy(new WebProxy
{
    Address = new Uri($"http://1.2.3.4:3128"), // our proxy address
});

s3Client = new AmazonS3Client(config);

However the code above doesn't work and it keeps timing out. We tried to figure out why and here's our guess. The container tries to assume a ECS TaskRole before it connects to the S3 bucket, but it times out when it's assuming the TaskRole, because of the proxy setting we have. This led us to think that we should use bypass AWS IPs which SDK uses to assume a role, so we came up with this:

config.SetWebProxy(new WebProxy
{
    Address = new Uri($"http://1.2.3.4:3128"),
    BypassProxyOnLocal = true,
    BypassList = new string[] { "169.254.169.254" ,"169.254.170.2" } // we think this is the IPs SDK uses for assuming role
});

However, the code above still doesn't work, we still get timeout error.

Expected Behavior

AWS S3 SDK should work for Proxy + Assuming role in ECS container

Current Behavior

We get timeout error when using AmazonS3Client with proxy config.

Possible Solution

This could be a bug in SDK or .Net Core, we don't know.

Steps to Reproduce (for bugs)

It's hard to reproduce this problem because you need to set up a lot things. Please refer to my general summary.

Context

This issue prevents us to use our proxy in ECS containers.

Your Environment

• AWSSDK.Core version used:
• Operating System and version: microsoft/dotnet:2.1.0-runtime
• Targeted .NET platform: netcoreapp21

Thank you

[github-actions[bot]]

We have noticed this issue has not recieved attention in a year. We will close this issue for now. If you think this is in error, please feel free to comment and reopen the issue.

[dscpinheiro]

I know it's been a while, but the latest version of the SDK does support the https_proxy and no_proxy variables, which can be used instead of the Bypass* properties.

Also if you were using ECS on EC2 and IMDSv2, there's an extra configuration step you need to take in order for your container to work properly: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instancedata-data-retrieval.html#imds-considerations and https://stackoverflow.com/a/71884476

[github-actions[bot]]

Comments on closed issues are hard for our team to see. If you need more assistance, please either tag a team member or open a new issue that references this one. If you wish to keep having a conversation with other community members under this issue feel free to do so.