Interface vpce endpoint is not working for JAVA SDK APIs

[mrahul487]

Describe the bug

I created one Interface vpce endpoint for S3 and I am using regional VPCE endpoint https://bucket.vpce-06ac6c2ed6870c0be-x5kbxukj.s3.us-west-1.vpce.amazonaws.com

Environnet: Deployed my code in EC2 in private subnet and created one VPCE endpoint in us-west-1 My S3 bucket is also in same region(us-west-1)

AmazonS3ClientBuilder s3ClientBuilder = AmazonS3ClientBuilder.standard() .withCredentials(credentialsProvider) .withForceGlobalBucketAccessEnabled(true) .withClientConfiguration(clientConfiguration) .withEndpointConfiguration("https://bucket.vpce-06ac6c2ed6870c0be-x5kbxukj.s3.us-west-1.vpce.amazonaws.com", "us-west-1");

SDK Version:

com.amazonaws aws-java-sdk-s3 1.12.481

When I am trying to use APIs its failing with below mentioned error

Unable to execute HTTP request: Connect to rahul-private-link.s3.us-west-1.amazonaws.com:443 [rahul-private-link.s3.us-west-1.amazonaws.com/52.219.113.106] failed: connect timed out

My VPCE endpoint is not used . Also createBucket fails with same error

CreateBucketRequest request = new CreateBucketRequest(bucketName,"us-west-1"); s3.createBucket(request);

Using CLI create buckets works fine and also if I do "Enable DNS Name" in endpoint it is works fine

aws s3api create-bucket --bucket abctest123 --region us-west-1 --endpoint-url https://bucket.vpce-06ac6c2ed6870c0be-x5kbxukj.s3.us-west-1.vpce.amazonaws.com --create-bucket-configuration LocationConstraint=us-west-1

Looks like APIs have problem with the VPCE endpoint and after failed reponse it falls back to public DNS names.

Expected Behavior

APIs should work as expected

Current Behavior

Not able to use the vpce endpoints for APIs

Reproduction Steps

1. Create Interface VPCE endpoint
2. Bucket also craeted in same region
3. Use the JAVA SDK APIs like doesBucketExistV2 , createBucket()

Possible Solution

NA

Additional Information/Context

No response

AWS Java SDK version used

1.12.481

JDK version used

openjdk 11.0.19 2023-04-18

Operating System and version

Ubuntu 22.04.2 LT

[mrahul487]

This is how I am creating my S3. Client AmazonS3ClientBuilder s3ClientBuilder = AmazonS3ClientBuilder.standard() .withCredentials(credentialsProvider) .withClientConfiguration(clientConfiguration) .withForceGlobalBucketAccessEnabled(true);

What i found out is if I remove withForceGlobalBucketAccessEnabled(true), from builder it works fine.

The purpose of withForceGlobalBucketAccessEnabled(true) is top check availability of Buckets in another regions as well. If I create an interface endpoint in region us-west-1 should my S3 bucket be in the same region(us-west-1 ) ?

[debora-ito]

What i found out is if I remove withForceGlobalBucketAccessEnabled(true), from builder it works fine.

ForceGlobalBucketAccess does some weird handwritten logic to determine which region the client should send requests to. My guess is that, because vpc endpoints were released after the ForceGlobalBucketAccess attribute was implemented, it does not work very well when it is enabled.

[debora-ito]

Glad you're not blocked by this anymore. Let us know if you have more questions.

[mrahul487]

Hello AWs/Aws-Sdk-Java Team,

Thanks for your reply. Actually our problem is not yet solved. We need to use ForceGlobalBucketAccess to true. Do we have any workaround for this?

Can you suggest me any SDK version which we can use?

On Tue, 25 Jul 2023 at 2:07 AM, Debora N. Ito @.***> wrote:

Glad you're not blocked by this anymore. Let us know if you have more questions.

— Reply to this email directly, view it on GitHub https://github.com/aws/aws-sdk-java/issues/2992#issuecomment-1648568569, or unsubscribe https://github.com/notifications/unsubscribe-auth/AG6RODJGV6HGWOKQLS5SISDXR3MJ3ANCNFSM6AAAAAAZM7Y3LQ . You are receiving this because you authored the thread.Message ID: @.***>

[debora-ito]

If you need to use ForceGlobalBucketAccess with other use cases that don't use VPCEs, you'll need to create two different s3 clients, one for VPCE and one with ForceGlobalBucketAccess enabled.

If you want ForceGlobalBucketAccess to work with VPCEs, that would be a feature request.

[github-actions[bot]]

It looks like this issue has not been active for more than five days. In the absence of more information, we will be closing this issue soon. If you find that this is still a problem, please add a comment to prevent automatic closure, or if the issue is already closed please feel free to reopen it.

[dheerajsah]

Hello AWS SDK team,

using a vpc endpoint for the URI in the below manner

client = (AmazonS3Client) AmazonS3ClientBuilder.standard() .withEndpointConfiguration( new AwsClientBuilder.EndpointConfiguration( URI.toString() + ":" + String.valueOf(port), signingRegionDefault)) .withCredentials(credentialsProvider) .withClientConfiguration(clientConfiguration) .withPathStyleAccessEnabled(this.isPathStyleAccessEnabled) .build(); List bucketsList = client.listBuckets();

Also does not works, with or without the ForceGlobalBucketAccess enabled. I am assuming, we need to now rely on V2 version using S3Client class to make it work. Although the library is packaged into a separate structure so one can use it simultaneously with v1 - it will still be a problem for those who are using frameworks tied over the v1 of the AWS SDK and expect a support for VPC to work in there, as v1 dependent reference classes like S3Object also don't work as returned values because they are in no a different package.

It would be good to have a support back ported to v1 of the SDK.

Using: 1.12.133 AWS SDK

Thanks.