Retry on `IllegalLocationConstraintException`

[avivfisher]

Describe the bug

For all AWS Regions launched after March 20, 2019 you need to specify the AWS Region with the request, Corresponding AWS clients implemented a retry mechanism which checks the bucket region with head_bucket and retry with the bucket region. See: https://github.com/aws/aws-sdk-java/blob/master/aws-java-sdk-s3/src/main/java/com/amazonaws/services/s3/AmazonS3Client.java#L5541

We've got S3RegionRedirectorv2 which does essentially this on some cases: is_special_head_object, is_wrong_signing_region, is_permanent_redirect, is_special_head_bucket, is_redirect_status But not IllegalLocationConstraintException which is the error code we'll get when specifying an incorrect region name. I was wondering if there is any particular reason we don't redirect on incorrect region_name.

Example output: {'Code': 'IllegalLocationConstraintException', 'Message': 'The af-south-1 location constraint is incompatible for the region specific endpoint this request was sent to.'}

Expected Behavior

On IllegalLocationConstraintException get the correct bucket region and retry.

Current Behavior

On IllegalLocationConstraintException the request isn't retried with the correct bucket region.

Reproduction Steps

import boto3 client = boto3.client("s3", region_name="af-south-1") client.upload_file("file.txt", "bucket-in-af-south-1", Key="key")

Possible Solution

Add to S3RegionRedirectorv2::redirect_from_error a logic which handles IllegalLocationConstraintException error_code, attempt to get the bucket name from the response or head_bucket and retry.

Additional Information/Context

No response

SDK version used

1.34.76

Environment details (OS name and version, etc.)

Ubuntu

[RyanFitzSimmonsAK]

Hi @avivfisher, thanks for reaching out. Could you clarify a few things, to make sure I correctly understand the ask?

Corresponding AWS clients implemented a retry mechanism which checks the bucket region with head_bucket and retry with the bucket region. See: https://github.com/aws/aws-sdk-java/blob/master/aws-java-sdk-s3/src/main/java/com/amazonaws/services/s3/AmazonS3Client.java#L5541

Are you saying that the Java SDK will automatically redirect requests from or to opt-in regions in the event that there is a mismatch between bucket region and client region? That's a documented restriction of S3, so I'd be very interested if that were the case.

import boto3 client = boto3.client("s3", region_name="af-south-1") client.upload_file("file.txt", "bucket-in-af-south-1", Key="key")

This code seems to be working for me, which makes sense given that the client and bucket regions match.

If your ask is to allow region redirects to and from opt-in regions, that'd be a valid feature request, but not one for specifically Boto3. That's the sort of behavior for which we'd want SDK parity.

[github-actions[bot]]

Greetings! It looks like this issue hasn’t been active in longer than five days. We encourage you to check if this is still an issue in the latest release. In the absence of more information, we will be closing this issue soon. If you find that this is still a problem, please feel free to provide a comment or upvote with a reaction on the initial post to prevent automatic closure. If the issue is already closed, please feel free to open a new one.