aws-cloudformation / aws-cloudformation-resource-providers-stepfunctions

The CloudFormation Resource Provider Package For AWS Step Functions
https://aws.amazon.com/step-functions/
Apache License 2.0
6 stars 4 forks source link

Enable ForceGlobalBucketAccess on S3 client #50

Closed hongkuntian closed 8 months ago

hongkuntian commented 8 months ago

Issue #, if available: N/A

Description of changes:

Add .withForceGlobalBucketAccessEnabled(true) to the S3 client builder.

This will enable cross-region + cross-account access to S3 buckets. While S3 bucket names are globally unique, the data within each bucket is stored in a specific region chosen by the user, and the StateMachine resource handler cannot handler the scenario where the S3 bucket (that is created in a separate account) location is different from the region where the Create Handler is invoked when the bucket is from a different account. (Seems to be a limitation of the S3 client)

Currently, if the bucket is created in a different account and if the region where the bucket is physically located differs from the client config in AwsClientBuilder.setRegion(String), then the GetObject call to S3 fails and the Create handler is unable to retrieve the definition from the S3 bucket, resulting in CREATE_FAILED

This change won't impact existing users of the handler. The client still first calls to DEFAULT_REGION_PROVIDER, and enabling this config modifies the behavior such that instead of immediately failing if the S3 bucket's location does not match, the client will correctly determine where to route the request if the bucket is not located in DEFAULT_REGION_PROVIDER.

When enabled, the region of the bucket can differ from the client config and still succeed, since the client will determine the correct region to route the request.

The following behavior is currently used when this mode is enabled:

  1. All requests that do not act on an existing bucket (for example, AmazonS3Client.createBucket(String)) will be routed to the region configured by AwsClientBuilder.setRegion(String), unless the region is manually overridden with CreateBucketRequest.setRegion(String), in which case the request will be routed to the region configured in the request.
  2. The first time a request is made that references an existing bucket (for example, AmazonS3Client.putObject(PutObjectRequest)) a request will be made to the region configured by AwsClientBuilder.setRegion(String) to determine the region in which the bucket was created. This location may be cached in the client for subsequent requests acting on that same bucket. Enabling this mode has several drawbacks, because it has the potential to increase latency in the event that the location of the bucket is physically far from the location from which the request was invoked. For this reason, it is strongly advised when possible to know the location of your buckets and create a region-specific client to access that bucket.

https://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/s3/AmazonS3Builder.html#setForceGlobalBucketAccessEnabled-java.lang.Boolean-

Contract tests passed ✅ with this change

The following scenarios were manually tested, and stack creation succeeded:

  1. same region + same account S3 access
  2. x-region S3 access
  3. x-account S3 access
  4. x-region + x-account S3 access

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.