S3 Endpoints

[rjocoleman]

I spotted this: https://github.com/aboisvert/s3cp/blob/master/lib/s3cp/utils.rb#L47

FYI: aws/s3 has got a method to get this programatically: https://github.com/aws/aws-sdk-ruby/blob/master/lib/aws/s3/bucket.rb#L257

It maps to this: http://docs.aws.amazon.com/general/latest/gr/rande.html#s3_region http://docs.aws.amazon.com/AmazonS3/latest/API/RESTBucketGETlocation.html

[aboisvert]

Make sense ... but slightly different than the endpoint configuration.

What I'm thinking is the following, 1) Use S3CP_ENDPOINT if set, "us-east-1" as default. 2) Check bucket location_constraint (this will use the endpoint from #1). If location is restricted, switch to the necessary region. Otherwise, continue with endpoint set in #1.

Does that fit what you had in mind?

On Sun, May 19, 2013 at 12:59 AM, Robert Coleman notifications@github.comwrote:

I spotted this: https://github.com/aboisvert/s3cp/blob/master/lib/s3cp/utils.rb#L47

FYI: aws/s3 has got a method to get this programatically: https://github.com/aws/aws-sdk-ruby/blob/master/lib/aws/s3/bucket.rb#L257

It maps to this: http://docs.aws.amazon.com/general/latest/gr/rande.html#s3_region http://docs.aws.amazon.com/AmazonS3/latest/API/RESTBucketGETlocation.html

— Reply to this email directly or view it on GitHubhttps://github.com/aboisvert/s3cp/issues/12 .

[rjocoleman]

• As I read it S3CP_ENDPOINT is a total override, it specifies the whole endpoint in Amazons jargon, and S3CP uses that if set.. With bucket location/LocationConstraint support I can't imagine a use case for this but that could be me being naive :blush: I expect this is for power users with very specific requirements. If this ENV variable is set don't move on to the next step.
• I expect the bucket location API call should serve most non us-east-1 users by automatically determining their endpoint and enable s3cp to work without configuration past the key, id and bucket. You don't need to query a specific endpoint for this, just the generic s3 API host = BucketName.s3.amazonaws.com I'd be hitting this via aws/s3 and building the endpoint from it's result. As I see it EU and US Standard are the outliers - the rest of the endpoints can be generated without much manipulation.