search

google / go-cloud

The Go Cloud Development Kit (Go CDK): A library and tools for open cloud development in Go.
https://gocloud.dev/
Apache License 2.0
9.57k stars 812 forks source link

s3blob: Need a way to set custom credentials provider for S3 when using v2 #3512

Open hairyhenderson opened 3 days ago

hairyhenderson commented 3 days ago

Describe the bug

The aws-sdk-go (v1) module is being removed soon, and so I'm trying to migrate to aws-sdk-go-v2 (see also #3489).

I have a need to sometimes use the anonymous CredentialsProvider, and with V1 I could achieve this by setting the ConfigProvider field with an AWS session properly configured with the anonymous provider.

I'm using the blob.BucketURLOpener interface to support non-S3 buckets as well via URLs, and so using s3blob.OpenBucketV2 with a custom client isn't an option.

To Reproduce

Attempt to use &s3blob.URLOpener{UseV2: true} to open S3 buckets with anonymous credential...

Expected behavior

There are a few ways to achieve this - one way would be to allow adding additional config.LoadOptions items to be provided to s3blob.V2ConfigFromURLParams via a new field in the s3blob.URLOpener struct.

Alternately, a new URL query parameter could be supported - something like anonymous=true. FWIW, that would be useful in the gcsblob package as well.

Version

v0.40.0 (latest)

Additional context

n/a

vangent commented 3 days ago

I will take a look. It would save me some time if you could provide some code snippets for how you use anonymous credential with AWSv1 + Go CDK today, and how you would access S3 with anonymous credentials without Go CDK. Thanks!

hairyhenderson commented 3 days ago

@vangent if you have a look at the diff in https://github.com/hairyhenderson/go-fsimpl/pull/892 you'll see the old v1 approach and the new one - I copied some of the code from here to override it temporarily and support an anonymous query param.

Let me know if that's useful, or if you want more explanation...