supabase / storage

S3 compatible object storage service that stores metadata in Postgres
https://supabase.com/docs/guides/storage
Apache License 2.0
791 stars 112 forks source link

Golang aws sdk v2 cannot upload to supabase storage #577

Open hmhuan opened 6 days ago

hmhuan commented 6 days ago

Bug report

Describe the bug

I got issue while using golang aws sdk v2 (github.com/aws/aws-sdk-go-v2) to integrate with supabase storage. The document said I should check again the error SignatureDoesNotMatch but I set access key id / access secret key / region that are all correct

operation error S3: PutObject, https response error StatusCode: 403, RequestID: , HostID: , api error SignatureDoesNotMatch: The request signature we calculated does not match the signature you provided. Check your key and signing method.

To Reproduce

Steps to reproduce the behavior, please provide code snippets or a repository:

  1. Go to '…'
  2. Click on '…'
  3. Scroll down to '…'
  4. See error

Expected behavior

A clear and concise description of what you expected to happen.

Screenshots

If applicable, add screenshots to help explain your problem.

System information

Additional context

Add any other context about the problem here.

hmhuan commented 5 days ago

I try to use Golang aws sdk v1, and I could upload object normally

encima commented 5 days ago

Hi there, thanks for opening. We cannot guarantee compatibility across all SDKs and libraries that support S3, thanks for reporting that v1 does work and that v2 does not!

For now, I would suggest using v1 if possible!

mercuryyy commented 3 days ago

@encima the issue is livekit pushed a mandatory change to v2 and so have others lately and we can not revert back to v1

https://github.com/supabase/storage/issues/569

Can your team possibly take a look and see if this can be updated to support v2 ?

fenos commented 1 day ago

Hello the issue seems related to the new go SDK, I'll open an issue on their end and see what they think.

For now a workaround that seem to work, is to remove the DisableAcceptEncodingGzip middleware which is on by default in the new SDK

func removeDisableGzip() func(*middleware.Stack) error {
    return func(stack *middleware.Stack) error {
        _, err := stack.Finalize.Remove("DisableAcceptEncodingGzip")
        return err
    }
}

func main() {
    ctx := context.Background()

    endpoint := "https://tenant.supabase.co/storage/v1/s3"
    accessKeyId := ""
    secretAccessKey := ""
    region := "ap-southeast-1"

    sdkConfig := aws.Config{
        Region:       region,
        BaseEndpoint: &endpoint,
        Credentials:  credentials.NewStaticCredentialsProvider(accessKeyId, secretAccessKey, ""),
        APIOptions:  []func(*middleware.Stack) error{removeDisableGzip()},
    }

    s3Client := s3.NewFromConfig(sdkConfig)

    result, err := s3Client.ListBuckets(ctx, &s3.ListBucketsInput{})
}
fenos commented 1 day ago

Related https://github.com/aws/aws-sdk-go-v2/issues/2848

hmhuan commented 1 day ago

Thanks for your sharing

frostbyte73 commented 1 day ago

I'm hesitant to use the workaround, since we don't know which provider(s) our end users are using or whether that would break compatibility with those. V1 is also deprecated and we started seeing other issues with it, so staying on it isn't really an option

https://aws.amazon.com/blogs/developer/announcing-end-of-support-for-aws-sdk-for-go-v1-on-july-31-2025/