google / go-cloud

The Go Cloud Development Kit (Go CDK): A library and tools for open cloud development in Go.
https://gocloud.dev/
Apache License 2.0
9.45k stars 799 forks source link

blob/s3blob: url encode CopySource #3402

Closed srerickson closed 3 months ago

srerickson commented 3 months ago

Describe the bug

For s3 buckets, Copy() returns NoSuchKey: The specified key does not exist if the object source key includes the url path escape sequence %2f. See the example below, which uses the source key test/example%2fkey.

I think the problem is that CopyObjectInput.CopySource is supposed to be URL encoded.

To Reproduce

package main

import (
    "context"
    "log"
    "net/url"

    "gocloud.dev/blob"
    _ "gocloud.dev/blob/s3blob"
)

func main() {
    var (
        ctx  = context.Background()
        key  = "test/example%2fkey"
        dst  = "test/copy"
        byts = []byte("testing")
    )
    bucket, err := blob.OpenBucket(ctx, "s3://yourtestbucket?region=us-west-2&awssdk=v2")
    if err != nil {
        log.Fatal(err)
    }
    defer bucket.Close()
    if err := bucket.WriteAll(ctx, key, byts, nil); err != nil {
        log.Fatal(err)
    }

    // works if source key is url-encoded
    if err := bucket.Copy(ctx, dst, url.PathEscape(key), nil); err != nil {
        log.Fatalf("with url encoded source: %v\n", err)
    }

    // but this fails with error: `NoSuchKey: The specified key does not exist`
    if err := bucket.Copy(ctx, dst, key, nil); err != nil {
        log.Fatalf("without url encoded source: %v\n", err)
    }
}

Expected behavior

If an object key works with WriteAll(), I expect it to work withCopy()

Version

go.mod:

go 1.22.1

require gocloud.dev v0.37.0