senrok / yadal

Yet Another Data Access Layer: Accessing S3, POSIX in the same way. Deeply inspired by Databend's OpenDAL
https://pkg.go.dev/github.com/senrok/yadal
Apache License 2.0
17 stars 0 forks source link

Bugs: When the path contains the special characters will fail to access the S3 service #2

Open WenyXu opened 1 year ago

WenyXu commented 1 year ago

When we try to operate resources where the path contains special characters, it generates an incorrect signature.

There is a example in the /tests/behavior/read_write_test.go#L78

// creates a file with special chars should be success
func testCreateFileWithSpecialChars(t *testing.T, op *yadal.Operator) {
    path := fmt.Sprintf("%s %s", uuid.New().String(), "!@#$%^&*()_+-=;'><,?.txt")

    o := op.Object(path)
    err := o.Create(context.TODO())
    assert.Nilf(t, err, "%s", err)

    meta, err := o.Metadata(context.TODO())
    assert.Nilf(t, err, "%s", err)

    assert.Equal(t, interfaces.FILE, meta.Mode())
    assert.Equal(t, uint64(0), *meta.ContentLength())

    err = o.Delete(context.TODO())
    assert.Nilf(t, err, "%s", err)
}

Then we will receive:

<?xml version="1.0" encoding="UTF-8"?>
<Error>
   <Code>SignatureDoesNotMatch</Code>
   <Message>The request signature we calculated does not match the signature you provided. Check your key and signing method.</Message>
   <Key>d9a2080a-c33a-4c83-a42b-e0f3255979d5 !@#$%^&amp;*()_+-=;'&gt;&lt;,?.txt</Key>
   <BucketName>test</BucketName>
   <Resource>/test/d9a2080a-c33a-4c83-a42b-e0f3255979d5 !@#$%^&amp;*()_+-=;'&gt;&lt;,?.txt</Resource>
   <Region>us-west-01</Region>
   <RequestId>171E56F294000824</RequestId>
   <HostId>d2593ff9-ea7b-4553-bc29-df692d774e27</HostId>
</Error>
Xuanwo commented 1 year ago

https://github.com/senrok/yadal/blob/926a5e61e7cbff93944b3f67264373b22a9a99fb/utils/path.go#L114

I'm guessing '(', ')', '!', '*', '\'' also need to be percent encoded.

BTW, this project is awesome!