Incorrect Canonical Resource when requesting a bucket (ie, doing `ls /dir`)

[GoogleCodeExporter]

The S3 API specifies that the Canonical Resource (part of the signature)
when requesting a list of objects in a bucket should be as follows:
    /bucketname/?querystring

However, s3fs creates the Canonical Resource like this:
    /bucketname?querystring
That is, without the trailing "/" after the bucket name.

This format will work correctly when the mounted bucket is located on S3,
as it appears that Amazon supports both forms (though they only document
the former).  

However, this format will not work in the case that, say, someone wanted to
write their own implementation of the S3 API, assuming they strictly follow
the API as documented.  The result will be a signature error, as the
resource part of the string being signed differs.

The attached patch fixes this issue, by properly constructing the resource
within the s3fs_readdir method.

I've tested with S3, and everything continues to work as expected.

Original issue reported on code.google.com by marl...@gmail.com on 24 Sep 2009 at 4:01

Attachments:

• correct-bucket-canonical-url.patch

[GoogleCodeExporter]

Here are comments from the wiki that appear to be related to this issue:

 Comment by jajcus,  Mar 15, 2010

I found a problem in the latest code:

Mar 14 18:53:41 jajo s3fs: URL is 
http://s3.amazonaws.com/bucket?delimiter=/&prefix=&max-keys=50
Mar 14 18:53:41 jajo s3fs: URL changed is 
http://bucket.s3.amazonaws.com?delimiter=/&prefix=&max-keys=50
Mar 14 18:53:41 jajo s3fs: connecting to URL 
http://bucket.s3.amazonaws.com?delimiter=/&prefix=&max-keys=50
Mar 14 18:53:41 jajo s3fs: ###response=400

The missing '/' before '?' in the rewritten url is causing a 400 error for me.

Here is a little patch that fixes this:

--- s3fs/s3fs.cpp.orig  2010-02-05 01:45:59.000000000 +0100
+++ s3fs/s3fs.cpp       2010-03-15 09:46:56.000000000 +0100
@@ -296,7 +296,10 @@
        int bucket_pos = url_str.find(token);
        int bucket_size = token.size();

-       url_str = url_str.substr(0,7) + bucket + "." + 
url_str.substr(7,bucket_pos - 7)  + url_str.substr((bucket_pos + bucket_size));
+       if (url_str[bucket_pos + bucket_size] != '/') 
+               url_str = url_str.substr(0,7) + bucket + "." + 
url_str.substr(7, bucket_pos - 7) + "/" + url_str.substr((bucket_pos + 
bucket_size));
+       else
+               url_str = url_str.substr(0,7) + bucket + "." + 
url_str.substr(7, bucket_pos - 7) + url_str.substr((bucket_pos + bucket_size));

        syslog(LOG_DEBUG, "URL changed is %s", url_str.c_str());

Original comment by dmoore4...@gmail.com on 20 Oct 2010 at 3:33

[GoogleCodeExporter]

Is this still an issue with the latest code?

Original comment by dmoore4...@gmail.com on 30 Dec 2010 at 7:25

[GoogleCodeExporter]

No response in over a month. ...closing this old issue.

Original comment by dmoore4...@gmail.com on 5 Feb 2011 at 1:37

• Changed state: WontFix