ListBucket mismatches signature and url

[GoogleCodeExporter]

When trying to list directories, I was getting 403 on the server side, which 
was translating to operation not permitted on the local side.

The AWS spec says that everything up until the query should be in the 
signature, and you are adding "/" to the end of the request when placed in the 
signature.  If I add "/" to turl before query then I get everything in the 
directory except the last directory within, but that's for another bug.

===================================================================
The following information is very important in order to help us to help
you.  Omission of the following details may delay your support request or
receive no attention at all.
===================================================================
Version of s3fs being used (s3fs --version): 1.74

Version of fuse being used (pkg-config --modversion fuse): 2.8.6

System information (uname -a): Linux prod-repo-1001.va.opower.it 
3.8.0-39-generic #58~precise1-Ubuntu SMP Fri May 2 21:33:40 UTC 2014 x86_64 
x86_64 x86_64 GNU/Linux

Distro (cat /etc/issue):Ubuntu 12.04.4 LTS \n \l

s3fs command line used (if applicable):

/etc/fstab entry (if applicable):

s3fs syslog messages (grep s3fs /var/log/syslog):
May 29 23:05:44 prod-repo-1001 s3fs: connecting to URL https://***/repo/
May 29 23:05:45 prod-repo-1001 s3fs: HTTP response code 200
May 29 23:05:49 prod-repo-1001 s3fs: connecting to URL 
https://***/repo?delimiter=/&prefix=&max-keys=1000
May 29 23:05:49 prod-repo-1001 s3fs: HTTP response code 403
May 29 23:05:49 prod-repo-1001 s3fs: HTTP response code 403 was returned, 
returning EPERM
May 29 23:05:49 prod-repo-1001 s3fs: Body Text: <?xml version="1.0" 
encoding="UTF-8"?><Error><Code>AccessDenied</Code></Error>
May 29 23:05:49 prod-repo-1001 s3fs: ListBucketRequest returns with error.
May 29 23:05:49 prod-repo-1001 s3fs: list_bucket returns error(-1).

Original issue reported on code.google.com by fpear...@gmail.com on 30 May 2014 at 5:37

[GoogleCodeExporter]

The only thing I can think of is that most servers are rewriting the url from 
dir to dir/ but it's not good to rely on that, this is definitely comparing the 
wrong signature for what is requested.

Original comment by fpear...@gmail.com on 30 May 2014 at 4:20

[GoogleCodeExporter]

Hi,

At first about error, it seems S3 returns AccessDenied error by list bucket 
request.
I think you should check ACL etc for this bucket.

And about "dir/" and "dir", the request for list bucket is url=xxxx/dir, and 
prefix="" by s3fs.
I think this is best way for listing.

Although it is not to limited to list bucket request, the directory name on 
S3(=object name) is not forced '/' charator for end of it's name.
So s3fs need to check the "dir" and "dir/" object name.

Regards,

Original comment by ggta...@gmail.com on 2 Jun 2014 at 4:41

• Changed state: WontFix

[GoogleCodeExporter]

Apologies.  The function is ListBucketRequest in curl.cpp

The url passed in is fine, but:

          CalcSignature("GET", "", "", date, (resource + "/"))).c_str());

You calculate the signature of the URL + "/".  Since you are unwilling to solve 
this by adding the "/" can you solve it by removing this "/" instead?

Original comment by fpear...@gmail.com on 5 Jun 2014 at 1:44

[GoogleCodeExporter]

I can confirm that if the URL matches the Signature, I no longer get the 
AccessDenied error.

Original comment by fpear...@gmail.com on 5 Jun 2014 at 1:47

[GoogleCodeExporter]

Hi, fpearson

I think that the calcurating signature with "(resource + "/")" in 
S3fsCurl::ListBucketRequest() is correct.
If building signature without "/" for resource(which is the bucket name), we 
get the response from s3 as "SignatureDoesNotMatch" error.
And the error message says that the signature should be including "/<bucket 
name>/" for base string.

I think your first report which is "<Code>AccessDenied</Code>" means not 
"SignatureDoesNotMatch".
So that it is not wrong signature.

I think your error maybe wrong accesskey/secretkey pair for s3 or not allowed 
accessing the bucket.

Please check those, and if you can, please run s3fs with "-d"(, "-f") option.

Regards,

Original comment by ggta...@gmail.com on 8 Jun 2014 at 2:07