Permission denied when trying to read an AWS S3 bucket

[pkerpedjiev]

When I mount a bucket, I can list the files in the directory but get a Permission Denied error when trying to read a file. Here is the log:

https://gist.github.com/pkerpedjiev/2682a7e065663e53fa5e8116395b536f

It strikes me that none of the requests have the Authorization header, although my credentials are in ~/.aws/credentials as expected.

goofys version 0.0.16-e8b06fca04811c9faff8e481c367d07b8f5e9a90
Ubuntu 16.04.3 LTS \n \l

As per @kahing's comment (https://github.com/kahing/goofys/issues/195#issuecomment-327035670), goofys things that this is an anonymous bucket.

2017/08/31 03:59:33.510178 s3.INFO anonymous bucket detected

The bucket does indeed have public list and header read permissions but the default profile also has read and write permissions. Given that my credentials should allow me read and write access, shouldn't that supersede the fact that the bucket has additional limited anonymous read and write permissions?

I should note that mounting, reading and writing works as expected when mounted with s3fs-fuse.

[kahing]

goofys by default would use anonymous credentials if it thinks that the bucket is public-read, and you can use --profile default to override that. Is it intentional that this bucket has public LIST and "Read bucket permissions" but requires credentials to read the objects? What's the use case here?

[pkerpedjiev]

I get the same result when using --profile default:

ubuntu@ip-172-31-4-151:~/projects/cooler-benchmarks [master|!P]$ fusermount -u data-aws; goofys --profile default -o allow_other -f --debug_s3 hg:hg.io/media data-aws/
fusermount: entry for /home/ubuntu/projects/benchmarks/data-aws not found in /etc/mtab
2017/09/06 20:50:30.556712 s3.DEBUG HEAD https://s3.amazonaws.com/hg = 200 [us-east-1]
2017/09/06 20:50:30.556765 s3.INFO anonymous bucket detected

It actually wasn't intentional that this bucket has public LIST and while the objects have read permissions. I'll change that, but I still don't think this is the expected behavior for goofys. In general, if there's credentials, they should be used, no?

[kahing]

I don't disagree with that and PR welcome :-)

[kahing]

for now, I made --profile default work again

[pkerpedjiev]

Thanks, that seems to work!

And sorry for not submitting a PR. My go abilities are non-existent (for now).