danilop / yas3fs

YAS3FS (Yet Another S3-backed File System) is a Filesystem in Userspace (FUSE) interface to Amazon S3. It was inspired by s3fs but rewritten from scratch to implement a distributed cache synchronized by Amazon SNS notifications. A web console is provided to easily monitor the nodes of a cluster.
http://danilop.github.io/yas3fs
MIT License
643 stars 98 forks source link

Trouble Mounting S3 Bucket Inside Docker Container #185

Closed echyam closed 5 years ago

echyam commented 5 years ago

Hello

I can mount a bucket within an ubuntu EC2 that has an IAM role assigned to it. But when I try to create a docker container hosted by the same EC2, I am unsuccessful.

I copied the steps from README into both my Dockerfile (also tried entrypoint):

sudo apt-get update
sudo apt-get -y install fuse python-pip 
sudo pip install yas3fs
sudo sed -i'' 's/^# *user_allow_other/user_allow_other/' /etc/fuse.conf
sudo chmod a+r /etc/fuse.conf
yas3fs -h # See the usage
mkdir /path/to/mount-folder

And then to mount

yas3fs s3://bucket-name /path/to/mount-folder

This is my output:

...
MainThread 2019-06-05T22:28:34.236 INFO Multipart maximum number of retries per part: '3'
MainThread 2019-06-05T22:28:34.236 INFO Default expiration for signed URLs via xattrs: '2592000'
MainThread 2019-06-05T22:28:34.236 INFO S3 Request Payer: 'False'
MainThread 2019-06-05T22:28:34.236 INFO Cache path (on disk): '/tmp/yas3fs-bucket-name'
MainThread 2019-06-05T22:28:34.277 INFO Unique node ID: 'node-id'
fuse: device not found, try 'modprobe fuse' first
MainThread 2019-06-05T22:28:34.278 ERROR Uncaught Exception: <class 'RuntimeError'> 1 <traceback object at 0x7f86d456eec8>
NoneType: None

I know it is not an AWS credentials error, because I can use the awscli to look inside my bucket. Any help with debugging would be appreciated! Thanks!

jazzl0ver commented 5 years ago

you need to load fuse module inside your docker container:

fuse: device not found, try 'modprobe fuse' first

i don't know how to do that, but i'm sure google does

echyam commented 5 years ago

I was trying to run the container within another service, and I read on stackoverflow that the containers have to be run with privileged mode with docker run --privileged <image>. For some reason that service wasn't allowing it, but a docker container outside the service was fine, so it's not an issue with yas3fs. Thanks!