DataBiosphere / dsub

Open-source command-line tool to run batch computing tasks and workflows on backend services such as Google Cloud.
Apache License 2.0
262 stars 44 forks source link

how to support fuse and fusera in dsub #273

Open YaCui opened 11 months ago

YaCui commented 11 months ago

Hi, dsub only includes --image to include a docker image, but I need to add more parameters with docker image to run fuse and fusera (as below)? How can I add these parameters?

sudo docker run --rm -it --privileged --cap-add SYS_ADMIN --cap-add MKNOD --device /dev/fuse fusera/fusera:alpine

Thanks!

wnojopra commented 11 months ago

Hi @YaCui,

Can you describe more about your use case? Do you need fuse in order to mount cloud storage? Are you hoping to do this on the local or google providers?

If you're hoping to mount cloud storage, with the google-v2 and google-cls-v2 providers you can use dsub's --mount flag to mount buckets. See Mounting a Google Cloud Storage bucket. The key thing about this flag is that it sets the enableFuse flag in the Lifesciences API which causes the docker to be run with CAP_SYS_ADMIN.

YaCui commented 11 months ago

Hi @wnojopra,

My use case is that I want to use fusera in the following cmds. But it always shows that fusera cannot find fuse although the fuse is installed. If I run docker as this (add some parameters): sudo docker run --rm -it --privileged --cap-add SYS_ADMIN --cap-add MKNOD --device /dev/fuse fusera/fusera:alpine. it works. But dsub seems cannot add these parameters.

dsub \ --provider google-v2 \ --project my-cloud-project \ --image 'marketplace.gcr.io/google/ubuntu1804:latest' \ --regions us-central1 \ --logging gs://my-bucket/logging/ \ --output OUT=gs://my-bucket/output/out.txt \ --command './fusera --help' \