search

elixir-cloud-aai / TESK

GA4GH Task Execution Service Root Project + Deployment scripts on Kubernetes
https://tesk.readthedocs.io
Apache License 2.0
39 stars 29 forks source link

feat: add FUSE support #199

Open athith-g opened 4 months ago

athith-g commented 4 months ago

There are instances where using a FUSE is useful/essential for the completion of a TES task. For example, for tasks that use crypt4gh-encrypted files, using crypt4ghfs would allow the files to remain encrypted at rest and only be decrypted during read operations, greatly enhancing security.

Currently, using a FUSE in Docker requires the use of --cap-add SYS_ADMIN or --privileged flags, per this discussion. As such, I was able to use cryptgh4fs in a standalone docker container, but not in a TES implementation.

Ideally, a user would specify an image with a FUSE and specify a mount point to mount the FUSE in a TES request, and file operations in all the executors would be done with the FUSE. However, allowing individual executors to run in privileged mode could also be useful.

lvarin commented 3 months ago

It is an interesting idea. I will check and prepare a demo of a crypt4gh FUSE TESK server.

I did some checks on my laptop and it works well, but it was a bit complicated to install. I also saw that the standalone crypt4gh command exists with decrypt and encrypt options and it was much easier to install. I wonder if another option to consider for achieving this is to use an innitContainer and an emptyDir using memory instead of storage.