data folders

[calys]

Having files "buried" inside projects is sometimes inconvenient when it comes to using scripts and/or notebooks. Many users like to have a place where they put files (scripts, notebooks, CAD files) that are common to several of their projects.

Suggested feature:

private "data folder", at the user level, like "My Documents":

• contains data (not projects/studies) - typically scripts and/or CAD files
• belongs to a single user
• accessible by all projects (e.g. automatically "mounted")

private "data folder", at the organization level, like "Public Documents":

• contains data (not projects/studies) - typically scripts and/or CAD files
• belongs to / shared by all users of an organization
• accessible by all projects of all users of that organization (e.g. automatically "mounted")

  ### Tasks
  - [ ] https://github.com/ITISFoundation/osparc-issues/issues/954
  - [ ] Investigate if rclone mount can be used in the dy-sidecar withou the use of docker volumes @GitHK

[mrnicegyu11]

Open Technical Questions after discussion @GitHK @mrnicegyu11 :

• Can the total size and/or number of files in a rclone mount be limited?
• This might translate to: Have a shared mounted S3 per group
• How would we handle the share for the everyone group?
• How are computational services and legacy services handled?
• Needs to be tested: Concurrent file opening/access
• A quick way to add this (without a docker volume plugin) would be to add SYS_ADMIN caps (or others), which is of course bad, but since the sidecar already mounts the docker socket there is no actual additional privileged added. With SYS_ADMIN caps, the sidecar can use rclone mount itself

FYI: I will unassign myself since for now this is not actionable to DevOps, it needs new backend code :) Feel free to re-add me once DevOps comes into play.

[GitHK]

references:

• for rclone mount permissions https://hub.docker.com/r/mumiehub/rclone-mount/
• parsing logs for information about progress https://forum.rclone.org/t/printing-vfs-cache-stats-to-log-file/31234/10