Per-user storage disks

[mzur]

Users could create their own storage disks on the fly. This enables "private" volume locations, in contrast to "public" remote volumes. Some ideas:

• In the user settings, there is a new section "Storage Disks"
• Users can create e.g. a new AWS S3 storage disk with their access credentials there. Credentials are encrypted.
• When users create a new volume, they can choose from their storage disks or a remote location
• If a storage disk is selected for a new volume, the volume file browser is activated
• Volumes are always connected to a storage disk (even it it is the special "remote" storage disk), the disk is no longer encoded in the volume URL
• Images/videos are loaded through the storage disk.
  • :warning: This means that all users who can access a volume, can access the files with credentials of another user.
  • Storage disks are generated on the fly for access, cf.: https://stackoverflow.com/a/66839449/1796523
• Possible supported storage disk types (PHP Flysystem adapters): S3, Swift, Elements, Dropbox, FTP...
• If a storage disk type supports it, use temporary public URLs to load images (if the user activates this feature for the storage disk). This provides the same advantages than public remote volumes regarding the loading speed of files from a nearby server instead of the BIIGLE server.

We need to discuss the security implications of the use of access credentials of different users.

Related to https://github.com/biigle/core/issues/373

[mzur]

This could complement the new user storage module nicely. Storage disks are also generated on the fly in this module (e.g. user-123://). Custom storage disks mentioned here could be stored as database model with an ID. These can be identified via the disk-[id]:// prefix. There can be global storage disks that are (only) available to all instance admins.

[mzur]

This will be implemented as part of the interface between BIIGLE and the Aruna Object Storage of NFDI4Biodiversity.

[mzur]

There is no way around storing encrypted access credentials in our DB. We may have to update the privacy notice, terms (no liability in case of leaked credentials) and display appropriate warning messages.

[mzur]

Todos:

• [x] Create a new biigle/user-disks module.
• [x] Implement a new UserDisk database model (with an ID, type, user ID and encrypted JSON field for credentials).
• [x] Implement a new user settings view to CRUD UserDisks. Show appropriate warning messages about access credentials when a new UserDisk is created.
• [x] Show a message "the disk is still used by x volumes" (if x>0) when a disk should be deleted.
• [x] Integrate UserDisks in the create volume form (just add them to the regularly visible disks for the user if the module is present) in biigle/core.
• [x] Implement a new "resolver" for the storage disks (similar to what biigle/user-storage does). This must be compatible with biigle/user-storage which replaces the entire FilesystemManager. Maybe implement a new FilesystemManager in biigle/core which supports the addition of "resolvers" through modules.
• [x] Implement UserDisks as read-only disks
• [x] Take care of what happens if a UserDisk was deleted or expires, even though volumes exist that use it. Show an appropriate warning message in the annotation tools?
• [x] Implement support for S3 UserDisks.
• [x] Serve images from storage disks via temporary public URLs, too (similar to videos) if possible.
• [x] Create a flow for automatic expiration of UserDisks. This means that the user disk is deleted after e.g. 6 months unless the disk is still used or the user renews the disk (similar to storage requests). This is done to not accumulate access credentials of disks that are not used any more. Users get warning notifications before a disk expires.
• [x] Write a manual article (include detailed instructions for each storage disk type, include CORS instructions).
• [x] Prepare an update to the terms and privacy notice. (https://github.com/biigle/biigle.de/pull/85)
• [ ] Implement support for Aruna Object Storage UserDisks. (see https://github.com/biigle/user-disks/issues/1)