We got the idea to implement a new module for self-service user storage. This would allow users to upload files in BIIGLE and use them for new volumes. This is the concept for now:
A prominent notice/button is added to the create volume form that redirects users to a new page where they can create "storage requests". Maybe implement this as button group: "remote files" opens the URL input field, "uploaded files" opens the volume file browser, "upload new files" redirects to the page for a new storage request.
The users can upload files as part of the storage request. Once all files were uploaded, the storage request is submitted.
Each new storage request must be reviewed by instance admins (to prevent the upload of undesired files). They get an email (similar to user registration confirmations) and can review the individual files if they want.
If a storage request is approved, the user is notified that the files are available for use. They can conveniently choose any approved files in the volume file browser for new volumes. The files of rejected storage requests are deleted (and the user is notified with a "reason").
Each user has a storage quota (e.g. 50 GB). Files of both pending and approved storage requests are counted. A storage request that exceeds the quota is denied.
Exceptions can be made for individual users who require an increased quota.
Users can delete their uploaded files in a dedicated view. They can either delete individual files or whole storage requests (and all associated files). More operations such as move/rename or even a simple download should not be provided. It should be discouraged to use BIIGLE as a primary data storage for the files.
If a user is deleted, all files associated with their storage requests are deleted as well (if they still exist). I.e. the user directory is deleted in the two storage disks (see below).
Storage requests have an expiration date (e.g. 12 months). Users get a notification 1 month before expiration, 1 week before expiration and when the request is expired (explaining that they have a grace period of 1 month). Storage requests (and all associated files) are automatically deleted 1 month after expiration.
If all files associated with the storage request were already deleted, the storage request is silently deleted 1 month before expiration (when the notification should be sent).
Users can apply for an extension of the expiration date of a storage request (by another 12 months). Admins get an email to confirm or deny the expiration.
The whole process must be easy for users! Maybe new storage requests can be coupled with the creation of a new volume? If the request is approved, the volume is automatically created.
Technical details:
Storage requests are implemented as new models that are stored in the DB and associated with the user who created them.
Two storage disks can be provided for the storage request. One stores the pending requests and one stores the approved files. Only the second disk is available in the volume file browser.
Files of an approved storage request are moved from one disk to the other.
Storage for individual users can be implemented with "virtual" storage disks. Files of a user are stored in a directory (e.g. the user ID). The virtual storage disk is configured with the directory prefix, so only files of the user are visible.
The upload quota is set as a config value. It can be overridden with a value in the user attrs. The value can be updated by global admins via an API request.
The used storage is also stored in the user attrs. All file operations (upload/delete) modify this value.
The upload does a MIME type check to only allow compatible image/video files.
A storage request consists of one or more directories of files. Users can add a (virtual) directory in the UI to create the storage request and then select the files that belong to the directory. Many directories can be added this way. Initially, it must be checked if any of the directory names are already used in the user's storage. Each file is uploaded with the directory name as prefix and stored in that directory in the user's storage disk.
Add instructions to configure upload_tmp_dir (maybe also open_basedir), post_max_size and upload_max_filesize for large file uploads (e.g. up to 10 GB per file). The temporary files must be stored on a disk with enough space. Configurable maximum file size validation is separate from the PHP ini configuration.
We got the idea to implement a new module for self-service user storage. This would allow users to upload files in BIIGLE and use them for new volumes. This is the concept for now:
The whole process must be easy for users! Maybe new storage requests can be coupled with the creation of a new volume? If the request is approved, the volume is automatically created.
Technical details:
attrs
. The value can be updated by global admins via an API request.attrs
. All file operations (upload/delete) modify this value.upload_tmp_dir
(maybe alsoopen_basedir
),post_max_size
andupload_max_filesize
for large file uploads (e.g. up to 10 GB per file). The temporary files must be stored on a disk with enough space. Configurable maximum file size validation is separate from the PHP ini configuration.