Data Upload Component

[tdipisa]

A dedicated service must be provided for files upload, the following need to be ensured:

• [x] REST API
• [ ] Asynchronous upload (with the possibility to see the status)
• [ ] Ability to pull (asynchronously) from external URLs (HTTP, FTP,,) as well as from well known sources like dropbox, box, google drive (we don't need them all for the moment).
• [ ] Only allow specific file extensions and verify files type (from configuration)
• [ ] Set a maximum length name for files (from configuration)
• [ ] Configurable max size of uploaded files
• [ ] Configurable parallelism of global uploads.
• [ ] Configurable parallelism of per-user uploads (nice to have)
• [ ] Configurable quota for uploaded files (ability to configure the max file size and limit the storage capacity)
• [ ] Pluggable storage system (the storage type can be configured in the backend, for a first implementation a FS storage is enough but we need to have in mind object storage)
• [ ] Allow file uploads only to authenticated users (TBD, see #8)

[simboss]

We need to make sure we do not et users overwhelme the server with uploads.

I know there are modules fr Django as well as for NGINX that we can reuse that can help with some of the requirements above. Notice that for a first version we do not need to address them all, but for this discussion we have to take them all into account.