Files: Downloading files directly to disk

[ChrisSchinnerl]

The UI currently downloads files first to RAM as a browser blob, that is then downloaded/saved from the browser into user storage. Move to regular browser downloads by adding alternative auth that works when opening a new link, or figure out a direct async stream.

• https://github.com/jimmywarting/StreamSaver.js
• https://github.com/whatwg/fs

[alexfreska]

Took another look at the issue. We want to be able to download files directly to disk but because we use a header for auth we cannot rely on a regular browser link download and instead use JavaScript to first download the file into memory as a work around.

The streamsaver library uses an https service workers to simulate an endpoint, but for http (which we use most often) requires hitting an external https MITM proxy endpoint which creates a point of centralization and does not work for us.

@ChrisSchinnerl @peterjan the best solution for us is allowing the auth token to be passed as a query parameter, in general, or at least just for the download endpoint. This would allow us to do simple native browser downloads directly to disk. We just need to ensure the token is handled safely, ie: not logged. As an example, I noticed the CCRI API we are working with supports header or query parameter for their token.

[peterjan]

One-time tokens might be a good option? The UI could request a download token and we have a middleware that authenticates download requests if it has a valid token in the query string. It might also make sense to research presigned URLs, because iirc that's what S3 provides for both uploads and downloads.

[alexfreska]

Presigned URLs is a great solution for this, and would be a good additional feature for our S3 support. The only issue with that is that the user has S3 disabled they could not download files from the UI.

Our own download token system could also work, maybe it could somehow use the same underlying code as presigned S3 urls.

Adding a query support to our auth header check is probably the simplest.

[ChrisSchinnerl]

Presigned URLs is a great solution for this, and would be a good additional feature for our S3 support. The only issue with that is that the user has S3 disabled they could not download files from the UI.

Our own download token system could also work, maybe it could somehow use the same underlying code as presigned S3 urls.

Adding a query support to our auth header check is probably the simplest.

I agree. Having a query param that we check if we can't find the header is probably the most straightforward way to do it short term. If we wanted S3-style pre-signed URLs we need to do more research first but it seems like it is similar to the v4 signing we already support with the difference that the information is added as query params.