Support for / protection against huge downloads

[steveandroulakis]

The current download scheme involves creating a copy of the archive that the user wants to download in a temporary file. If the user wants to download a truly massive collection of datafiles, this scheme is going to fail because there won't be enough temporary file space. A second problem is that the larger a download is, the greater the chance it will fail to complete due to network failures, server restarts, browser restarts and so on. Finally, requesting a lot of huge downloads is an easy way to implement a denial of service attack ...

Here are some things we could do to address these issues:

• Rather than writing the archive to a temporary file, use a fixed size in-memory buffer.
• Implement restartable downloads. (I'm not sure how ...)
• Implement download size limits depending on user identity / priviledges.
• Implement system-wide download throttling; e.g. limit the number of simultaneous downloads.

original LH ticket

This ticket has 0 attachment(s).

[steveandroulakis]

Support for / protection against huge downloads

(For the record, we currently have a configurable upper limit on the size of a download. But this is a one-size-fits-all limit, independent of user identiy and privileges.)

by Stephen Crawley