some uploads failing due to client timeouts

[jbrzusto]

This is happening occasionally. The most recent case is for 3 large files uploaded from a Mac. The session began with this request:

sgdata:443 A.B.C.D - - [18/Feb/2018:15:42:12 +0000] "GET /upload/upload-from-computer.php HTTP/1.1" 302 7574 "https://motus.org/data/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.132 Safari/537.36"

• each file was uploaded successfully to /sgm/uploads/partial
• the client browser requested upload-process-form.php, which succeeded, along with the requests that page includes
• the next request from the client, 20 seconds after the above successful requests, timed out before the first request header was received:

sgdata:443 A.B.C.D - - [18/Feb/2018:18:38:01 +0000] "-" 408 137 "-" "-"

In case this is simply due to connection congestion (the upload speed here averaged ~ 60 kBytes / s), I've increased the timeouts for Apache in /etc/apache2/mods-enabled/reqtimeout.conf:

...
        # 2018-02-19: increase timeouts because uploads from clients with less reliable
        # internet access are sometimes failing with "408" codes.
        RequestReadTimeout header=60-120,minrate=100
        RequestReadTimeout body=60-120,minrate=100

[jbrzusto]

The problem is the ProjectSend is timing out in this situation:

• user selects several large files for upload
• user hits "continue"
• browser begins uploading files
• user (sensibly) walks away from computer, having better things to do than watch a slowly-incrementing percentage counter
• browser completes upload and presents project-selection page
• user returns, selects projects, and hits "Continue"
• server detects a timeout upon resubmission of the upload-process-form.php page, ignores the project assignments, and redirects the user back to the index.php page, typically with a forced re-login

Possible Fixes

• have user select project for each file before uploading begins.
  or
• disable keep-alives on project-selection page

A side benefit of the first approach is we can also calculate the hash of the file and check whether its a duplicate before (and possibly instead of) uploading it.

[jbrzusto]

We're going for none-of-the-above. The timeouts are caused by PHP's session management so we just need to increase PHP session timeouts, from 24 minutes to something larger, e.g. 24 hours:

So in /etc/php5/apache2/php.ini, we change:

session.gc_maxlifetime = 1440

to

session.gc_maxlifetime = 86400