haiwen / seafdav

Seafile webdav server
Other
84 stars 68 forks source link

syncing large files problematic, http code 499 in nginx logs #85

Closed laur89 closed 1 month ago

laur89 commented 2 months ago

Describe the bug Uploading large files (say ~2-3GB) from a phone to seafdav is oftentimes problematic. Sometimes files end up corrupted, and deletions fail to sync altogether.

To Reproduce

  1. create ~2GB file in local device
  2. try to upload said file to seafdav
  3. notice 499 http codes in nginx logs
  4. retry the process
  5. client detects conflicts
  6. (sometimes the original upload fails with uploaded file being smaller than expected, but that behavior is inconsistent)
  7. delete already synced file on local device
  8. sync the deletion with webdav remote
  9. notice deletion is not propagated to remote

Expected behavior Files creation & deletions synced properly, no 499 HTTP codes in nginx logs.

Relevant portion from nginx' seafdav.access.log from a sync session:

Manual sync was triggered at 15:20:00. Note the signal-2024-09... files are the large ones that cause problems.

- my.ip.address [23/Sep/2024:15:20:02 +0200] "PROPFIND /seafdav/phone-bcf01b/config/ HTTP/2.0" 401 139 "-" "OkHttp" 0.003
- my.ip.address [23/Sep/2024:15:20:02 +0200] "PROPFIND /seafdav/phone-bcf01b/config/ HTTP/2.0" 207 591 "-" "OkHttp" 0.046
- my.ip.address [23/Sep/2024:15:20:02 +0200] "PROPFIND /seafdav/phone-bcf01b/config/ HTTP/2.0" 207 2250 "-" "OkHttp" 0.059
- my.ip.address [23/Sep/2024:15:20:02 +0200] "PROPFIND /seafdav/phone-bcf01b/config/foldersync/ HTTP/2.0" 207 3761 "-" "OkHttp" 0.038
- my.ip.address [23/Sep/2024:15:20:02 +0200] "PROPFIND /seafdav/phone-bcf01b/config/signal/ HTTP/2.0" 207 2488 "-" "OkHttp" 0.059
- my.ip.address [23/Sep/2024:15:28:12 +0200] "PUT /seafdav/phone-bcf01b/config/signal/signal-2024-09-23-05-06-29.backup HTTP/2.0" 499 0 "-" "OkHttp" 490.165
- my.ip.address [23/Sep/2024:15:28:12 +0200] "PUT /seafdav/phone-bcf01b/config/signal/signal-2024-09-22-05-09-16.backup HTTP/2.0" 499 0 "-" "OkHttp" 490.274
- my.ip.address [23/Sep/2024:15:36:41 +0200] "PUT /seafdav/phone-bcf01b/config/signal/signal-2024-09-23-05-06-29.backup HTTP/2.0" 499 0 "-" "OkHttp" 508.055
- my.ip.address [23/Sep/2024:15:36:41 +0200] "PUT /seafdav/phone-bcf01b/config/signal/signal-2024-09-22-05-09-16.backup HTTP/2.0" 499 0 "-" "OkHttp" 508.087
- my.ip.address [23/Sep/2024:15:45:22 +0200] "PUT /seafdav/phone-bcf01b/config/signal/signal-2024-09-22-05-09-16.backup HTTP/2.0" 499 0 "-" "OkHttp" 520.007
- my.ip.address [23/Sep/2024:15:45:22 +0200] "PUT /seafdav/phone-bcf01b/config/signal/signal-2024-09-23-05-06-29.backup HTTP/2.0" 499 0 "-" "OkHttp" 520.080

seafdav.log:

15:20:02.033 - INFO    : 172.19.0.1 - (anonymous) - [2024-09-23 13:20:02] "PROPFIND /phone-bcf01b/config/" length=461, depth=0, elap=0.000sec -> 401 Not Authorized
15:20:02.085 - INFO    : 172.19.0.1 - my.email@gmail.com - [2024-09-23 13:20:02] "PROPFIND /phone-bcf01b/config/" length=461, depth=0, elap=0.044sec -> 207 Multi-Status
15:20:02.158 - INFO    : 172.19.0.1 - my.email@gmail.com - [2024-09-23 13:20:02] "PROPFIND /phone-bcf01b/config/" length=461, depth=1, elap=0.059sec -> 207 Multi-Status
15:20:02.210 - INFO    : 172.19.0.1 - my.email@gmail.com - [2024-09-23 13:20:02] "PROPFIND /phone-bcf01b/config/foldersync/" length=461, depth=1, elap=0.037sec -> 207 Multi-Status
15:20:02.288 - INFO    : 172.19.0.1 - my.email@gmail.com - [2024-09-23 13:20:02] "PROPFIND /phone-bcf01b/config/signal/" length=461, depth=1, elap=0.056sec -> 207 Multi-Status
15:28:13.306 - INFO    : 172.19.0.1 - my.email@gmail.com - [2024-09-23 13:28:13] "PUT /phone-bcf01b/config/signal/signal-2024-09-23-05-06-29.backup" length=2938778705, elap=490.978sec -> 201 Created
15:28:13.677 - INFO    : 172.19.0.1 - my.email@gmail.com - [2024-09-23 13:28:13] "PUT /phone-bcf01b/config/signal/signal-2024-09-22-05-09-16.backup" length=2935266922, elap=491.349sec -> 204 No Content
15:36:42.400 - INFO    : 172.19.0.1 - my.email@gmail.com - [2024-09-23 13:36:42] "PUT /phone-bcf01b/config/signal/signal-2024-09-23-05-06-29.backup" length=2938778705, elap=508.857sec -> 204 No Content
15:36:42.789 - INFO    : 172.19.0.1 - my.email@gmail.com - [2024-09-23 13:36:42] "PUT /phone-bcf01b/config/signal/signal-2024-09-22-05-09-16.backup" length=2935266922, elap=509.188sec -> 204 No Content
15:45:23.616 - INFO    : 172.19.0.1 - my.email@gmail.com - [2024-09-23 13:45:23] "PUT /phone-bcf01b/config/signal/signal-2024-09-22-05-09-16.backup" length=2935266922, elap=520.910sec -> 204 No Content
15:45:23.981 - INFO    : 172.19.0.1 - my.email@gmail.com - [2024-09-23 13:45:23] "PUT /phone-bcf01b/config/signal/signal-2024-09-23-05-06-29.backup" length=2938778705, elap=521.330sec -> 204 No Content

Environment:

Where exactly to run wsgidav command? Running it from docker container doesn't find it on our PATH.

Insert output of `wsgidav --version -v`.

Which WSGI server was used (cheroot, ext-wsgiutils, gevent, gunicorn, paste, uvicorn, wsgiref, ...)?

Unsure, but seafile is running on jammy-1.0.2 image, relevant dependencies installed by this statement

WebDAV client: FolderSync Pro 4.0.1 running on Android 14.

Additional context