search

nextcloud / server

☁️ Nextcloud server, a safe home for all your data
https://nextcloud.com
GNU Affero General Public License v3.0
27.16k stars 4.03k forks source link

Bulk Upload - Computed md5 hash is incorrect #29984

Open apg1980 opened 2 years ago

apg1980 commented 2 years ago

mariadb 10.5.13 php8.0.13 Apache ubuntu 20.04 lts

i got some of this messages in the log after upgrade:#{"reqId":"57SS2UgAK0IhKBiUN97i","level":3,"time":"2021-11-30T14:44:25+01:00","remoteAddr":"----","user":"----","app":"no app in context","method":"POST","url":"/remote.php/dav/bulk","message":"Computed md5 hash is incorrect.","userAgent":"Mozilla/5.0 (Windows) mirall/3.4.0stable-Win64 (build 20211129) (Nextcloud, windows-10.0.22000 ClientArchitecture: x86_64 OsArchitecture: x86_64)","version":"23.0.0.10","id":"61a636a484c44"}

szaimen commented 2 years ago

cc @artonge @nextcloud/desktop

Bojan023 commented 2 years ago

Can confirm this bug:

{"reqId":"dutAwDOv0zC9LF010aJv","level":3,"time":"2022-01-31T09:26:41+00:00","remoteAddr":"xxx.xxx.xxx.xxx","user":"user","app":"no app in context","method":"POST","url":"/remote.php/dav/bulk","message":"Computed md5 hash is incorrect.","userAgent":"Mozilla/5.0 (Windows) mirall/3.4.1stable-Win64 (build 20211221) (Nextcloud, windows-10.0.19042 ClientArchitecture: x86_64 OsArchitecture: x86_64)","version":"23.0.0.10","id":"61f7ac82811a4"}

AndyXheli commented 2 years ago

I can confirm this bug also on NC 23.0.0

AndyXheli commented 2 years ago

I moved back to Desktop client 3.6 and no more issues.

fightforlife commented 2 years ago

Same issue on nc23 and windows client 3.4.0

mgallien commented 2 years ago

can somebody test if this PR is fixing this bug https://github.com/nextcloud/server/pull/30999 ?

szaimen commented 1 year ago

Hi, please update to 24.0.8 or better 25.0.2 and report back if it fixes the issue. Thank you!

AlektroNik commented 1 year ago

Hi, please update to 24.0.8 or better 25.0.2 and report back if it fixes the issue. Thank you!

Hi. On 25.0.2 + client 3.6.4 (Windows) the same problem.

[no app in context] Ошибка: Computed md5 hash is incorrect.
POST /remote.php/dav/bulk
at 2023-01-16T15:38:10+00:00
SunnyPaprika commented 1 year ago

Same with 25.0.3

{"reqId":"Ko73h5uiNcnTOsVYCHdY","level":3,"time":"2023-01-19T12:28:45+00:00","remoteAddr":"XXXXXX","user":"XXXXXX","app":"no app in context","method":"POST","url":"/remote.php/dav/bulk","message":"Computed md5 hash is incorrect.","userAgent":"Mozilla/5.0 (Windows) mirall/3.6.4stable-Win64 (build 20221208) (Nextcloud, windows-10.0.19044 ClientArchitecture: x86_64 OsArchitecture: x86_64)","version":"25.0.3.2","data":[]}

Windows Desktop Client GUI becomes really slow and says "network error 99" for a lot of files.

I found this in my postgreSQL log.

Jan 19 13:28:44 server postgres[2228472]: 2023-01-19 13:28:44.965 CET [2228472] ERROR:  duplicate key value violates unique constraint "oc_file_metadata_pkey"
Jan 19 13:28:44 server postgres[2228472]: 2023-01-19 13:28:44.965 CET [2228472] DETAIL:  Key (id, group_name)=(33821, size) already exists.
Jan 19 13:28:44 server postgres[2228472]: 2023-01-19 13:28:44.965 CET [2228472] STATEMENT:  INSERT INTO "oc_file_metadata" ("group_name", "id", "metadata") VALUES($1, $2, $3)
allexzander commented 1 year ago

@mgallien ^^

x29a commented 1 year ago

Faced the issue as well on Nextcloud 25.0.2 with Windows Desktop Client 3.4.1, reverted back to 3.3.6 and files sync.

Now the log shows issue #34958 but seems to work anyway.

Hope this helps to narrow down the problem.

lukasszz commented 1 year ago

The same problem 25.0.3.2

"method":"POST",
"url":"/remote.php/dav/bulk",
"message":"Computed md5 hash is incorrect.",
"userAgent":"Mozilla/5.0 (Windows) mirall/3.4.1stable-Win64 (build 20211221) (Nextcloud, windows-10.0.22000 ClientArchitecture: x86_64 OsArchitecture: x86_64)","version":"25.0.3.2","data":[]}
Nextcloud version | 25.0.3.2
HTTPD service | up
PHP service | up
MariaDB service | up
Redis service | up
Blisk commented 1 year ago

I have the same error on 26.0.11 [no app in context] Error: Computed md5 hash is incorrect.

POST /remote.php/dav/bulk

MichaKersloot commented 1 year ago

Problem still not solved?

{"reqId":"xxx","level":3,"time":"2023-03-30T20:03:03+00:00","remoteAddr":"xxx","user":"xxxl","app":"no app in context","method":"POST","url":"/remote.php/dav/bulk","message":"Computed md5 hash is incorrect.","userAgent":"Mozilla/5.0 (Windows) mirall/3.7.4stable-Win64 (build 20230309) (Nextcloud, windows-10.0.22621 ClientArchitecture: x86_64 OsArchitecture: x86_64)","version":"25.0.5.1","data":[],"id":"xxx"}

AlektroNik commented 1 year ago

Problem still not solved?

{"reqId":"xxx","level":3,"time":"2023-03-30T20:03:03+00:00","remoteAddr":"xxx","user":"xxxl","app":"no app in context","method":"POST","url":"/remote.php/dav/bulk","message":"Computed md5 hash is incorrect.","userAgent":"Mozilla/5.0 (Windows) mirall/3.7.4stable-Win64 (build 20230309) (Nextcloud, windows-10.0.22621 ClientArchitecture: x86_64 OsArchitecture: x86_64)","version":"25.0.5.1","data":[],"id":"xxx"}

Try setting the Download and upload speed to the server as "Unlimited" in the Nextcloud client settings.

MichaKersloot commented 1 year ago

Hi, took some time to check, but those settings are already on 'Unlimited' . I've found the setting: 'bulkupload.enabled' => 'false', and this seems (no surprise) to solve this.

AndyXheli commented 1 year ago

Still an issue NC 25.0.5

{"reqId":"EiwS9y875QXwQMSUzAMv","level":3,"time":"2023-03-26T14:59:15-05:00","remoteAddr":"89.9...7.7","user":"admin","app":"no app in context","method":"POST","url":"/remote.php/dav/bulk","message":"Computed md5 hash is incorrect.","userAgent":"Mozilla/5.0 (Windows) mirall/3.6.6stable-Win64 (build 20230119) (Nextcloud, windows-10.0.19045 ClientArchitecture: x86_64 OsArchitecture: x86_64)","version":"25.0.4.1","data":[],"id":"643961564fe7e"}

labor4 commented 1 year ago

can we rule out, this doesn't fix the md5 problem? (transaction_isolation = READ-COMMITTED)

MichaKersloot commented 1 year ago

No clue. I'm using Nextcloud AIO and that seems to pull postgresql

labor4 commented 1 year ago

No clue. I'm using Nextcloud AIO and that seems to pull postgresql

the postgresql equivalent would be default_transaction_isolation = 'read committed'

in postgresql.conf

szaimen commented 1 year ago

The default is read comitted in postgresql apparently: https://postgresqlco.nf/doc/en/param/default_transaction_isolation/

c-schulz-rd13 commented 1 year ago

Still an issue here as well!! NC 25.0.05 MariaDB with (transaction_isolation = READ-COMMITTED) (setting verified via 'mysqladmin variables') On Mac, Docker Desktop, NC docker-compose

I also see problems related to this issue.

AndyXheli commented 1 year ago

Still an issue on NC 26.0.2

{"reqId":"wLOnG3yVfgfWFsfSOGuI","level":3,"time":"2023-06-08T15:55:37-05:00","remoteAddr":"192.168.254.190","user":"admin","app":"no app in context","method":"POST","url":"/remote.php/dav/bulk","message":"Computed md5 hash is incorrect.","userAgent":"Mozilla/5.0 (Windows) mirall/3.6.6stable-Win64 (build 20230119) (Nextcloud, windows-10.0.19045 ClientArchitecture: x86_64 OsArchitecture: x86_64)","version":"26.0.2.1","data":[],"id":"64834c55551d1"}

sjash commented 1 year ago

@AndyXheli I agree the issue still persist with version NC 26.0.2 and NC 26.0.3. I'm using desktop client version 3.9 and I have configured transaction_isolation = READ-COMMITTED` and 'bulkupload.enabled' => 'false' . And I still get this error.

{"reqId":"2ZozmYPDiPZgK5Adt1iN","level":3,"time":"2023-07-03T15:55:56+05:30","remoteAddr":"10.20.4.25","user":"s.jash","app":"no app in context","method":"POST","url":"/remote.php/dav/bulk","message":"Computed md5 hash is incorrect.","userAgent":"Mozilla/5.0 (Windows) mirall/3.9.0stable-Win64 (build 20230613) (Nextcloud, windows-10.0.19044 ClientArchitecture: x86_64 OsArchitecture: x86_64)","version":"26.0.3.2","data":[]}

Sieboldianus commented 1 year ago

The first time I saw these errors was today, when adding Zotero storage folder to nextcloud. The folder contained about 15000 files called .zotero-ft-cache and all of these would trigger the Computed md5 hash error in the server logs.

antonol68 commented 1 year ago

Still an issue on NC 27.0.1

{"reqId":"V5iUwTOsnuMg5Rf2d5Ou","level":3,"time":"2023-08-07T08:59:41+00:00","remoteAddr":"10.101.96.64","user":"user","app":"no app in context","method":"POST","url":"/remote.php/dav/bulk","message":"Computed md5 hash is incorrect.","userAgent":"Mozilla/5.0 (Windows) mirall/3.4.1stable-Win64 (build 20211221) (Nextcloud, windows-10.0.19045 ClientArchitecture: x86_64 OsArchitecture: x86_64)","version":"27.0.1.2","data":[],"id":"64d0b29bd7660"}

Cl00e9ment commented 1 year ago

So there is a hash mismatch between what the client computes and what the server computes. This means that it could also be a client bug. Seeing that all reports show a Windows user agent, I suspect that this a Windows client bug.

Also, on my NC instance, if I remember correctly, before disabling bulk upload with bulkupload.enabled' => 'false', it was a Windows PC that failed to synchronize and was creating those errors in the server logs.

ganglere commented 1 year ago

I'm having the same issue on a fresh install of NC AIO 27.0.2.1. Setting 'bulkupload.enabled' => 'false', did not work, but downgrading the Windows client to 3.3.6 did.

mgallien commented 1 year ago

if there is still an issue when running latest server and desktop client releases, it cannot fixed without providing server and client logs fixes were https://github.com/nextcloud/desktop/pull/5871 and https://github.com/nextcloud/server/pull/39221 as you can see one should run the 3.9.1 client release and any of the latest server releases including the PR or a backport of it

kesselb commented 11 months ago

Found in a customer log:

{
  "reqId": "",
  "level": 3,
  "time": "October 31, 2023 23:48:51",
  "remoteAddr": "",
  "user": "",
  "app": "no app in context",
  "method": "POST",
  "url": "/remote.php/dav/bulk",
  "message": "Computed md5 hash is incorrect.",
  "userAgent": "Mozilla/5.0 (Windows) mirall/3.10.0stable-Win64 (build 20230915) (Nextcloud, windows-10.0.22621 ClientArchitecture: x86_64 OsArchitecture: x86_64)",
  "version": "27.1.2.2",
  "data": []
}
ostasevych commented 8 months ago

First time found with NC 27.1.6, client version Win 3.11.0.

patrikroy commented 8 months ago

Since a few days, and after syncing a fair amount of files, I got the same error on a NC 28.0.2, client version 3.11.1. Not sure if it is related, but the client computer went to sleep during the sync, and never managed to recover from it - client froze even after reboot. Had to reinstall it to continue the sync.

{
  "reqId": "",
  "level": 3,
  "time": "2024-02-04T07:48:15+00:00",
  "remoteAddr": "192.168.0.254",
  "user": "",
  "app": "no app in context",
  "method": "POST",
  "url": "/remote.php/dav/bulk",
  "message": "Computed md5 hash is incorrect.",
  "userAgent": "Mozilla/5.0 (Windows) mirall/3.11.1stable-Win64 (build 20240125) (Nextcloud, windows-10.0.22631 ClientArchitecture: x86_64 OsArchitecture: x86_64)",
  "version": "28.0.2.5",
  "data": [],
  "id": ""
}

[EDIT] : after ~10 occurrences in the logs, the error is no longer being reported.

melroy89 commented 7 months ago

Same here. Using the latest Nextcloud server version (28.0.3). However with a old Nextcloud desktop client version (in my case v3.4.2), so make sure to update also the desktop client! Latest version of the client is version 3.12.0 at the time of writing.

{"reqId":"xLZ2NZarido7s5FtFEPC","level":3,"time":"2024-03-01T18:54:27+00:00","remoteAddr":"our_local_ip_address","user":"secret_user","app":"no app in context","method":"POST","url":"/remote.php/dav/bulk","message":"Computed md5 hash is incorrect.","userAgent":"Mozilla/5.0 (Linux) mirall/3.4.2-1ubuntu1 (Nextcloud, linuxmint-6.5.0-21-generic ClientArchitecture: x86_64 OsArchitecture: x86_64)","version":"28.0.3.2","data":[]}
{"reqId":"wZ4LpjIs1N4wehX7Tx9u","level":3,"time":"2024-03-01T18:54:27+00:00","remoteAddr":"our_local_ip_address","user":"secret_user","app":"no app in context","method":"POST","url":"/remote.php/dav/bulk","message":"Computed md5 hash is incorrect.","userAgent":"Mozilla/5.0 (Linux) mirall/3.4.2-1ubuntu1 (Nextcloud, linuxmint-6.5.0-21-generic ClientArchitecture: x86_64 OsArchitecture: x86_64)","version":"28.0.3.2","data":[]}
{"reqId":"tjvJfaWro8CT2JgNWdTl","level":3,"time":"2024-03-01T18:54:27+00:00","remoteAddr":"our_local_ip_address","user":"secret_user","app":"no app in context","method":"POST","url":"/remote.php/dav/bulk","message":"Computed md5 hash is incorrect.","userAgent":"Mozilla/5.0 (Linux) mirall/3.4.2-1ubuntu1 (Nextcloud, linuxmint-6.5.0-21-generic ClientArchitecture: x86_64 OsArchitecture: x86_64)","version":"28.0.3.2","data":[]}
{"reqId":"o4RvE3DEH3XiitHlNHYd","level":3,"time":"2024-03-01T18:54:27+00:00","remoteAddr":"our_local_ip_address","user":"secret_user","app":"no app in context","method":"POST","url":"/remote.php/dav/bulk","message":"Computed md5 hash is incorrect.","userAgent":"Mozilla/5.0 (Linux) mirall/3.4.2-1ubuntu1 (Nextcloud, linuxmint-6.5.0-21-generic ClientArchitecture: x86_64 OsArchitecture: x86_64)","version":"28.0.3.2","data":[]}
{"reqId":"5pSe5wNcUcrjsq2iyanI","level":3,"time":"2024-03-01T18:54:28+00:00","remoteAddr":"our_local_ip_address","user":"secret_user","app":"no app in context","method":"POST","url":"/remote.php/dav/bulk","message":"Computed md5 hash is incorrect.","userAgent":"Mozilla/5.0 (Linux) mirall/3.4.2-1ubuntu1 (Nextcloud, linuxmint-6.5.0-21-generic ClientArchitecture: x86_64 OsArchitecture: x86_64)","version":"28.0.3.2","data":[]}
{"reqId":"Zzu0HJoe9cr7PEczAbEI","level":3,"time":"2024-03-01T18:54:28+00:00","remoteAddr":"our_local_ip_address","user":"secret_user","app":"no app in context","method":"POST","url":"/remote.php/dav/bulk","message":"Computed md5 hash is incorrect.","userAgent":"Mozilla/5.0 (Linux) mirall/3.4.2-1ubuntu1 (Nextcloud, linuxmint-6.5.0-21-generic ClientArchitecture: x86_64 OsArchitecture: x86_64)","version":"28.0.3.2","data":[]}
{"reqId":"rkS6KUhnArwLaTa9NNem","level":3,"time":"2024-03-01T18:54:28+00:00","remoteAddr":"our_local_ip_address","user":"secret_user","app":"no app in context","method":"POST","url":"/remote.php/dav/bulk","message":"Computed md5 hash is incorrect.","userAgent":"Mozilla/5.0 (Linux) mirall/3.4.2-1ubuntu1 (Nextcloud, linuxmint-6.5.0-21-generic ClientArchitecture: x86_64 OsArchitecture: x86_64)","version":"28.0.3.2","data":[]}
{"reqId":"JtLjaDOGpI6TaNLVK4aP","level":3,"time":"2024-03-01T18:54:28+00:00","remoteAddr":"our_local_ip_address","user":"secret_user","app":"no app in context","method":"POST","url":"/remote.php/dav/bulk","message":"Computed md5 hash is incorrect.","userAgent":"Mozilla/5.0 (Linux) mirall/3.4.2-1ubuntu1 (Nextcloud, linuxmint-6.5.0-21-generic ClientArchitecture: x86_64 OsArchitecture: x86_64)","version":"28.0.3.2","data":[]}
ItsMeRitch commented 6 months ago

Problem still not solved? {"reqId":"xxx","level":3,"time":"2023-03-30T20:03:03+00:00","remoteAddr":"xxx","user":"xxxl","app":"no app in context","method":"POST","url":"/remote.php/dav/bulk","message":"Computed md5 hash is incorrect.","userAgent":"Mozilla/5.0 (Windows) mirall/3.7.4stable-Win64 (build 20230309) (Nextcloud, windows-10.0.22621 ClientArchitecture: x86_64 OsArchitecture: x86_64)","version":"25.0.5.1","data":[],"id":"xxx"}

Try setting the Download and upload speed to the server as "Unlimited" in the Nextcloud client settings.

Can confirm I was having this issue with MacOS client 3.12.3 whilst using a VPN to the nextcloud server. To fix, I force quit the client, start it, paused the sync before it could start and freeze and then went into the network tab and set it to "No Limit". Sync immediately worked once resumed.

melroy89 commented 6 months ago

This is not good, correct? This can cause various sync issues? I really hope somebody knows how to fix this.

ItsMeRitch commented 6 months ago

This is not good, correct? This can cause various sync issues? I really hope somebody knows how to fix this.

Did removing the limits not help with your issue?

melroy89 commented 6 months ago

I didn't had any network limits set to begin with. ..

frlan commented 3 months ago

I still can see it here too. NC 29.0.3 and Client 3.11 on Debian with PHP 8.1

NorbiPeti commented 3 weeks ago

I got this issue with the client completely freezing each time it attempts to upload some files. Removing the upload limit seems to fix this in my case (thanks for that). I turned off the automatic upload limit in nextcloud.cfg (useUploadLimit=0 instead of -1). Pausing from the config also works to get the client running in any case.

Running Nexrtcloud Hub 8 (29.0.7), I updated while this issue was present (to this version) and that didn't solve the issue. Also updated the client (3.14.0).

melroy89 commented 3 weeks ago

Also updated the client (3.14.0).

Also.. try to downgrade your client. I got another ticket open here regarding 3.14: https://github.com/nextcloud/desktop/issues/7171