Renaming a folder with large number of files leads to desktop app not responding

karan commented 3 years ago

How to use GitHub

Please use the 👍 reaction to show that you are affected by the same issue.
Please don't comment if you have no relevant information to add. It's just extra noise for everyone subscribed to this issue.
Subscribe to receive notifications on status change and new comments.

Expected behaviour

I renamed a large top-level folder from _archive to zz-archive on the NC Android app. I can browse that on the web UI and in the Android app just fine. I expect the desktop client to sync that without issues.

Actual behaviour

However, the desktop app seems to hang after 10 mins of syncing. I have to force quit the app. Tried it multiple times, and with both 3.0.1 and 3.0.2.

Steps to reproduce

Create a folder with large number of files and directories (mine has 61,684 items and is 7GB+).
Rename the folder in Android app
Sync on desktop

Client configuration

Client version: 3.0.2 and 3.0.1

Operating system: Mac 10.15.5

OS language: English

Qt version used by client package (Linux only, see also Settings dialog):

Client package (From Nextcloud or distro) (Linux only):

Installation path of client:

Server configuration

Nextcloud version: 19.0.3

Storage backend (external storage): Local

Logs

Please use Gist (https://gist.github.com/) or a similar code paster for longer logs.

Client logfile:

There's a lot of PII in this but here's the last few lines before the client hangs:

2020-10-04 13:15:24:629 [ info nextcloud.sync.csync.reconciler ]: Rename origin in other tree (_archive/REMOVED/app/build/intermediates/blame/res/debug/multi/values-fr.json) found
2020-10-04 13:15:24:629 [ info nextcloud.sync.csync.reconciler ]: Switching _archive/REMOVED/app/build/intermediates/blame/res/debug/multi/values-fr.json to RENAME to zz-archive/REMOVED/app/build/intermediates/blame/res/debug/multi/values-fr.json
2020-10-04 13:15:24:629 [ debug nextcloud.sync.csync.reconciler ] [ _csync_merge_algorithm_visitor ]: INSTRUCTION_NONE               server file: zz-archive/REMOVED/app/build/intermediates/blame/res/debug/multi/values-fr.json
2020-10-04 13:15:24:654 [ info nextcloud.sync.csync.reconciler ]: Trying rename origin by csync_rename mapping _archive/REMOVED_PRIVACY/node_modules/jade/node_modules/monocle/node_modules/readdirp/node_modules/minimatch/node_modules/brace-expansion/_travis.yml
2020-10-04 13:15:24:654 [ debug nextcloud.sync.database.sql ] [ OCC::SqlQuery::bindValue ]: SQL bind 1 QVariant(QByteArray, "00075901oc6din706fso")
2020-10-04 13:15:24:654 [ debug nextcloud.sync.database.sql ] [ OCC::SqlQuery::exec ]:  SQL exec "SELECT path, inode, modtime, type, md5, fileid, remotePerm, filesize,  ignoredChildrenRemote, contentchecksumtype.name || ':' || contentChecksum, e2eMangledName, isE2eEncrypted  FROM metadata  LEFT JOIN checksumtype as contentchecksumtype ON metadata.contentChecksumTypeId == contentchecksumtype.id WHERE fileid=?1"
2020-10-04 13:15:24:654 [ info nextcloud.sync.csync.reconciler ]: Rename origin in other tree (_archive/REMOVED_PRIVACY/node_modules/jade/node_modules/monocle/node_modules/readdirp/node_modules/minimatch/node_modules/brace-expansion/_travis.yml) found
2020-10-04 13:15:24:654 [ info nextcloud.sync.csync.reconciler ]: Switching _archive/REMOVED_PRIVACY/node_modules/jade/node_modules/monocle/node_modules/readdirp/node_modules/minimatch/node_modules/brace-expansion/_travis.yml to RENAME to zz-archive/REMOVED_PRIVACY/node_modules/jade/node_modules/monocle/node_modules/readdirp/node_modules/minimatch/node_modules/brace-expansion/_travis.yml
2020-10-04 13:15:24:654 [ debug nextcloud.sync.csync.reconciler ] [ _csync_merge_algorithm_visitor ]: INSTRUCTION_NONE               server file: zz-archive/REMOVED_PRIVACY/node_modules/jade/node_modules/monocle/node_modules/readdirp/node_modules/minimatch/node_modules/brace-expansion/_travis.yml
2020-10-04 13:15:24:684 [ info nextcloud.sync.csync.reconciler ]: Trying rename origin by csync_rename mapping _archive/REMOVED_FOR_PRIVACY/SimpleNotepad/res/values-es/strings.xml
2020-10-04 13:15:24:684 [ debug nextcloud.sync.database.sql ] [ OCC::SqlQuery::bindValue ]: SQL bind 1 QVariant(QByteArray, "00140647oc6din706fso")
2020-10-04 13:15:24:684 [ debug nextcloud.sync.database.sql ] [ OCC::SqlQuery::exec ]:  SQL exec "SELECT path, inode, modtime, type, md5, fileid, remotePerm, filesize,  ignoredChildrenRemote, contentchecksumtype.name || ':' || contentChecksum, e2eMangledName, isE2eEncrypted  FROM metadata  LEFT JOIN checksumtype as contentchecksumtype ON metadata.contentChecksumTypeId == contentchecksumtype.id WHERE fileid=?1"
2020-10-04 13:15:24:684 [ info nextcloud.sync.csync.reconciler ]: Rename origin in other tree (_archive/REMOVED_FOR_PRIVACY/SimpleNotepad/res/values-es/strings.xml) found
2020-10-04 13:15:24:684 [ info nextcloud.sync.csync.reconciler ]: Switching _archive/REMOVED_FOR_PRIVACY/SimpleNotepad/res/values-es/strings.xml to RENAME to zz-archive/REMOVED_FOR_PRIVACY/SimpleNotepad/res/values-es/strings.xml
2020-10-04 13:15:24:684 [ debug nextcloud.sync.csync.reconciler ] [ _csync_merge_algorithm_visitor ]: INSTRUCTION_NONE               server file: zz-archive/REMOVED_FOR_PRIVACY/SimpleNotepad/res/values-es/strings.xml
2020-10-04 13:15:24:712 [ info nextcloud.sync.csync.reconciler ]: Trying rename origin by csync_rename mapping _archive/REMOVED_PRIVACY/node_modules/grunt-contrib-watch/node_modules/gaze/node_modules/globule/node_modules/minimatch/node_modules/lru-cache/README.md

Web server error log: Not relevant - don't think it's a server issue
Server logfile: nextcloud log (data/nextcloud.log): Not relevant - don't think it's a server issue

karan commented 3 years ago

As a workaround, I quit NC, renamed on the Mac _archive to zz-archive, then restarted NC client and now it's at least syncing fine.

Still, this beats the point of "syncing" if I have to make changes on all target devices manually.

github-actions[bot] commented 3 years ago

This bug report did not receive an update in the last 4 weeks. Please take a look again and update the issue with new details, otherwise the issue will be automatically closed in 2 weeks. Thank you!

nextcloud / desktop