Nextcloud-Client creating conflicts when it should not

[Bockeman]

How to use GitHub

• Please use the 👍 reaction to show that you are affected by the same issue.
• Please don't comment if you have no relevant information to add. It's just extra noise for everyone subscribed to this issue.
• Subscribe to receive notifications on status change and new comments.

Expected behaviour

Conflicts should only be created when more than one client uploads to the server the same file with different contents.

Actual behaviour

I have several Nextcoud-client 3.0.1 all running on Win10 machines. One of the sync folders is /AppData/Roaming/Mozilla; this is an "active" folder tree with several files changing continuously. But I have only one Mozilla Firefox browser open.

I am getting several conflicts generated every hour or so.

Steps to reproduce

1. Add a sync folder pointing at a directory where the contents are changing continuously.
2. Wait for conflicts!

Client configuration

Client version: 3.0.1 Operating system: Win10 OS language: English

Server configuration

Nextcloud version: 19.0.3 Storage backend (external storage): No external storage. Files on server are NFS mounted (actually gluster).

Logs

The following are generated from scripts running on a linux server with the Win10 machine mounted under /mnt/veriulan/d/Data/BobW and show 4 separate occassions (with dates shown) when conflicts were generated:

Compare sizes and dates

-rwxr-xr-x 1 bobw warren 254622 2020-09-23 16:51 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Mozilla/Firefox/Profiles/6qw6a8eb.default/sessionstore-backups/recovery (conflicted copy 2020-09-23 165141).jsonlz4 -rwxr-xr-x 1 bobw warren 254616 2020-09-23 16:51 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Mozilla/Firefox/Profiles/6qw6a8eb.default/sessionstore-backups/recovery.jsonlz4

2020-09-23 17:01:43 2020-09-23 17:01:55

Compare sizes and dates

-rwxr-xr-x 1 bobw warren 2514944 2020-09-23 17:20 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Mozilla/Firefox/Profiles/6qw6a8eb.default/storage/default/https+++web.whatsapp.com/idb/3166453069wcaw (conflicted copy 2020-09-23 172056).sqlite -rwxr-xr-x 1 bobw warren 2514944 2020-09-23 17:29 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Mozilla/Firefox/Profiles/6qw6a8eb.default/storage/default/https+++web.whatsapp.com/idb/3166453069wcaw.sqlite

2020-09-23 17:29:59 2020-09-23 17:30:04

Compare sizes and dates

-rwxr-xr-x 1 bobw warren 2514944 2020-09-23 17:36 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Mozilla/Firefox/Profiles/6qw6a8eb.default/storage/default/https+++web.whatsapp.com/idb/3166453069wcaw (conflicted copy 2020-09-23 173605).sqlite -rwxr-xr-x 1 bobw warren 114688 2020-09-23 17:56 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Mozilla/Firefox/Profiles/6qw6a8eb.default/storage/default/https+++web.whatsapp.com/idb/3166453069wcaw.sqlite

-rwxr-xr-x 1 bobw warren 2514944 2020-09-23 17:58 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Mozilla/Firefox/Profiles/6qw6a8eb.default/storage/default/https+++web.whatsapp.com/idb/3166453069wcaw (conflicted copy 2020-09-23 175858).sqlite -rwxr-xr-x 1 bobw warren 114688 2020-09-23 17:56 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Mozilla/Firefox/Profiles/6qw6a8eb.default/storage/default/https+++web.whatsapp.com/idb/3166453069wcaw.sqlite

2020-09-23 19:39:10 2020-09-23 19:39:16

Compare sizes and dates

-rwxr-xr-x 1 bobw warren 261050 2020-09-23 20:10 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Mozilla/Firefox/Profiles/6qw6a8eb.default/sessionstore-backups/recovery (conflicted copy 2020-09-23 201018).baklz4 -rwxr-xr-x 1 bobw warren 264483 2020-09-23 20:45 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Mozilla/Firefox/Profiles/6qw6a8eb.default/sessionstore-backups/recovery.baklz4

-rwxr-xr-x 1 bobw warren 261112 2020-09-23 20:10 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Mozilla/Firefox/Profiles/6qw6a8eb.default/sessionstore-backups/recovery (conflicted copy 2020-09-23 201033).jsonlz4 -rwxr-xr-x 1 bobw warren 264488 2020-09-23 20:45 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Mozilla/Firefox/Profiles/6qw6a8eb.default/sessionstore-backups/recovery.jsonlz4

-rwxr-xr-x 1 bobw warren 262513 2020-09-23 20:18 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Mozilla/Firefox/Profiles/6qw6a8eb.default/sessionstore-backups/recovery (conflicted copy 2020-09-23 201853).jsonlz4 -rwxr-xr-x 1 bobw warren 264488 2020-09-23 20:45 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Mozilla/Firefox/Profiles/6qw6a8eb.default/sessionstore-backups/recovery.jsonlz4

-rwxr-xr-x 1 bobw warren 262061 2020-09-23 20:22 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Mozilla/Firefox/Profiles/6qw6a8eb.default/sessionstore-backups/recovery (conflicted copy 2020-09-23 202248).jsonlz4 -rwxr-xr-x 1 bobw warren 264488 2020-09-23 20:45 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Mozilla/Firefox/Profiles/6qw6a8eb.default/sessionstore-backups/recovery.jsonlz4

-rwxr-xr-x 1 bobw warren 264463 2020-09-23 20:38 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Mozilla/Firefox/Profiles/6qw6a8eb.default/sessionstore-backups/recovery (conflicted copy 2020-09-23 203812).baklz4 -rwxr-xr-x 1 bobw warren 264483 2020-09-23 20:45 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Mozilla/Firefox/Profiles/6qw6a8eb.default/sessionstore-backups/recovery.baklz4

-rwxr-xr-x 1 bobw warren 264409 2020-09-23 20:38 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Mozilla/Firefox/Profiles/6qw6a8eb.default/sessionstore-backups/recovery (conflicted copy 2020-09-23 203834).jsonlz4 -rwxr-xr-x 1 bobw warren 264488 2020-09-23 20:45 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Mozilla/Firefox/Profiles/6qw6a8eb.default/sessionstore-backups/recovery.jsonlz4

-rwxr-xr-x 1 bobw warren 2523136 2020-09-23 20:08 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Mozilla/Firefox/Profiles/6qw6a8eb.default/storage/default/https+++web.whatsapp.com/idb/3166453069wcaw (conflicted copy 2020-09-23 200812).sqlite -rwxr-xr-x 1 bobw warren 2531328 2020-09-23 20:45 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Mozilla/Firefox/Profiles/6qw6a8eb.default/storage/default/https+++web.whatsapp.com/idb/3166453069wcaw.sqlite

2020-09-23 20:46:07 2020-09-23 20:46:13

[edmael]

Same is happening here with Win10, Nextcloud Desktop 3.0.1 and Nextcloud Server 13.0.5.

[codewichtel]

Same happening in Win10 , Nextcloud Desktop 3.0.1 and NC Server 16 17 18 19

[Bockeman]

Another example conflict. This conflict should not happen. The files changing on the client are due to activity. The only files changing on the server are the nextcloud-client uploads to the server.

As a separate issue, "Click for details" has no effect. https://github.com/nextcloud/desktop/issues/1195

A debug log is here nextcloud_200929_1637.log

[rasos]

After several conflicted copy incidents we have been observing randomly timestamp changes on some files on the storage backend. By appointing incrond on some frequently affected files (including files which are not of interest to nextcloud syncing like .htaccess) to report any changes to syslog we could exclude that the server itself is touching any files. Our storage backend (Hetzner Storage Box) is being mounted via sshfs (which turned out more performant and more reliable than Samba/CIFS that had problems with directory names containing spaces at the end). We also mount it read-only (!) on a second server to run backups every second day - which was roughly the frequency of some changed file timestamps. After pausing the backup process we have not seen any conflicted files anymore. When reading a file with the command less on the nextcloud, once we could observe, that this also changed the timestamp, but this behavior was not yet reproducible.

So as the problem of @Bockeman occurs also with files that are NFS mounted to gluster we should be looking more closely at the reliability of storage backends.

[pandiloko]

I'm having this problem with Android Client and Docker server installation (20.0.4). Sorry, I don't understand the nature of the problem but is it not possible to obtain a MD5 sum in case of conflict to let the user know that the changes are related to date (access, creation, whatever) and not content? Even a option to ignore date changes would be fine.

[rojaro]

I am having this issue with the Windows 10 laptop of my wife quite often (at least a dozen times in within the last 12 months):

• Client and server machines use the same NTP server.
• Windows client version is 3.1.1
• Server version is 20.0.5
• There is always a newer and an older file.
• Usually the newer file is on the client, rarely on the server.
• The difference between the timestamps is usually just a few minutes, but often also a day or two.
• More often than not, the changes have been made a few months in the past.
• All files have been uploaded to the server by the very same client and user (no other clients/users have access).
• Storage on the server resides on vanilla Ext4 filesystem which is exclusive to Nextcloud (No other services are using the storage space)

Conflict resolution is also really annoying as i can only resolve one file at a time:

• If i choose to keep a local or remote file the client starts synchronizing which takes some time (~ 10 seconds).
• If i try to resolve another file while the synchronization is still running, the client will reproducably crash and i have to manually restart it.

Once i've finished manually resolving the conflicts everything seems to be fine for some time until my wife calls me again because the conflicts are back.

No problems with our Linux and Android clients ...

[rasos]

We could nail down the problem with an external storage that was changing the wrong timestamp. Instead of atime (when a backup was reading files) it always changed mtime and ctime. Check it with the stat command on any file and see if that is changed correctly, even if you just read. We've built now our own storage system and are just migrating data.

[er-vin]

We could nail down the problem with an external storage that was changing the wrong timestamp. Instead of atime (when a backup was reading files) it always changed mtime and ctime. Check it with the stat command on any file and see if that is changed correctly, even if you just read. We've built now our own storage system and are just migrating data.

I advise everyone here to check this kind of things. Indeed with an external storage behaving that way there's nothing else the client can do but report conflicts.

[github-actions[bot]]

This bug report did not receive an update in the last 4 weeks. Please take a look again and update the issue with new details, otherwise the issue will be automatically closed in 2 weeks. Thank you!

[Bockeman]

I'm now on Nextcloud Client version 3.1.3 and I am still getting conflicts.

The general case appears to be when client files change during a Nextcloud sync, with one client and (obviusly) one server. In my opinion, if there is only one client, there should never be any conflicts under any circumstances, whether files change during the sync process, or there are network drop-outs. This should be all managed by the local and server database files with appropriate checksums to cover potential network issues.

My experience continues to be that Nextcloud Client is creating conflicts when it should not. (Though, in fairness, such conflicts have been occurring less frequently).

[ikogan]

I'm even seeing this issue when using the built in markdown editor in Nextcloud. I did stat on a file:

  File: CC Transition Plan.md
  Size: 6906            Blocks: 29         IO Block: 7168   regular file
Device: 4ah/74d Inode: 329327      Links: 1
Access: (0700/-rwx------)  Uid: (1234/   kogan)   Gid: (1234/   kogan)
Access: 2021-02-25 13:53:08.275518704 -0500
Modify: 2021-02-25 14:39:43.303202466 -0500
Change: 2021-02-25 14:39:43.303202466 -0500
 Birth: 

Then I save the file in the nextcloud gui and run stat again:

  File: CC Transition Plan.md
  Size: 6905            Blocks: 29         IO Block: 7168   regular file
Device: 4ah/74d Inode: 329327      Links: 1
Access: (0700/-rwx------)  Uid: (1234/   kogan)   Gid: (1234/   kogan)
Access: 2021-02-25 13:53:08.275518704 -0500
Modify: 2021-02-25 14:40:00.383787486 -0500
Change: 2021-02-25 14:40:00.383787486 -0500
 Birth: -

A few seconds later, the GUI will tell me the file was edited "outside of the editor". I then stat the file again:

  File: CC Transition Plan.md
  Size: 6905            Blocks: 29         IO Block: 7168   regular file
Device: 4ah/74d Inode: 329327      Links: 1
Access: (0700/-rwx------)  Uid: (1234/   kogan)   Gid: (1234/   kogan)
Access: 2021-02-25 13:53:08.275518704 -0500
Modify: 2021-02-25 14:40:00.383787486 -0500
Change: 2021-02-25 14:40:00.383787486 -0500
 Birth: -

When I looked down at my local clock, it read 14:40 as well. I don't know the exact number of milliseconds but this does not seem to be working as intended. These files are indeed on an "External SSH" share.

[sunjam]

I find the most conflicts appear when editing either git or a small database file from a path being synced by the desktop app. Agreed that there are fewer conflicts, but I'm still struggling in being unable to merge changes.

Example Conflict between desktop and server. The file on device will be the most recent since I just edited it. "Choose a copy to keep" as File on server is now out of sync, but if not resolved... Conflict now affects editing the file on other devices. They must also choose a copy to keep, local or server.

To resolve:

• I write down in a note what copy of the file is on what device
• backup each copy outside of nextcloud with a new naming scheme or tar archive.
• manually merge changes
• delete as many copies as needed, possibly on both the devices and server
• upload and sync the proper copy

This is particularly difficult because these conflicted files are accessed from computers in multiple geographic locations I do not have remote access to, so I've never been able to merge and resolve all changes other than eventually moving to a new name scheme that is not marked as a conflict in order to move forward.

No doubt there is a better way for me to manage this.

On Thu, Feb 25, 2021, 11:44 AM Ilya Kogan notifications@github.com wrote:

I'm even seeing this issue when using the built in markdown editor in Nextcloud. I did stat on a file:

File: CC Transition Plan.md Size: 6906 Blocks: 29 IO Block: 7168 regular file Device: 4ah/74d Inode: 329327 Links: 1 Access: (0700/-rwx------) Uid: (1234/ kogan) Gid: (1234/ kogan) Access: 2021-02-25 13:53:08.275518704 -0500 Modify: 2021-02-25 14:39:43.303202466 -0500 Change: 2021-02-25 14:39:43.303202466 -0500 Birth:

Then I save the file in the nextcloud gui and run stat again:

File: CC Transition Plan.md Size: 6905 Blocks: 29 IO Block: 7168 regular file Device: 4ah/74d Inode: 329327 Links: 1 Access: (0700/-rwx------) Uid: (1234/ kogan) Gid: (1234/ kogan) Access: 2021-02-25 13:53:08.275518704 -0500 Modify: 2021-02-25 14:40:00.383787486 -0500 Change: 2021-02-25 14:40:00.383787486 -0500 Birth: -

A few seconds later, the GUI will tell me the file was edited "outside of the editor". I then stat the file again:

File: CC Transition Plan.md Size: 6905 Blocks: 29 IO Block: 7168 regular file Device: 4ah/74d Inode: 329327 Links: 1 Access: (0700/-rwx------) Uid: (1234/ kogan) Gid: (1234/ kogan) Access: 2021-02-25 13:53:08.275518704 -0500 Modify: 2021-02-25 14:40:00.383787486 -0500 Change: 2021-02-25 14:40:00.383787486 -0500 Birth: -

When I looked down at my local clock, it read 14:40 as well. I don't know the exact number of milliseconds but this does not seem to be working as intended.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/nextcloud/desktop/issues/2467#issuecomment-786154632, or unsubscribe https://github.com/notifications/unsubscribe-auth/AANUKZQLJ6MNUEXRBNOH3CTTA2SAJANCNFSM4RXOZKIQ .

[pandiloko]

I always had problems with Autoupload. It just doesn't work reliably. I really don't understand why is this happening. The only thing that should work, the basis and main focus of Nextcloud should be a secure and reliable sync, in my opinion. It isn't apparently. I don't care about all the fancy apps and bells and whistles. Or maybe, but first and foremost sync MUST work. Always.

If I disable Autoupload and enable it again. It actually re-uploads the whole library. The files don't retain the original creation date and files_versions saves a copy of each file which is byte for byte the same as the re-uploaded one. I have the feeling the solution might be a md5sum (or blake2 or whatever) away because it is almost working but just not yet.

Sorry for this heated feedback. I honestly greatly appreciate this project and someday it will be great but today a 644MB video didn't wanted to upload and I did again the disable/enable trick and while I was watching reupload my 6000+ photos and videos again I had enough. I pulled the plug of my Nextcloud instance and went back to seafile, which doesn't have all the fancy features but it does the sync right (creation date is also not preserved, though).

[github-actions[bot]]

This bug report did not receive an update in the last 4 weeks. Please take a look again and update the issue with new details, otherwise the issue will be automatically closed in 2 weeks. Thank you!

[Bockeman]

With respect, I still think the synchronization algorithm needs further refinement.

The mode of operation is one client which is actively and regularly updating files, and one server with no activity on the directories being synchronised. Other clients are being updated, but this is stricly read only. Thus NC is effectively working as a live backup from the client to the server with a distributed backup to other clients. Any file on the client may change or disappear during synchronization.

In this scenario there should never be any conflicts. Conflicts that do arise are therefore because of a problem with the synchronization algorithm or implementation. As with any distributed system, other independent network or disk activity may mean that response times are sometimes longer than usual (with possible timeouts), but the synchronization algorithm should handle this gracefully with retries, or at worst some "unable to sync error". This scenario should never cause conflicts, yet NC continues to do so.

Below are some recent examples. Notice the dates and frequency; more than one conflict per week.

-rwxr-xr-x 1 bobw warren 50151 2021-03-26 11:02 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Thunderbird/Profiles/rdvpq1tu.default/prefs (conflicted copy 2021-03-26 110259).js -rwxr-xr-x 1 bobw warren 50151 2021-03-26 11:05 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Thunderbird/Profiles/rdvpq1tu.default/prefs.js

-rwxr-xr-x 1 bobw warren 53 2021-03-26 11:03 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Thunderbird/Profiles/rdvpq1tu.default/sessionCheckpoints (conflicted copy 2021-03-26 110301).json -rwxr-xr-x 1 bobw warren 204 2021-03-26 11:01 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Thunderbird/Profiles/rdvpq1tu.default/sessionCheckpoints.json

-rwxr-xr-x 1 bobw warren 524288 2021-03-26 11:09 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Thunderbird/Profiles/rdvpq1tu.default/cookies (conflicted copy 2021-03-26 110923).sqlite -rwxr-xr-x 1 bobw warren 524288 2021-03-26 11:01 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Thunderbird/Profiles/rdvpq1tu.default/cookies.sqlite

-rwxr-xr-x 1 bobw warren 8826 2021-03-26 11:09 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Thunderbird/Profiles/rdvpq1tu.default/Mail/pop3.blueyonder.co.uk/Inbox (conflicted copy 2021-03-26 110923).msf -rwxr-xr-x 1 bobw warren 7255 2021-03-26 11:01 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Thunderbird/Profiles/rdvpq1tu.default/Mail/pop3.blueyonder.co.uk/Inbox.msf

-rwxr-xr-x 1 bobw warren 1173870 2021-03-26 11:09 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Thunderbird/Profiles/rdvpq1tu.default/panacea (conflicted copy 2021-03-26 110923).dat -rwxr-xr-x 1 bobw warren 1173870 2021-03-26 11:01 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Thunderbird/Profiles/rdvpq1tu.default/panacea.dat

-rwxr-xr-x 1 bobw warren 5242880 2021-03-26 11:09 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Thunderbird/Profiles/rdvpq1tu.default/places (conflicted copy 2021-03-26 110923).sqlite -rwxr-xr-x 1 bobw warren 5242880 2021-03-26 11:01 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Thunderbird/Profiles/rdvpq1tu.default/places.sqlite

-rwxr-xr-x 1 bobw warren 1048576 2021-04-04 01:13 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Mozilla/Firefox/Profiles/6qw6a8eb.default/cookies (conflicted copy 2021-04-04 011354).sqlite -rwxr-xr-x 1 bobw warren 1048576 2021-04-04 01:18 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Mozilla/Firefox/Profiles/6qw6a8eb.default/cookies.sqlite

-rwxr-xr-x 1 bobw warren 11206656 2021-04-06 00:19 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Mozilla/Firefox/Profiles/6qw6a8eb.default/webappsstore (conflicted copy 2021-04-06 001936).sqlite -rwxr-xr-x 1 bobw warren 11206656 2021-04-05 00:21 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Mozilla/Firefox/Profiles/6qw6a8eb.default/webappsstore.sqlite

-rwxr-xr-x 1 bobw warren 10485760 2021-04-08 20:40 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Mozilla/Firefox/Profiles/6qw6a8eb.default/places (conflicted copy 2021-04-08 204009).sqlite -rwxr-xr-x 1 bobw warren 5242880 2021-04-08 20:40 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Mozilla/Firefox/Profiles/6qw6a8eb.default/places.sqlite

-rwxr-xr-x 1 bobw warren 11206656 2021-04-08 20:40 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Mozilla/Firefox/Profiles/6qw6a8eb.default/webappsstore (conflicted copy 2021-04-08 204010).sqlite -rwxr-xr-x 1 bobw warren 11206656 2021-04-08 15:05 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Mozilla/Firefox/Profiles/6qw6a8eb.default/webappsstore.sqlite

-rwxr-xr-x 1 bobw warren 1048576 2021-04-10 01:23 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Mozilla/Firefox/Profiles/6qw6a8eb.default/cookies (conflicted copy 2021-04-10 012324).sqlite -rwxr-xr-x 1 bobw warren 1048576 2021-04-09 08:04 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Mozilla/Firefox/Profiles/6qw6a8eb.default/cookies.sqlite

[github-actions[bot]]

This bug report did not receive an update in the last 4 weeks. Please take a look again and update the issue with new details, otherwise the issue will be automatically closed in 2 weeks. Thank you!

[Bockeman]

I upgraded to client 3.2.1. Same issue is still present.

Can I provide more information or other assistance in order for this bug to at least get some attention?

[Bockeman]

Would I be wrong in thinking that this is undesirable behaviour?

Another conflict (client 3.2.1), the only active machines are the client (changing files) and the server (purely hosting).

Compare sizes and dates

-rwxr-xr-x 1 bobw warren 1175152 2021-05-13 10:58 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Thunderbird/Profiles/rdvpq1tu.default/panacea (conflicted copy 2021-05-13 105808).dat -rwxr-xr-x 1 bobw warren 1171456 2021-05-13 10:58 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Thunderbird/Profiles/rdvpq1tu.default/panacea.dat

2021-05-13 11:03:37 2021-05-13 11:03:45

[FlexW]

Would I be wrong in thinking that this is undesirable behaviour?

Another conflict (client 3.2.1), the only active machines are the client (changing files) and the server (purely hosting).

Compare sizes and dates

-rwxr-xr-x 1 bobw warren 1175152 2021-05-13 10:58 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Thunderbird/Profiles/rdvpq1tu.default/panacea (conflicted copy 2021-05-13 105808).dat -rwxr-xr-x 1 bobw warren 1171456 2021-05-13 10:58 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Thunderbird/Profiles/rdvpq1tu.default/panacea.dat

2021-05-13 11:03:37 2021-05-13 11:03:45

Did you save the logs when this happened? If yes, please send them to me https://cloud.nextcloud.com/s/gF7E326o5fPiXFi . If not, please save the logs next time you encounter this problem.

[Bockeman]

@FlexW Thanks for following up on this.

Please could you be explicit concerning "the logs".

On my server

config/config.php
 'datadirectory' => '/srv/gluvol1/vault',
  'loglevel' => 2,
grep 2021-05-13T \
  /srv/gluvol1/vault/nextcloud.log \
  > /srv/gluvol1/vault/nextcloud.log_210513

I posted this to your upload folder. Though you will observe that there are no messages relating to the conflict at 10:58.

On my client AppData/Local/Temp/Nextcloud-logdir is empty I just turned on [/] Enable logging to temporary folder (which I found only after a lot of research and pressing the magic/hidden F12 key. IMO this sort of thing should not be so hidden.)

What else should I enable, or where else should I look?

[FlexW]

I'm at the moment only interessted in the logs of the client. You can get them if you click on Generate Debug Archive...in Settings -> General. Other then that, you find the logs normally in C:/Users/User/AppData/Roaming/Nextcloud/logs

[Bockeman]

I uploaded the Debug Archive, The files under AppData/Roaming/Nextcloud/logs are only the most recent day or so (and are excluded from backups).

[FlexW]

I uploaded the Debug Archive, The files under AppData/Roaming/Nextcloud/logs are only the most recent day or so (and are excluded from backups).

Yes, they get deleted after 24 hours. You can increase the time the logs are kept with setting logExpire=72 in nextcloud.cfg under the [General] section. Setting it to 72 will keep the logs for 72 hours. You can increase the value to any that you want. I guess then the logs are not useful that you send me? Because they didn't captured the conflict?

[Bockeman]

I concur, I couldn't see anything useful in the logs I sent you. I don't need to increase the LogExpire time, because next time I get a conflict, I am confident I can capture what you need. (Assuming indeed that what you want is in the log .gz files). Sometimes I get a burst of several conflicts in a week, otherwise it may be weeks before another conflict. So it may be a while before I can post such here.

[Bockeman]

I had another conflict today. This time I hope I caught the data that might be of some use to you, and uploaded the Debug Archive.

  # Compare sizes and dates
  -rwxr-xr-x 1 bobw warren 14516224 2021-05-27 12:45 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Mozilla/Firefox/Profiles/6qw6a8eb.default/webappsstore (conflicted copy 2021-05-27 124552).sqlite
  -rwxr-xr-x 1 bobw warren 14516224 2021-05-27 10:53 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Mozilla/Firefox/Profiles/6qw6a8eb.default/webappsstore.sqlite

  2021-05-27 13:08:41
  2021-05-27 13:08:59

[Bockeman]

And another one under a different sync'd folder. I uploaded to Debug Archive 210529.zip

  # Compare sizes and dates
  -rwxr-xr-x 1 bobw warren 54329 2021-05-21 15:50 /mnt/veriulan/d/Data/BobW/Document/Finance/Quicken/BACKUP/Warren2 (conflicted copy 2021-05-21 155046).QPH
  -rwxr-xr-x 1 bobw warren 54545 2021-05-29 10:59 /mnt/veriulan/d/Data/BobW/Document/Finance/Quicken/BACKUP/Warren2.QPH

  -rwxr-xr-x 1 bobw warren 54329 2021-05-21 15:51 /mnt/veriulan/d/Data/BobW/Document/Finance/Quicken/BACKUP/Warren3 (conflicted copy 2021-05-21 155100).QPH
  -rwxr-xr-x 1 bobw warren 54545 2021-05-29 10:59 /mnt/veriulan/d/Data/BobW/Document/Finance/Quicken/BACKUP/Warren3.QPH

  2021-05-29 11:55:41
  2021-05-29 11:55:42

[EtlamGit]

Here another way to reproduce these kind of conflicts:

• Configure Minecraft to store the save folder in a Nextcloud synced folder
• Play Minecraft and afterwards you have several conflicting files, even with a completely new created world

In this case, only one computer is syncing to the Nextcloud. Only the server and this single client are exchanging files whereas only the client is changing the content.

This use-case was working perfectly up to about 6 month ago!

[Bockeman]

I just had two more conflicts (client 3.2.2):

  # Compare sizes and dates
  -rwxr-xr-x 1 bobw warren 18152 2021-06-19 11:04 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Mozilla/Firefox/Profiles/6qw6a8eb.default/prefs (conflicted copy 2021-06-19 110427).js
  -rwxr-xr-x 1 bobw warren 18088 2021-06-19 11:10 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Mozilla/Firefox/Profiles/6qw6a8eb.default/prefs.js

  2021-06-19 11:19:52
  2021-06-19 11:20:11

  # Compare sizes and dates
  -rwxr-xr-x 1 bobw warren 18561 2021-06-19 11:22 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Mozilla/Firefox/Profiles/6qw6a8eb.default/prefs (conflicted copy 2021-06-19 112248).js
  -rwxr-xr-x 1 bobw warren 18561 2021-06-19 11:23 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Mozilla/Firefox/Profiles/6qw6a8eb.default/prefs.js

  2021-06-19 11:23:49
  2021-06-19 11:23:57

I was not able to upload the debug archive because the previously supplied upload share link is no longer available.

I can live with this rate of failure (once every few weeks or so), but it does serve Nextcloud's reputation well.

Please could someone from Nextcloud acknowledge that this is a bug that deserves attention and propose a way forward (like more information required, or supply a patch that extracts more useful debug information). Thanks in advance.

[FlexW]

@Bockeman Thanks for your reports. They are very helpful and this bug needs definitely fixing. As I already said I wanted to look into this, but have a lot of other things on my plate at the moment. I updated the link https://cloud.nextcloud.com/s/6w5abTCtdHJQjrw .

[Bockeman]

@FlexW Much appreciate you response. I'm happy to wait for you to get some time to look at this properly. PS the new link you created is a download link, not an upload link.

[FlexW]

@Bockeman thanks. I changed that.

[Bockeman]

I've just uploaded "Debug Archive 210619.zip"

[Bockeman]

Another conflict, "Debug Archive 210623.zip" uploaded

  # Compare sizes and dates
  -rwxr-xr-x 1 bobw warren 54545 2021-06-16 14:32 /mnt/veriulan/d/Data/BobW/Document/Finance/Quicken/BACKUP/Warren2 (conflicted copy 2021-06-16 143218).QPH
  -rwxr-xr-x 1 bobw warren 56131 2021-06-23 16:40 /mnt/veriulan/d/Data/BobW/Document/Finance/Quicken/BACKUP/Warren2.QPH

  2021-06-23 16:48:52
  2021-06-23 16:48:53

[Bockeman]

I upgraded to 3.2.3. I got another conflict and uploaded "Debug Archive 210703".

  # Compare sizes and dates
  -rwxr-xr-x 1 bobw warren 1176305 2021-07-03 01:11 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Thunderbird/Profiles/rdvpq1tu.default/panacea (conflicted copy 2021-07-03 011106).dat
  -rwxr-xr-x 1 bobw warren 1171456 2021-07-03 01:11 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Thunderbird/Profiles/rdvpq1tu.default/panacea.dat

  2021-07-03 01:18:23
  2021-07-03 01:18:34

[Bockeman]

I upgraded to 3.2.4, got another conflict and uploaded "Debug Archive 210710.zip"

When do you think you might be able to give this some attention?

[Bockeman]

Another conflict uploaded "Debug Archive 210713.zip"

[FlexW]

@Bockeman Thanks for the uploads, they helped a lot. I looked into them and can see the problem. However, I was not yet able to find the root cause. For now, you can stop uploading the debug archives :)

[tomasodehnal]

@FlexW @Bockeman I saw the same issue when I had the Synology DS Cloud app set up to backup the same folder. When I stopped syncing the folder using the Synology app I stopped seeing the conflicts. I'm not sure if it applies, just wanted to add my observations.

[Bockeman]

I have no other sync app running, @tomasodehnal (any change, like reducing activity, like stopping your Synology app, could reduce the occurrence of conflicts. That might appear like a solution, but not actually be the fix to a fundamental problem). On the server side, the file system is gluster, but that should not make any difference to how the client handles potential conflicts (apart from, say, more variation in response times under some circumstances). Having a more responsive server has reduced the frequency (mean time between conflicts), but there should be never be any conflicts in my situation -- given only files change on the client, though noting that the client might be very active/busy. The important point here is that @flex has now seen enough data to identify the problem. I'm crossing my fingers for insight to the root cause and a fix/patch in the not too distant future.

[RedKage]

I have been having this issue for like, ever. Since OwnCloud 9 I think. I cannot reproduce it in a controlled fashion.

However, my steps to reproduce are to use a Windows program installed on the Nextcloud filesystem which is constantly writing little files or moving stuff around. In other words having a live program like a Chromium browser with its data folder on Nextcloud. Or a Firefox with its cache folder.

I have the issue also with PuTTy and a file getting overwritten frequently to handle randomness. Pretty much installing the PortableApps platform on Nextcloud, with some web browsers and tools and use them live.

I have conflicts with only one client and one server. Nothing else. Client is creating conflicts against itself.

[Millesimus]

I'm having a similar issue with client 3.3.0 on a Linux system with an CIFS mount. The strange thing is that when I open the conflict dialogue, the local file (on the CIFS mounted drive) is always shown with a modification timestamp of now (not literally, just a most recent timestamp).

[Bockeman]

Now upgraded to Client 3.3.2. Still getting conflicts. Have debug archives of all conflicts since May 2021. Is there anything we (users) can do to raise the priority of this or at least get a date when some progress might be made?

[Bockeman]

Continuing with regular updates, now at Client 3.3.6. Still getting conflicts. Have debug archives of all conflicts since May 2021. @FlexW could you give us an update, or any sort of optimistic news?

[wonx]

I confirm that this bug is still present in Nextcloud 3.4.0 in Ubuntu. By the way, I reported this bug a while ago, which I think refers to the same problem: https://github.com/nextcloud/desktop/issues/3913

[Bockeman]

I am continuing with regular updates, and I am now at Client 3.4.0. I am still getting conflicts. I have debug archives of all conflicts since May 2021. @FlexW could you give us an update, or any sort of optimistic news?

@wonx thanks for adding.

Given clarifications above (a conflict should never happen when a single client is accessing files), there may be some classes of behaviour that may be more easy to fix than others. The conflict I got today is the same as many previous conflicts. It is characterised by a Windows application (Quicken) that keeps its own backups and rotates the backup set by moving/renaming previous backups. That's pretty standard. But I can see that an external agent, such as Nextcloud Client, could misconstrue what is happening when several files are renamed. Does this help in identifying a possible fix?

[claell]

I didn't read everything carefully here, but I also get the same problems. Is it possible that this happens when a file gets changed locally while the same file is currently uploaded to the server? I could also assume that the timestamps (likely on the server) are sometimes not correct.

[ruedigerkupper]

I ruled out the timestamp problem: time is exactly the same on all machines involved. Moreover, NC does not use timestamps any longer for distinguishing file versions. (It did so in former versions, but not any longer. File version numbers get stored in the database — at least that's what I read.) So currently, my focus is on database config and performance. But I can't find any issue there...

[FlexW]

@Bockeman Unfourtonley no. Even if I had the same problem in the past, I couldn't reproduce it. I tried scripts that randomly created millions of files and changed them in all different time intervals. I've also tested with git repositories and other applications that are known to cause conflicts, but I had no luck. It's difficult to fix a bug if you can't reproduce it in a controlled fashion.

@claell I would think that is not the problem because the code accounts for that, and I also did tests covering this scenario.

Small disclaimer: I'm not part of Nextcloud anymore, so don't expect a lot (if any) updates on this topic from me.

[claell]

Alright.

@FlexW sad to hear that you are not working on this anymore.

[wonx]

In my case it happens in three different computers, one with Windows 10, another with MacOS and another with Linux. Anytime a file is edited while Nextcloud is running (even if it's not transferring anything) will cause a conflict. My files are stored in an external storage, so that might be related to the problem.

After editing a file, it seems that the new version is uploaded to the server, the previous version restored in the client, and then Nextcloud shows a conflict between the two (even creating the "conflicted copy" file with the date), where the server version is always newer. There's a screenshot in https://github.com/nextcloud/desktop/issues/3913.