search

pbek / QOwnNotes

QOwnNotes is a plain-text file notepad and todo-list manager with Markdown support and Nextcloud / ownCloud integration.
https://www.qownnotes.org/
GNU General Public License v2.0
4.63k stars 410 forks source link

Metadata file 'notes.sqlite' causes sync conflict with multiple clients #1111

Closed brainchild0 closed 5 years ago

brainchild0 commented 5 years ago

Expected behaviour

Multiple instances of client connect to same remote endpoint without any synchronization conflicts, except when the same note has been modified on both sides.

Actual behaviour

File notes.sqlite is reported as synchronization conflict.

Steps to reproduce

Configure an instances of QOwnNotes on each of two systems, to connect to the same location in the same Nextcloud instance. Configure Nextcloud clients on each system also to synchronize those locations.

Suggestion

Store tag, trash, and related metadata in a separate configuration directory. Also possible to use a hidden file file, although Nextcloud clients can be configured to synchronize hidden files, thereby reintroducing the same problem.

Workaround

Per site, clients can be configured to exclude files based on name patterns.

pbek commented 5 years ago

Because the folder on the server was previously synchronized with a different desktop system that also runs QON.

but why was it then locally empty for you? or did you only sync for the first time afterwards? if yes, what has this to do with QON?

brainchild0 commented 5 years ago

Suppose you have a system called Desktop 1 with successful synchronization with the server.

Now suppose you buy a new system, called Deskop 2. You install QON, set a notes folder, and add a new note foo.md. QON creates a database file in the local folder, notes.sqlite. The database file is different from the one of the same name shared by the server and Desktoip 1.

Now suppose you configure the notes folder in Desktop 2 to sync against the server. The new notes file is uploaded to the server without conflict, but the database file cannot be synced without a conflict because a file with the same name already exists on the server, and neither file is a newer version than the other.

pbek commented 5 years ago

That is why you're synchronizing your folders before you set up qownnotes.

brainchild0 notifications@github.com schrieb am Do., 28. März 2019, 17:36:

Suppose you have a system called Desktop 1 with successful synchronization with the server.

Now suppose you buy a new system, called Deskop 2. You install QON, set a notes folder, and add a new note foo.md. QON creates a database file in the local folder, notes.sqlite. The database file is different from the one of the same name shared by the server and Desktoip 1.

Now suppose you configure the notes folder in Desktop 2 to sync against the server. The new notes file is uploaded to the server without conflict, but the database file cannot be synced without a conflict because a file with the same name already exists on the server, and neither file is a newer version than the other.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/pbek/QOwnNotes/issues/1111#issuecomment-477674518, or mute the thread https://github.com/notifications/unsubscribe-auth/ABtv1ano257ye0za17PLB73mhdL0gSkUks5vbO-pgaJpZM4Z5C5X .

brainchild0 commented 5 years ago

I don't understand.

I stated the particular conditions under which the problem appears. This is how bugs are reported. If I had stated the conditions where no problem appears, then I would not successfully be filing a bug report.

pbek commented 5 years ago

As much as I enjoy writing with you, would you be willing to file a bug for the Nextcloud client if you experience problems with syncing files to Nextcloud? I don't think I'm able to help you with that as much as I like to.

brainchild0 commented 5 years ago

There is no bug in Nextcloud that is causing this problem.

chefschaffner commented 5 years ago

Hi there, as I have the same problem, I try to clarify brainchild0's point. I agree, it's not a problem of nextcloud. Furthermore I guess you will not be able to fix this problem for all cases. If you are syncing two devices and you change the same file at both devices before syncing, you will have the described problem. QOwnNotes cannot fix that, nor can Nextcloud. Steps should be as this:

  1. Sync the two devices
  2. Change contents of an existing file "test.txt" at device one
  3. Change contents of the same file "test.txt" at device two
  4. Sync the two devices
  5. described sync error will occur As the sqlite-file from QON contains informations of all notes, this will happen if you change any note in the described way, because one file contains informations of all notes. If the sqlite file wouldn't exist, the conflict would only occur, if the same note was changed on both sides. As the sqlite-file is changes every time you alter any note, the probability, that the sync-error occurs is much higher. I could think of storing meta-informations in single files (one per note, e.g. Note is "test.txt", Metafile is "test.meta"), but I don't know, if that was possible for QON. Hope I could help a little bit - I like QON very much (Using it on several clients, Windows, Linux & Android) and avoiding this sync-error would be a big enhancement in my eyes.
pbek commented 5 years ago

Thank you for your attempt to help. notes.sqlite doesn't store any information but tags and tag-note-assignments. So the file will not be altered every time you edit a note, just if you add a tag to a note.

brainchild0 commented 5 years ago

And of course it also appears in the case documented originally, when the first sync to Nextcloud occurs after the folder being designated by QON as the notes folder.

A new thought occurs to me, which might alleviate problems from this case. Could the creation of the notes.sql file occur lazily, that is, only after the first tag is added? Such a solution could have the ancillary benefit that a user not using tags need not carry the extra file in the notes folder.

pbek commented 5 years ago

The tag tree wouldn't work if there are no tag tables present. And there is other information stored too, like the trash...

brainchild0 commented 5 years ago

The tag tree wouldn't work if there are no tag tables present.

I'm not sure what the "tag tree" is, or why it is needed before any notes are created.

And there is other information stored too, like the trash...

Until the folder is populated with notes, there is no trash. So this information can also be created lazily.

Is there any information that cannot be deferred? If a folder is vacant of notes, as every folder is until a note is first added, can it not quite suitably be be represented without the SQLite database?

pbek commented 5 years ago

"tag tree"

The one in the user interface.

pbek commented 5 years ago

can it not quite suitably be be represented without the SQLite database?

the user interface will not work without the database

gitowok commented 5 years ago

Heres my workaround: rm notes.sqlite ; touch notes.sqlite ; sudo chown root:root notes.sqlite

This hard blocks QOwnNotes from accessing it. The user interface seems to work fine sofar. I dont use tag/trash functionality. I had to do this on all instances of QOwnNotes that try to access it, otherwise nextcloud would erase the file and recreate it with read/write permissions.

I wish there was a simple switch to turn off tags, trash and notes.sqlite entirely.

pbek commented 5 years ago

I wish there was a simple switch to turn off tags, trash and notes.sqlite entirely.

too deeply ingrained to make that possible easily, I'd rather want to have the sync problem fixed

pbek commented 5 years ago

Meanwhile...

19.7.9

pbek commented 5 years ago

There now is a new release, could you please test it and report if it works for you?

pbek commented 5 years ago

I will close this issue until there is more information.

brainchild0 commented 5 years ago

Confirmed new behavior (tested commit 8a8ec98):

pbek commented 5 years ago

Thank you for your review, @brainchild0. Is this solution satisfying enough for you?

brainchild0 commented 5 years ago

I characterize this solution as satisfying.

I think it is a good solution, considering the context and circumstances. The quality of the user experience improves because the conflict can be promptly resolved automatically rather than needing to be resolved manually, or left unresolved.

As some earlier comments plainly imply, other overall application designs are conceivable that avoid the central problem and provide more dynamic and flexible behavior, but such is far beyond the scope of this issue.

One small note is that a user who is less knowledgeable might have difficulty deciding how to respond to the question. The wording of the question suggests that an affirmative response is preferred, but such a user may be aided by phrasing that more explicitly or forcefully indicates which answer to give, in cases of doubt. For this question, I suspect that it would be very rare that any user would ever answer in the negative. The idea that an affirmative response is normal and a negative one exceptional can be reflected by wording such as, "Proceed with automatic deletion of conflict?".

pbek commented 5 years ago

I think it is a good solution, considering the context and circumstances. The quality of the user experience improves because the conflict can be promptly resolved automatically rather than needing to be resolved manually, or left unresolved.

I also think it's the best solution, since I can't hope to find another way to get around the sync problem.

"Proceed with automatic deletion of conflict?".

Good idea, I'll do that.

pbek commented 5 years ago

19.8.2