search

laurent22 / joplin

Joplin - the privacy-focused note taking app with sync capabilities for Windows, macOS, Linux, Android and iOS.
https://joplinapp.org
Other
45.32k stars 4.92k forks source link

Random loss of the latest input in a note #11124

Open dejafait opened 4 days ago

dejafait commented 4 days ago

Operating system

Android

Joplin version

3.0.8

Desktop version info

No response

Current behaviour

I've struggled with random input loss for weeks. Typically when I come back to a note I worked on earlier or the day before, I realize the last word of a sentence I wrote is missing, or its last letters, or an important link I pasted. The input lost is always the latest input in the note.

This is a serious bug that can break the user trust.

I've not been able to figure out a way to deterministically reproduce it but I've been able to narrow it down enough for this bug report, so here are my findings.

Let's say you start with a note titled A (the issue happens both in titles and bodies btw). On your device D1 you rename it to AA then AAA. Sometimes D1 will only sync the AA change and not the AAA change.

The note is AAA on D1 but AA on all your other devices. Syncing manually on D1 doesn't fix anything, D1 incorrectly believes it has synced everything, this is very disturbing when you see it with your own eyes like I did. D1 and D2 both claim sync is up to date but D1 shows AAA and D2 shows AA.

Two possible endings:

a) The next time I work on this note is on D1 again. The issue is fixed with my next change because the note was still AAA on D1. This is why I guess the issue fixes itself 90% of the time and is not so apparent. The issue happened but I never realized it did.

b) The next time I work on this note is on D2. I see note AA, realize the input loss, my trust in Joplin is hurt. Joplin loses my data in my back. I manually fix the missing input (pretty difficult if it was a hard to find pasted link). Let's say I rename AA to AAB. The AAB change is propagated to D1 and overwrites AAA.

Expected behaviour

The app should never ever believe it has synced all changes when it hasn't.

Logs

No response

Notes

I have a hard time wrapping my head around the fact that the app believes it has synced all changes of a note when it hasn't. This should be solvable with hashing functions. Keep a SHA256 of every item and compare it between local and sync target to determine whether the content is identical or not.

mrjo118 commented 3 days ago

I've struggled with random input loss for weeks. Typically when I come back to a note I worked on earlier or the day before, I realize the last word of a sentence I wrote is missing, or its last letters, or an important link I pasted. The input lost is always the latest input in the note.

This is a serious bug that can break the user trust.

I definitely noticed this happening a few times when I was briefly on 3.0.8. I didn't do experiments with multiple devices so can't say for sure if the differing result across 2 devices is the same issue as the first part of your report, but the quoted section describes exactly what I experienced when working with a single device. At the time I thought it might have been because I had an outdated version of Gboard, which caused issues in the past. I have since upgraded Gboard and upgraded Joplin to the pre-release versions 3.1.2 and then 3.1.3 and I don't think I have experienced this anymore. I do sometimes notice an additional line break in the last area I edited when I review a previously edited note, but honestly by the time I notice it I can't reliably remember if anything is actually missing (though I'm more convinced that their hasn't been any lost content since using the pre-releases) or if I just left the line break there by mistake. But there potentially may be some remnant of the issue still present in the latest code, as this has happened a few times within a few weeks and I'm generally quite disciplined about keeping my note formatting clean.

mrjo118 commented 3 days ago

Let's say you start with a note titled A (the issue happens both in titles and bodies btw). On your device D1 you rename it to AA then AAA. Sometimes D1 will only sync the AA change and not the AAA change.

Regarding this point. When that happens, did you try manually pressing the sync button on D1 and seeing if it syncs the latest change? What I noticed on the Android app since I started using Joplin is that the auto sync which happens immediately after changing a note often does not sync all the latest objects, and in the past this has resulted in note conflicts for me when editing the same note in sequence on 2 devices.

Due to this, my workflow with Joplin is that after I am done writing notes on the app, I wait for the sync to complete upon exiting the last note I edited. Then when it has completed I manually press the sync button and wait for that to complete before exiting the app. It's a bit annoying having to do that, but the process only takes about 10 seconds each time and I haven't had any issues with missing items or conflicts when syncing to another device since doing that.

dejafait commented 2 days ago

Let's say you start with a note titled A (the issue happens both in titles and bodies btw). On your device D1 you rename it to AA then AAA. Sometimes D1 will only sync the AA change and not the AAA change.

Regarding this point. When that happens, did you try manually pressing the sync button on D1 and seeing if it syncs the latest change?

Yes I manually trigger sync all the time during these tests, over and over. When I say "D1 will only sync the AA change and not the AAA change" I mean that it stays the case even after manually syncing. The app simply incorrectly believes it has synced all changes when it hasn't. This is the disturbing part.

What I noticed on the Android app since I started using Joplin is that the auto sync which happens immediately after changing a note often does not sync all the latest objects, and in the past this has resulted in note conflicts for me when editing the same note in sequence on 2 devices.

Due to this, my workflow with Joplin is that after I am done writing notes on the app, I wait for the sync to complete upon exiting the last note I edited. Then when it has completed I manually press the sync button and wait for that to complete before exiting the app. It's a bit annoying having to do that, but the process only takes about 10 seconds each time and I haven't had any issues with missing items or conflicts when syncing to another device since doing that.

This is a different issue from the one I am describing here. You are talking about the need of, as I call it, "babysit the sync". You need to constantly worry about sync and end up oversyncing manually all the time. Because the sync should just work but doesn't.

For this direction I have submitted this other issue: https://github.com/laurent22/joplin/issues/11081 please upvote it if you think this makes sense.

mrjo118 commented 20 hours ago

Some questions: -With the reproduction steps you mentioned, how long on average would it take you to reproduce the issue? -Is this something you experienced in Joplin 2, or do you think it is a new issue since upgrading to Joplin 3? -Do you have E2E encryption enabled? -What method of sync are you using? i.e. Joplin cloud, OneDrive, Dropbox

I had a go doing the reproduction steps for about 15 minutes and could not reproduce the issue. It was possible for me to reproduce an incomplete sync by making a second subsequent edit while a sync triggered by an initial edit was still running, but this was resolved by simply doing a manual sync after the auto sync completed. There are some differences though for what I was doing: -I am repeatedly editing the note title on an Android phone and syncing with a Windows PC (rather than Android to Android) -I am using Joplin 3.1.3 instead of 3.0.8. The reason for this is on my spare device I can't use 3.0.8 due to a bug on older devices which wont allow me to go onto the settings screen -I am using Dropbox as the sync target (potentially different) -I didn't use E2E encryption (potentially different)

A couple of thoughts. If you are willing to sideload a pre-release of the Android app, maybe you could try the latest version at least on the most used of your Android devices and see if you still get the issues over a period of time?

My other thought is maybe that updating the note on the sync target does work, but it might be that the secondary Android device is not correctly updating or rendering the note locally after updating from the sync source. Next time the problem happens, you could try syncing your profile with the desktop app and see if the notes are up to date. I see on your other issue https://github.com/laurent22/joplin/issues/11122 that @personalizedrefrigerator mentioned a few issues with regards to the note list not updating properly

dejafait commented 14 hours ago

The day I wrote this bug report I was able to reproduce the bug in 5-10 minutes maybe, editing only the note title for simplicity. Sync target is my own Joplin Server. No E2EE. I only started using Joplin intensely since v3 thanks to android plugins so I cannot say for v2.

My clients are 2 MacOS and 5 android. I was playing with 3 android tablets when the bug happened. AAA on D1 but AA on D2 and D3. AA was on at least 2 android devices thus not the issue of a single device. Unfortunately I did not think of checking on desktop at the time.

I am interested in side loading a pre release on my android devices, there are a lot of bug fixes I am interested in. Let me know.

The fact you could not reproduce it in 15 minutes on 3.1.3 is actually a good sign, maybe the issue is already fixed.

mrjo118 commented 13 hours ago

It does seem more likely from what you said that it is a sync issue then.

But yeah if you're happy to side load a pre-release in the meantime to trial if the issue has been fixed, you can get the apks here https://github.com/laurent22/joplin-android/tags