Desktop: Joplin Freezing During Syncing and Decrypting On Linux Kernel 5.5+

[dimyself]

I just started using Joplin recently, and since first using it's been locking up/freezing/unresponsive. It seems to be getting stuck in a syncing loop. It will try to sync, and something is not allowing sync to stop/cancel. Even if I manually click on cancel when it's syncing, it just says "cancelling" and stays spinning but won't stop.

At this point, when I click on any notebooks/notes, nothing happens. The notes don't load, the screen doesn't refresh to the note I click on. The only thing that works at this point is I can click and open menus/settings.

I then have to kill the app and relaunch it.

Is anyone else having issues on Linux with Joplin being buggy and freezing?? I'd really like to resolve this so I can use Joplin! I'm not sure if this is something on my system, or if others on Linux are having this issue?? It's pretty much unusable for me at this point! The bug also happens even if I don't click on Sync. I will come back to Joplin to make a new note, and it will be in this stuck sync state on it's own without me doing anything.

It is a major bug on my system. It happens frequently and I can reproduce it easily.

Environment

Joplin version: Joplin 1.0.179 (prod, linux); Sync Version: 1; Revision: b4e325d (master) Platform: Arch Linux OS specifcs:

Steps To Reproduce

1. I can launch Joplin, and to reproduce, I simply click on Sync in the lower left corner a few times, and it gets stuck in the sync loop error/bug.
2. It happens every time. I have to kill/relaunch at this point.

Describe what you expected to happen:

Logfile

Console shows normal activity before the problem when clicking on a different note: webview_domReady Connect {props: {…}, context: {…}, refs: {…}, updater: {…}, version:

Then when I initiate the bug by clicking on sync several times, nothing shows up in console whatsoever. It only has the last reported event from before the bug.

Here is my updated log.txt file: https://pastebin.com/CDdhuL25

Whenever I initially launch joplin in debug, I get these messages in the console (in case they’re important) console.log file : https://pastebin.com/zzjdguTX

[BlissfulTarpon]

@bedwardly-down Thanks for the video, followed your every move. When I launched it though, I don't get the same screen as you do for the Revision. Here is mine:

Which I'm pretty certain is not the correct version if I compare to yours? I did launch it the same way you did with ./dist/Joplin[...].Appimage.

For what it's worth, it seems to be working just fine now (for the last 5 minutes, which is more than I've had it working in the last 2 weeks)...?

I'm really confused haha.

EDIT: So far so good. I've attached a copy of my log. I've tried all the operations (add, remove, edit, pictures etc) and everything works like it did before. Don't know why it says I'm using the master branch instead of yours though.

And I confirm the "old" version still hangs like it did. This is the version that hangs:

LOG: log.txt

[bedwardly-down]

@matcharles, easy fix. The following steps will ensure you have a clean repo to test in. Also, make sure you are only doing these steps in your Joplin directory. Step 2 and 4 can wipe out all of your important documents, downloads, etc, if you use them anywhere else since they delete everything without stopping. Sudo is only used here as a safety measure for anyone who finds these instructions.

1. cd ElectronClient/app
2. sudo rm -rfv *
3. cd ../..
4. sudo rm -rfv *
5. git reset --hard
6. git checkout bug-tracker-2518

Follow the build instructions from BUILD.md

[BlissfulTarpon]

ok brb trying this out.

[bedwardly-down]

I'm heading off to work. I will be checking in some while I'm out but don't expect any super great insight until I'm able to fully focus on this issue. At least we're getting somewhere with it. :D

[BlissfulTarpon]

Yes I definitely think you fixed it but I will do some troubleshooting today and report back, thanks!

[bedwardly-down]

Sounds good. If you can offer some way to show that it's working fine, including version screenshots like above, I'll do a bit more testing myself during lunch break on 5.5 (since switching kernel versions is super simple overall), we'll go from there.

I would think that if Electron 8 doesn't break anything major and doesn't cause issues for other users and platforms, a bug like debug mode breaking the Todo creation part of the app could be handled as a separate bug altogether. It's not a normal daily driver breaking issue

[BlissfulTarpon]

So far so good with the previous method. I'm trying to clean up the folder and rebuild to make sure I have the right version, but I get this at the final step before building:

 ~/joplin-bug-tracker-2518 $ git checkout bug-tracker-2518
error: pathspec 'bug-tracker-2518' did not match any file(s) known to git

[bedwardly-down]

Try git pull origin bug-tracker-2518. It should have pulled it during clone but that should fix it

[BlissfulTarpon]

~/joplin-bug-tracker-2518 $ git pull origin bug-tracker-2518
ERROR: Repository not found.
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

[bedwardly-down]

Delete the full repo and run this

git clone -b bug-tracker-2518 https://github.com/bedwardly-down/joplin.git joplin-bug-2518

That should get you set. Then build like normal.

[BlissfulTarpon]

I'm sorry @bedwardly-down , I feel like a retard asking you all these questions which are probably some basic stuff for all of you. I can't even get it to build again like I did this morning so I will just stop trying. For what it's worth, the version I was running this morning never crashed once on me. I used it intermitently for about 3 hours.

Thanks a lot for you help, I'll let you guys figure this out because I just can't keep up!

[bedwardly-down]

It looks like Electron 8 doesn't fix this issue with Kernel 5.5 on my end when building from the latest stable branch (1.0.179). I'm going to attempt on the master, but I don't think it will work since the master isn't considered stable on the latest Electron 7 release.

Live test: https://youtu.be/tTs83JB1TO4

For anyone else that wants to test it out, here's an AppImage, since that's what is officially supported here and should be the only format that is tested: https://my.pcloud.com/publink/show?code=XZudcskZt31O9LhVUCjFIC8ThawnNpYkw9zX

[m-angelov]

Hey @matcharles , if you feel like everything in your repo is borked, just delete the whole directory and clone it again.

@bedwardly-down my initial report is not accurate. I've forgotten to checkout the branch for this bug, so I've built the 1.0.184 AppImage. Good thing I've noticed, because I encountered the bug again and was very disappointed.

So I rebuilt it again, and unfortunately the bug is still present :/ I have a video, that I'll process and upload in a bit.

Also forgot to mention, that now I can replicate the todo bug (the disappearing sidebar).

My system:

• OS: Arch, updated a few hours ago, with kernel 5.5.4-arch1-1
• Joplin: built from your repo/branch, something specific is I use the dark theme. The about window:

Ah, you ninja'd me :) If you think another video will be helpful, I will upload it.

[bedwardly-down]

@m-angelov the more tests that are verifiable, the better. Thanks for testing it yourself and I'd appreciate you uploading it too.

And, @matcharles, I am messy with making code commits and bad at rebasing and some of the more advanced git features, so don't feel bad, but learning the basics is extremely useful for testing. You'll eventually get there if you keep working at it.

[bedwardly-down]

Also, @m-angelov, if you can get that video uploaded and posted here, I'll take that as Electron 8 not being a viable fix if the issue is still there with it. Master looks like it's going through some major changes right now to make building so much quicker and easier. That also means that it may be harder to test on too.

[bedwardly-down]

Master branch builds, syncs, and decrypts substantially faster but Electron 8 breaks it so much worse. The toolbar up top loses all text and the app froze hard for longer.

https://youtu.be/KAtPLMutKoo

@laurent22, Electron 8 is not a fix and breaks Joplin even worse on Kernel 5.5. Downgrading to 5.4, except when 5.5 is absolutely necessary for hardware reasons (which would be very new hardware for the most part), looks like the best option right now until further notice.

[bedwardly-down]

Just had a Solus report. This is not an arch bug but a kernel 5.5 bug

https://discourse.joplinapp.org/t/joplin-kind-of-freezes-cant-select-different-note-notebook/6023/5?u=bedwardly-down

[bedwardly-down]

As suggested above, could someone test Joplin with the --no-gpu flag? It may require it being enabled on build and that's not something im sure how to do yet

[m-angelov]

@bedwardly-down Here's a clip from my system: https://my.pcloud.com/publink/show?code=XZNo7DkZT6hSwbeSdFRYAajlDfe1d5l0zhTX

There seems to be some wonkiness in the screencap (something like screen tearing, so the background flashes), but you can see what's going on in Joplin.

I'll try researching the --no-gpu flag.

[bedwardly-down]

Thanks for the screen recording. That's exactly the same issue that others are having. @m-angelov, you've been valuable here.

Also, for anyone that's interested, I've gone to Reddit to get some more insight. Linux-5.5.5 just hit the repos this morning and several users there have stated that it fixed the problem for them. Can I get others to test it out, please?

https://www.reddit.com/r/archlinux/comments/f7ts27/psa_joplin_is_broken_on_arch_and_multiple_other/

[bedwardly-down]

Because 5.5.5 isn't available in the repos yet for Artix, I'm building and installing it myself. Expect some tests coming from me soon.

[m-angelov]

@bedwardly-down just got 5.5.5, and spent the last 5-6 minutes creating and deleting notes, syncing in between. No issues with the official 1.0.179 AppImage.

I have to get off the computer now, but I'll test more later.

[bedwardly-down]

No worries. Thanks again. I'm still waiting for 5.5.5 to finish building on my end, but will be joining you there on that soon.

[BlissfulTarpon]

Alright so for what it's worth, running on Linux KALEL 5.5.5-arch1-1 #1 SMP PREEMPT Thu, 20 Feb 2020 18:23:09 +0000 x86_64 GNU/Linux, the old installed version still hangs: But the AppImage that I downloaded from @bedwardly-down 's post works fine. I've been creating, deleting, exporting, adding attachements etc for a good 30 minutes now without any problem. This is the version I'm referring:

[ChALkeR]

Can anyone reproduce this with glibc < 2.31?

If not (or if you are seeing **CRASHING**:seccomp-bpf failure in syscall 0230 in the terminal), https://github.com/electron/electron/issues/22291 might be related.

See https://github.com/laurent22/joplin/issues/2507#issuecomment-590005167.

[bedwardly-down]

Thanks for the update, @matcharles, although, due to your tests being questionable earlier on here, I'll only accept this with video of the issue and others getting the same results. Thanks for understanding.

Because I'm using Artix linux, getting 5.5.5 to build, install and load properly is a bit more difficult than it was in Arch. It turns out that to load modules requires extra steps that are not in most other distributions I've built and loaded kernels on, so I can't directly test out the new kernel until it's either officially released or I can get the quirks worked out.

I'm rolling back to LTS until I can get 5.5.5 to work.

[bedwardly-down]

Welcome, @ChALkeR. When I get off from work and have time, I'll definitely check into that more. I know a user on the forums that was using Void Linux with musl instead of glibc couldn't run an older version of Joplin, so that may be a totally different bug but the current goal here is to get Joplin working so that people that are using this as their daily driver don't have to suffer until the issues can get resolved upstream.

[ChALkeR]

@bedwardly-down Hi! =)

I got here from #2507 (which was marked as a duplicate of this), and which was clearly affected by the glibc incompatibility which surfaced on Arch (and other up-to-date distros). That does not affect Electron 8 though, so if Electron 8 build still has the problem described in this issue, these seem to be separate problems.

That also completely explains the results @matcharles is seeing here: https://github.com/laurent22/joplin/issues/2518#issuecomment-590000530

[bedwardly-down]

I'll have to see what version of glibc Artix is using. It should be pretty much the main version used by Arch since the main packages that are different between distros are those affected by systemd being fully patched out. The majority of releases are the exact same otherwise.

If they're different, that could explain why Electron 8 was exhibiting the issue for me but not others. Thanks for the info

[m-angelov]

Small tidbit of information:

• The official 1.0.179 hangs (confirming @matcharles's experience). Sorry, but I don't have any video. After my initial success, reported previously, I decided that it's fixed, and it blew up in my face 30 minutes later. It was behaving the same as in the previous videos.
• I'll start using the bug-tracker-2518 version now, and will report what's up with it.

Edit:

• The crash is reproducible with the bug-tracker-2518 version, as well as with the master branch (1.0.184). I have some stuff to do now, so will do a recording later, but the behaviour is exactly the same as in my previous video :|

[bedwardly-down]

@ChALkeR I finally got around to checking into the glibc bug you were referring to. I am on 2.31-1 and not running into the exact same issue you were referring to. In the console, when running Joplin with debug mode enabled, I am not receiving anything close to the issue you're referring to. Thanks for bringing it up, anyways

[ChALkeR]

@bedwardly-down try opening devtools (not detached) and resizing the window (might need several attempts).

[bedwardly-down]

@ChALkeR Thanks, but I'm not going to test that further because it's not directly related to the current bug or solving it. Thanks again.

[m-angelov]

@ChALkeR @bedwardly-down for what's it worth, I spent the last 2-3 minutes resizing, moving, minimizing, etc the window and there was no **CRASHING** in the console, or in the logs.

Arch, kernel 5.5.5, glibc 2.31, Joplin 1.0.179 (bug-tracker-2518).

[bedwardly-down]

I'm on Linux-5.5.5 now and building bug-tracker-2518 as we speak.

[bedwardly-down]

Still Broken on Official Release: https://youtu.be/UDSzB4KAsbA

Hangs indefinitely but can be canceled on Electron 8: https://youtu.be/-CLOROGdnXg

[bedwardly-down]

@m-angelov, when you're able to, can you test master please? The bug isn't showing up on so far for me. It has been hanging at the end of sync, but nothing out of the ordinary from what was already happening before the bug showed up. Thanks

EDIT: spoke too soon. It showed up right when I hit sent. Grrrr

https://youtu.be/r_K-L3IkHTI

[Ridbowt]

System: Host: user-kubuntu Kernel: 5.5.0-5.2-liquorix-amd64 x86_64 bits: 64 Desktop: KDE Plasma 5.12.9 Distro: Ubuntu 18.04.4 LTS Joplin stable 1.0.179

Joplin won't react. Clicking on notes does nothing. I mean, like in the videos upper...

[Guibrou]

Checking in with fedora 31 :

Operating System: Fedora 31 (Workstation Edition) Kernel: Linux 5.5.5-200.fc31.x86_64 Architecture: x86-64 Joplin 1.0.179

The behaviour of Joplin is the same as describred in the videos : when syncing, it sometimes loops and stops responding.

[bedwardly-down]

Since this bug is getting out of hand and spreading to other users, node-fetch is on version 2 and does have API breaking changes (as posted here). I'm checking into what would need to be changed for Joplin to upgrade, since that's the next most likely candidate for this issue.

[m-angelov]

@bedwardly-down I built master (1.0.185, Revision: cc759afe) and it's working for now, even after 20-note/todo post-sync-delete-sync. But based on your experience (same build, as far as I saw from your video), I'm just waiting for it to crash :/

Unfortunately I don't have any idea about node-js and electron, so I can't help with the bug itself, but I'm up for testing.

Edit: Just crashed. It was minimized, went in to check a note, Sync started, then froze. I have about 20 of those in the terminal I've started it from: [36613:0224/103937.751805:ERROR:buffer_manager.cc(488)] [.DisplayCompositor]GL ERROR :GL_INVALID_OPERATION : glBufferData: <- error from previous GL command

[bedwardly-down]

I'm not a Node-js dev either, but gotta do something here. We may just get lucky and Node-fetch v2 just straight up works out of the gate with no changes needing to be made. Ha

[bedwardly-down]

@m-angelov the error you're showing isn't Joplin related. It's a common one that shows in Electron and Chromium apps due to how they access the GPU to draw to the screen. They're nothing to worry about and have been around forever. Ha

[bedwardly-down]

Upgrading node-fetch doesn't solve the problem at all. After looking through the code, it's only used for checking for a new version of the app, so definitely back to square one here.

[hpfmn]

Isn't there anybody that knows anything about how to debug this node/electron stuff? I tried to debug the main thread by running joplin with --env --inspect-brk=5858 and then connect via chromium and it's developer tools. But during the hang, also the debugging seems to hang. So I'm quite sure the whole electron system hangs.

EDIT: because I feel, all that is done here is just poking around without, nailing the problem down to anything specific.

[bedwardly-down]

@hpfmn I know as much as you do, probably less, tbh. I do agree that we don't know the exact issue yet, but figuring out what it's not is definitely better than just doing nothing. Let me reiterate what we know so far

1. Upgrading Electron doesn't fix the issue, so it's most likely not Electron that's the issue
2. Upgrading Node-fetch on Master has no effect because it's not directly related to the current issue in the code (check ElectronClient/checkForUpdates.js for the only instance that's related to the Desktop client)
3. It is without a doubt related to Kernel 5.5.x but without knowing the exact cause in relation in to Joplin, there's no way to file a bug report upstream that can be tested and proven to not be Joplin being the problem itself.
4. Until an OSX or FreeBSD user says otherwise, this is a Linux exclusive bug, and whatever is causing the issue is not an issue for the other desktop platforms, so it's going to be harder to pinpoint and test

I also think that if the bug is Joplin based and not upstream, it has something to do with ReactNativeClient/lib/synchronizer.js. That's the source of all Sync functions across all clients. It just needs to be followed up to where it works in ElectronClient since this bug is only affecting Linux Desktop clients.

[bedwardly-down]

According to the changelog for SQLite3, it doesn't look like it supports Electron 8 yet, so it's still possibly a bug related to that upstream. It could be why upgrading to Electron 8 doesn't solve the issue if the bug is there, but that's still shooting in the dark.

[bedwardly-down]

@hpfmn @m-angelov @matcharles @Ridbowt what syncing services do you use?

I'm using PCloud's webdav implementation.

[Ridbowt]

I use Dropbox.

[wisp3rwind]

@bedwardly-down same issue here, syncing to self-hosted Nextcloud (most recent 16.x)