search

zadam / trilium

Build your personal knowledge base with Trilium Notes
GNU Affero General Public License v3.0
27.52k stars 1.88k forks source link

"Long" notes are "slow" to paste or edit - But what is "long"? And "slow"? #3850

Open LorenAmelang opened 1 year ago

LorenAmelang commented 1 year ago

Trilium Version

0.59.3

What operating system are you using?

Windows

What is your setup?

Local + server sync

Operating System Version

Win 10 22H2 19045.2846

Description

This is like https://github.com/zadam/trilium/issues/1595, except a smaller note and lockups for way longer!

Created a new blank note, tried to paste in text, 177266 chars, 9385 lines, nothing happened... Killed Trilium.

On retry, test typing and small pastes worked, full paste locked up for 8 minutes, then the text appeared! Still used most of 2 CPUs for another 3 minutes.

Saved note shows Type: text (text/html) Note size: 201 KiB - not even in the Largest notes list, where I have several 4MB+ text notes...

Note opens normally, but when I click Edit the text disappears for 7 minutes!

Can scroll and place cursor, but typing locks up for 16 minutes, same WriteBarrier thread shown in screenshot!

Trilium paste to new note lockup ProcExp

I realize "long" notes are difficult for CKEditor, but how do I guess what will trigger it? This note seemed to be simple text, and not that much of it compared to other notes. But eight minutes of lockup after a paste is a nasty surprise! And 16 minutes of lockup after typing four backspaces is something only a compulsive debugger would ever wait out!

Is there really no alternative to this random risk? It seems nobody knows what exactly will choke CKEditor, so I guess the only answer would be an escape button. There appear to be three other processes running alongside the maxed out CPU - couldn't they watch an abort button and kill the wait?

If a note is that unweildy I'm willing to store it some other way, but having to hard kill Trilium doesn't seem right.

Error logs

trilium-2023-04-16 New Note paste lockup.txt

zadam commented 1 year ago

There's not much which can be done here from my side.

There appear to be three other processes running alongside the maxed out CPU - couldn't they watch an abort button and kill the wait?

JavaScript is single threaded and support cooperative multitasking only. The other processes you see can't help, there could be in theory some watchdog trying to detect such lockup, but all it could do is killing the whole process anyway.

LorenAmelang commented 1 year ago

I moved my whine to the CKEditor GitHub, and have learned a lot... But their online test editor handles either version of my super-slow note content in a tiny fraction of the time Trilium takes. Looks like it is time to bring this back here.

Maybe start with my latest conclusions: https://github.com/ckeditor/ckeditor5/issues/13899

And search back through all the posts for further clues.

The TL:DR is "Copy directly from your web page, paste directly to Trilium, do not try to strip complexity or edit the content as text!" You end up with double spaced content, but it happens reasonably quickly and can actually be edited!

The "hallucinations" remain a bizarre mystery...

zadam commented 1 year ago

Hmm, I took your test content and compared online version and trilium, and it seems comparable - both take minutes.

LorenAmelang commented 1 year ago

@zadam

"I took your test content and compared online version and trilium, and it seems comparable - both take minutes."

But how many minutes?

I did try pasting the whole transcript into the online CKEditor instance.

I tried https://ckeditor.com/docs/ckeditor5/latest/examples/builds-custom/full-featured-editor.html

It handles the whole Transcript in under a minute, shows real newlines if I paste directly from YouTube, soft breaks if I've limited the paste to text only.

I tried pasting a full fresh copy of the transcript into a new Trilium note, and it took about 2 minutes instead of over 8 minutes. The key seems to be whether Trilium shows "soft breaks" (no extra space between lines) or "hard breaks" (double spaced lines). If I strip out the HTML version of the clipboard (PureText extension or paste through a text editor), I get soft breaks and it takes way longer.

What pasting route did you use, and which appearance did you get in your tests?

Neither of the files I posted to the CKEditor GitHub include the HTML version of the clipboard, which Trilium can handle much more quickly. Try going to https://www.youtube.com/watch?v=aMyJvxE59DU and copying the transcript fresh, pasting directly to Trilium. Maybe still minutes, but under 1/4 as many...

Still, I pasted the full text-only version to the online CKEditor in under a minute. Serious mystery remains!