search

baumgarr / nixnote2

Nixnote - A clone of Evernote for Linux
http://www.nixnote.org
Other
1.11k stars 157 forks source link

slow response when editing notes #248

Open madmad-code opened 7 years ago

madmad-code commented 7 years ago

I'm running nixnote2-daily+20161229 from the Ubuntu PPA on Ubuntu 14.04 LTS. Qt version 5.2.1.

Every time I open a node and start to edit it, there's a long > 3 seconds of freeze before the editing can be done. Viewing the note is without delay, but for example if I press spacebar, it will freeze for > 3 seconds. After the first delay, everything will be fine. This happens every time I edit the note the first time in the session. I have 4k+ notes.

The trace from this: DEBUG 2017-01-05 12:43:45.328 ( gui/nbrowserwindow.cpp @ 560 ) Checking thumbanail DEBUG 2017-01-05 12:43:45.329 ( gui/nbrowserwindow.cpp @ 570 ) Exiting setContent TRACE 2017-01-05 12:43:45.329 ( sql/notetable.cpp @ 879 ) Fetching Resources? false With binary? false TRACE 2017-01-05 12:43:45.382 ( sql/notetable.cpp @ 884 ) Fetched resources DEBUG 2017-01-05 12:43:51.484 ( sql/notetable.cpp @ 1091 ) QSqlError(-1, "", "") TRACE 2017-01-05 12:43:51.488 ( sql/notetable.cpp @ 879 ) Fetching Resources? false With binary? false TRACE 2017-01-05 12:43:51.547 ( sql/notetable.cpp @ 884 ) Fetched resources DEBUG 2017-01-05 12:43:51.547 ( utilities/noteindexer.cpp @ 51 ) Indexing note: "1 - App Research" TRACE 2017-01-05 12:43:52.872 ( sql/notetable.cpp @ 879 ) Fetching Resources? false With binary? false TRACE 2017-01-05 12:43:52.931 ( sql/notetable.cpp @ 884 ) Fetched resources

Thanks. Fantastic works by the way.

baumgarr commented 7 years ago

I added some tracing information to see where it may be slowing down based on the information you provided. Wait a couple of days & update it from the PPA and give me the same trace information.

Thanks.

madmad-code commented 7 years ago

I updated to nixnote2-daily+20170108, but there's no difference in the trace. I don't see any new update after 8 Jan. Thanks.

TRACE 2017-01-13 15:12:33.062 ( sql/notetable.cpp @ 879 ) Fetching Resources? false With binary? false TRACE 2017-01-13 15:12:33.121 ( sql/notetable.cpp @ 884 ) Fetched resources TRACE 2017-01-13 15:12:35.700 ( sql/notetable.cpp @ 1077 ) Entering setIndexNeeded : TRACE 2017-01-13 15:12:35.704 ( sql/notetable.cpp @ 1113 ) Calling indexNote TRACE 2017-01-13 15:12:35.705 ( sql/notetable.cpp @ 879 ) Fetching Resources? false With binary? false TRACE 2017-01-13 15:12:35.764 ( sql/notetable.cpp @ 884 ) Fetched resources DEBUG 2017-01-13 15:12:35.764 ( utilities/noteindexer.cpp @ 51 ) Indexing note: "NAS power meter" TRACE 2017-01-13 15:12:37.048 ( sql/notetable.cpp @ 1115 ) Exiting setIndexNeeded : TRACE 2017-01-13 15:12:37.048 ( sql/notetable.cpp @ 879 ) Fetching Resources? false With binary? false TRACE 2017-01-13 15:12:37.109 ( sql/notetable.cpp @ 884 ) Fetched resources TRACE 2017-01-13 15:19:23.424 ( gui/browserWidgets/tageditornewtag.cpp @ 217 ) Entering event : 15TagEditorNewTag

TRACE 2017-01-13 15:19:23.440 ( gui/browserWidgets/tageditornewtag.cpp @ 230 ) Exiting event : 15TagEditorNewTag TRACE 2017-01-13 15:19:23.496 ( gui/browserWidgets/tageditornewtag.cpp @ 217 ) Entering event : 15TagEditorNewTag

baumgarr commented 7 years ago

Sorry. I'm just getting back to this prob/em.

I don't see the messages I would expect, so that is strange.

Is this happening on every note or just the ones with attachments or images? Do you experience the delay when creating a new note?

You said it happens the first time you edit the note in the session. If you edit a note, then view a different note, then return to the first one, does the delay happen again when editing the first note?

Sorry for the delay.

madmad-code commented 7 years ago

New (even empty) note also has the same issue (complete trace is attached, relevant trace below). So if I edit a note and go to another note and back, I don't see any delay. In the case of no delay, I don't see this line in the trace: "DEBUG 2017-01-22 20:28:46.530 ( utilities/noteindexer.cpp @ 51 ) Indexing note: "Untitled note" " If I do a sync, I will see the delay if I edit the note again. It seems, the delay appears if "Indexing note" is done.

I tried adding another account with only few hundreds notes, and it works fine there. There is no indexing delay there. I haven't tried to remove the account and re-sync everything in case you still want additional trace. Let me know if I should try that.

Thanks!

DEBUG 2017-01-22 20:28:46.408 ( sql/notetable.cpp @ 160 ) Adding note( 27976 ) "Untitled note" TRACE 2017-01-22 20:28:46.475 ( sql/notetable.cpp @ 879 ) Fetching Resources? false With binary? false TRACE 2017-01-22 20:28:46.530 ( sql/notetable.cpp @ 884 ) Fetched resources DEBUG 2017-01-22 20:28:46.530 ( utilities/noteindexer.cpp @ 51 ) Indexing note: "Untitled note" DEBUG 2017-01-22 20:28:47.799 ( nixnote.cpp @ 1546 ) starting NixNote.updateSelectionCriteria() TRACE 2017-01-22 20:28:47.799 ( filters/filterengine.cpp @ 46 ) Entering filter : DEBUG 2017-01-22 20:28:47.799 ( filters/filterengine.cpp @ 50 ) Purging filters

trace.txt

baumgarr commented 7 years ago

So the delay appears to be database size related.

When you first view a note it reads it into a cache table in memory. This prevents it from having to go through the conversion from Evernote's file format to HTML each time you view a note. I would say that the conversion is causing the delay since edits after the first one are faster, except you said that the delay doesn't happen until you edit the note. If it was the conversion it would happen the first time you viewed a note.

I would say that the delay is caused by the need to delete the note & reindex it, except it should happen each time you edit the note, not just the first time.

I would say that it is caused by the database being fragmented, except it should happen each new note you view, not just the ones you edit.

So, I'm at a loss to definitively say what is happening.

The first thing I'd try is this:

Does that help anything?

madmad-code commented 7 years ago

I vacuumed the database, but I still see the delay when editing. There is always a 1+ second delay on "( utilities/noteindexer.cpp @ 51 ) Indexing note:"

The database size is about 500+MB FYI.

Thanks

baumgarr commented 7 years ago

Are you saying that nixnote.db is about 500MB, or the db-1 directory & contents are about 500MB?

madmad-code commented 7 years ago

The nixnote.db is 500+MB and the whole db-1 directory is 4.6GB.

baumgarr commented 7 years ago

I'm not sure there is much I can do at the moment to correct it without some major overhaul of how indexing works and I don't have an easy way to test any changes since I don't have a DB of that size. I don't want to do that until after the final 2.0 release.

I'll leave this open to remind me to look at it again after the 2.0 release, but about the only other thing I can recommend is Googling sqlite3 and look to see if any of the performance tuning options help you.

Sorry.

madmad-code commented 7 years ago

I will try... Thanks for the effort, I really appreciate your works!

baumgarr commented 7 years ago

I may have some solution to this problem. If you want to experiment with it before the next release send me an email and I'll give you an experimental build. Thanks.

baumgarr commented 7 years ago

I have something that may help. Are you using a 32 or 64 bit build. I'm assuming your using Linux.