nixnote crashes when tags get out of sync

[timoc]

Update:I have an issue where it crashes before/during sync with an index out of bounds.

IT seems to be in the tagging part of the sync.

[robert7]

you need to post the log file - at least the end (see wiki howto) some command line functions don't work now - this will be fixed in beta-3 - i'll look at it

[timoc]

Sorry been to busy trying to use it to make bug reports.

./NixNote-2.1.0-beta-2-x86_64.AppImage --syncAndExit
ASSERT: "uint(i) < uint(size())" in file /usr/include/x86_64-linux-gnu/qt5/QtCore/qbitarray.h, line 118

and

./NixNote-2.1.0-beta-2-x86_64.AppImage --sync
DEBUG 2018-08-25 12:27:05.875 src/communication/communicationmanager.cpp:155 Inside CommunicationManager::getUserInfo
DEBUG 2018-08-25 12:27:06.248 src/threads/syncrunner.cpp:949 Beginning to upload new & altered tags
DEBUG 2018-08-25 12:27:06.248 src/threads/syncrunner.cpp:956 Found changed tag "@tag1
DEBUG 2018-08-25 12:27:06.249 src/threads/syncrunner.cpp:967 Tag has no parent or parent is unaltered
DEBUG 2018-08-25 12:27:06.249 src/threads/syncrunner.cpp:1012 Tag with this name exists.
DEBUG 2018-08-25 12:27:06.252 src/threads/syncrunner.cpp:956 Found changed tag "@tag2
DEBUG 2018-08-25 12:27:06.253 src/threads/syncrunner.cpp:967 Tag has no parent or parent is unaltered
DEBUG 2018-08-25 12:27:06.253 src/threads/syncrunner.cpp:1012 Tag with this name exists.
DEBUG 2018-08-25 12:27:06.256 src/threads/syncrunner.cpp:956 Found changed tag "@tag3
DEBUG 2018-08-25 12:27:06.256 src/threads/syncrunner.cpp:967 Tag has no parent or parent is unaltered
DEBUG 2018-08-25 12:27:06.257 src/threads/syncrunner.cpp:1012 Tag with this name exists.
DEBUG 2018-08-25 12:27:06.260 src/threads/syncrunner.cpp:956 Found changed tag "@tag4
DEBUG 2018-08-25 12:27:06.260 src/threads/syncrunner.cpp:967 Tag has no parent or parent is unaltered
DEBUG 2018-08-25 12:27:06.260 src/threads/syncrunner.cpp:1012 Tag with this name exists.
DEBUG 2018-08-25 12:27:06.263 src/threads/syncrunner.cpp:956 Found changed tag "@tag5
DEBUG 2018-08-25 12:27:06.264 src/threads/syncrunner.cpp:967 Tag has no parent or parent is unaltered
DEBUG 2018-08-25 12:27:06.264 src/threads/syncrunner.cpp:1012 Tag with this name exists.
DEBUG 2018-08-25 12:27:06.267 src/threads/syncrunner.cpp:956 Found changed tag "@tag6
DEBUG 2018-08-25 12:27:06.267 src/threads/syncrunner.cpp:967 Tag has no parent or parent is unaltered
DEBUG 2018-08-25 12:27:06.267 src/threads/syncrunner.cpp:1012 Tag with this name exists.
DEBUG 2018-08-25 12:27:06.269 src/threads/syncrunner.cpp:956 Found changed tag "@tag7
DEBUG 2018-08-25 12:27:06.269 src/threads/syncrunner.cpp:967 Tag has no parent or parent is unaltered
DEBUG 2018-08-25 12:27:06.269 src/threads/syncrunner.cpp:1012 Tag with this name exists.
DEBUG 2018-08-25 12:27:06.272 src/threads/syncrunner.cpp:956 Found changed tag "@@"
DEBUG 2018-08-25 12:27:06.272 src/threads/syncrunner.cpp:967 Tag has no parent or parent is unaltered
DEBUG 2018-08-25 12:27:06.272 src/threads/syncrunner.cpp:1012 Tag with this name exists.
DEBUG 2018-08-25 12:27:06.274 src/threads/syncrunner.cpp:956 Found changed tag "@@tag8
DEBUG 2018-08-25 12:27:06.275 src/threads/syncrunner.cpp:956 Found changed tag "@@tag9
ASSERT failure in QList<T>::operator[]: "index out of range", file /usr/include/x86_64-linux-gnu/qt5/QtCore/qlist.h, line 518
Aborted (core dumped)

I had also had some core dumps manipulating the tags, renames etc. I suspect the differences between the remote and local tag heirachy is causing the issue. When i updated the heirachy to the way i thought it should be via the web client, nixnote started again.

[robert7]

"--syncAndExit" doesn't work (but "sync" does - from version beta-3a); i'll fix "--syncAndExit" soon.

Regarding crash after wild remote tag changes: ... maybe. Unfortunately the original code isn't very clean and this may be difficult to reproduce and fix.

And honestly: don't do it. :)

Recommendation (how I do it):

1. if you need to do wild tag refactoring, then do it on remote (via web client or Evernote windows client - e.g. using Virtualbox), then delete nixnote data and do full resync - and don't do this often :)
2. although I use tags and spent quite time on them... I find more and more, this is often just not worth the time (at least for my use cases). So previously I spent lot of time tagging, but when I needed to find the content, it didn't work (not technically, but the content wasn't tagged like I supposed)
3. So I think, we don't need "tagging" - we need better search :)
4. So now I still use tags, but only seldom and mainly as "relevance hint"

=> although the tag sync after grave remote changes is bug, i'll don't intend to fix - it isn't worth

If you find a crash in "normal use case" - like tag added/removed on remote crashes the app or tag added/removed in nixnote crashes the app - i will try to fix of course. But here I'll only fix the "happy path" - everything is out of scope (as you need to understand my time for this project is very limited).

[robert7]

Anyway great thx for the bug reports :)

[nopivnick]

if you need to do wild tag refactoring

what constitutes 'wild tag refactoring'?

does 'wild' mean 'extensive' or do you mean any change in tag hierarchy made in any client other than NixNote?

does moving so much as a single tag into or out of a nested hierarchy trigger the bug?

So I think, we don't need "tagging" - we need better search :)

for what it's worth, I hate that Evernote captures that 'snippet' content when you bookmark a url using the web clipper precisely because it throws off search in the app.

[robert7]

I wrote a wiki article :) how to do a wild changes in your tag structure..

[robert7]

what constitutes 'wild tag refactoring'?

I mean greater/big changes in the tag hierarchy. Adding/removing tags from many notes. It should work, but it may not work in nixnote. You need to consider, that e.g. adding a tag to 1000 notes means, that 1000 notes will need to be synced.

does moving so much as a single tag into or out of a nested hierarchy trigger the bug?

It should not. But... I just tried... and if I moved a tag in a hierarchy in nixnote, it was not synced to cloud. If I did it in web client, its synced to Windows client and back to nixnote.

So for now I suggest, if you need to change the tag hierarchy: do it on remote side and sync it down to nixnote. If you plan it a bit, you should not need it often.

I hate that Evernote captures that 'snippet' content when you bookmark a url using the web clipper precisely because it throws off search in the app

Not sure what you mean here. Post a screen shoot or add more explanation..

[robert7]

So I did a pending cleanup in my tag structure... I did move some tags in hierarchy (like 10 various hierarchy changes).. and I did assign one tag to a group of ~500 notes (by mistake ... but OK). It did this in official Evernote Windows client via Virtualbox - as I think this is safest. Now I wait for the notes to sync down to nixnote.

Note, that if you change many notes on remote - you almost for sure hit temporary sync limit - message is like "RateLimitExceeded[19]: API rate limit exceeded. Application will continue to sync in XY minutes".

This is limit imposed by Evernote to all "unofficial clients" - more info in Evernote documentation - rate limits. Currently (in beta-2) unfortunately the sync warning/error status info is covered by the status info "Count: ....". I'll fix that later.

To handle rate limit, you don't need to do anything, just let NixNote run for enough time - it may need few hours - as the rate limit is per hour and you may need bunch of "limit chunks/hours" to get all the changes from remote.

[robert7]

I updated the wiki article. There is a way, how to the tag structure refactoring safely. You just need to take a bit care a plan ahead.

[nopivnick]

You need to consider, that e.g. adding a tag to 1000 notes means, that 1000 notes will need to be synced.

I assume (in this particular context) re-naming a tag that's already assigned to (in your example) 1000 notes is the same as adding a new tag in bulk?

Not sure what you mean here. Post a screen shoot or add more explanation..

when you clip a web page as a Bookmark with Evernote's Web Clipper they save "a snippet of text" (described here).

they think they're being clever / helpful but it throws off search if the "snippet" contains a term that satisfies the search string.

[robert7]

I assume (in this particular context) re-naming a tag that's already assigned to (in your example) 1000 notes is the same as adding a new tag in bulk?

No. it seems that renaming a tag will only synchronize that tag (=> little data to transfer).

But assigning a tag to 1000 notes, will result in need to transfer full content of all 1000 notes. This may be huge, if you have e.g. images inside. This will for sure trigger "rate limit" => you could e.g. let the sync run over night to finish it.

I did like this yesterday and all went well - so all may tag structure changes synced well down to nixnote - just it taken quite long time (due to Evernote rate limits).

[robert7]

when you clip a web page as a Bookmark with Evernote's Web Clipper they save "a snippet of text" .... helpful but it throws off search if the "snippet" contains a term that satisfies the search string.

I did just that, and it seems to work OK (I could not repeat the part with "throws off search if the "snippet" contains a term that satisfies the search string" I use web clipper, but most times I clip the "selection"

[robert7]

closing; see wiki article for hotwo change the tag structure