Ctrl-Z end up on a file and corrupt the note

[monperrus]

Sometimes, I type Ctrl-Z in a note. Some of them end up being written on disk.

Then, tomboy-ng cannot load them anymore because they are considered corrupted.

Thanks Davo!

[davidbannon]

Martin, what version of tomboy-ng are you using ? All recent versions 'should' consume a ctrl-z in undoing your last 'do'. Does undo work for you ? Ctrl characters in a note are a bad thing ! I have a feeling that I don't filter out unused ones on Linux and Windows but do filter them on MacOS (no idea why the difference). But Ctrl-Z is definitely used. I'll do some tests....

Davo

[davidbannon]

No, sorry, I am wrong. All Ctrl keys are filtered out, including Ctrl-Z. Been the case for a long time. So, I cannot think how they are getting through to your notes. Could it be that you are typing that Ctrl-Z into the Find section rather than direct to the note ? Any idea what else you might be doing special at the time ? I cannot reproduce the problem here with a quick test ... Davo

[davidbannon]

Martin, been looking over the code, cannot see how this is happening. But something that costs a note must be fixed. While I could filter the content while saving it would be a bit slower. But as saving is now done in a separate thread, so not so bad. But I would really rather track it down because unexplained things sometimes hide bigger problems. So, some questions -

Firstly, are you sure its Ctrl-Z, not some other control character ? (I allow a number of ctrl char to go through to editor because it can handle them, Ctrl-C, Ctrl-V etc.)

Your operating system ? I have not tested for this on Windows yet. Windows apparently adds a ctrl-Z to the end of a file but being beyond the end of the XML should do nothing. But my way of saving might trigger that addition...

Do you work with the Undo-er turned on or disabled ?

I don't suppose you have one such corrupted note you could send me ?

I am putting the polish on version 0.35 right now but have already caught a number of bugs, I'd sure like to understand your issue here.

Davo

[monperrus]

Firstly, are you sure its Ctrl-Z, not some other control character ?

I read "Ctrl-Z" in "less".

Your operating system ?

Linux

Do you work with the Undo-er turned on or disabled ?

turned on, you're right "Ctrl-Z" should be captured by undo.

I don't suppose you have one such corrupted note you could send me ?

no, sorry

thanks for your help in troubleshooting.

[davidbannon]

OK, I cannot replicate but I have added an extra filter in the save code that ensures that no control character (except CR/LF) make it into the saved file. Thats in the new release a day or so ago.

Davo

[davidbannon]

OK, while not being able to replicate this problem, the newly released version 0.35 does have some additional safe guards against it happening. So, I'll close Thanks for the report. Davo

[monperrus]

Thanks Davo!

[monperrus]

FTR, today a note was corrupted with non-printable "^R"

-<monospace>% EF^RCF: High Performance Smart Contract Fuzzing for Exploit Generation April 2023</monospace>
+<monospace>% EFCF: High Performance Smart Contract Fuzzing for Exploit Generation April 2023</monospace>

[davidbannon]

Hmm, monperrus, do you have any idea as to how it got into the system ? Entered directly from a keyboard, copy and pasted from another document ? Come via sync (ie github), come in on an imported document ?

I believe I have blocked all such paths but obviously I have not. Do you use Ctrl-R as an OS shortcut to something ? Seems it refreshes browers and has a number of other roles.

What really worries me is that tomboy-ng will not save ctrl-R, ascii 18, its definitely on a list of ctrl characters filtered out during the save process (ref Davo, notenormal.pas, line 300). Does anything else have write access to your notes ? (old Tomboy, old tomboy-ng, g-note, tomboy-reborn, the nextcloud sync tool, cannot remember its name.....

Davo

[monperrus]

Entered directly from a keyboard, copy and pasted from another document ?

almost sure copy-pasted

its definitely on a list of ctrl characters filtered out during the save process (ref Davo, notenormal.pas, line 300)

yes, but that's from a recent commit https://github.com/tomboy-notes/tomboy-ng/commit/98669c35f214834ee3d97cd171c0d704f01304b6, which is maybe not in the version I'm using (tomboy-ng version 0.34a)

[monperrus]

just installed tomboy-ng_0.36c-0_amd64.deb

[davidbannon]

Good, that fix I mentioned came in AFTER 0.35 so, yes, appears you were using a version with (EDIT without) the fix. Lets hope this gets it. Davo