Closed aberfan closed 1 year ago
Are we talking about an highlighting issue or a preview issue?
It's wholly a preview issue. In the editor, lines show up. The preview window simply deletes them. This is repeatable for me.
The expected behavior is that I'd actually see everything inside the code fence, not a bunch of empty lines. It appears that it's rendering the HTML; it's like what happens here on github if you don't use the backticks to escape the HTML.
And, to make sure it's not something stupid I'm doing, I extracted v23.3.7 to a new directory, didn't change any setting, ran as portable (in case that is the thing that's different) and pasted in one of the notes where this is happening.
Edit: Removed code and screenshots; see post below.
This is really bugging me. Since I can't help with code, I've tried to come up with better information.
Here's a broken note:
Snippet2
========
Testing:
```js
<input id="test" name="attribution" type="radio" value="fully">
<BR>
<span>Text</span>
![image](https://user-images.githubusercontent.com/72705523/226489638-9015b116-4f65-4a91-9e01-9ab54e963cc7.png)
The screenshot is how that's rendered.
But.
Here's the same exact thing as above:
![image](https://user-images.githubusercontent.com/72705523/226489711-87043a5e-bd2f-4e29-9b07-6bc80adb884f.png)
There's only one difference between them: in the second, I removed a single space after the word "Testing:".
This is completely virgin. I did this wholly in QOwnNotes, on a different system, without changing any settings. I'm curious if this is reproducible for anyone. I mentioned something similar in the other (@ in links) thread where it suddenly rendered if I added a line break. Unfortunately, in my real note (which is 641 lines), I haven't been able to get the actual blocks to properly show up in preview; there's a lot of non-code sections before it.
Here's some system information, and again, this is from simply extracting the newest QOwnNotes into a directory and running and entering the snippet above, so I'm not doing anything funky here.
<details>
<summary>
QOwnNotes Debug Information
</summary>
===========================
Current Date: Mon Mar 20 18:54:41 2023
Version: 23.3.7
Build date: Mar 20 2023
Build number: 1023
Platform: windows
Operating System: Windows 10 Version 2009
Build architecture: x86_64
Current architecture: x86_64
Release: GitHub Actions
Qt Version (build): 5.15.2
Qt Version (runtime): 5.15.2
Portable mode: yes
Settings path / key: L:/tmp/test/Data/PBE/QOwnNotes.ini
Application database path: L:\tmp\test\Data\QOwnNotes.sqlite
Application arguments: QOwnNotes.exe
, QOwnNotes.exe
, --portable
, --allow-multiple-instances
Qt Debug: 0
Locale (system): en_US
Locale (interface): empty
Primary screen resolution: 1920x1080
Icon theme: breeze-qownnotes
Notes in current note folder: 4
Calendar items: 0
Enabled scripts: 0
Enabled: false
Selected language: empty
Language codes: empty
Language names: empty
Application dictionaries path: L:\tmp\test\Data\dicts
currentNoteFolderId: 1
default
id: 1
isCurrent: yes
activeTagId: 0
localPath: L:\tmp\test\Data\Notes
remotePath: Notes
cloudConnectionId: 1
isShowSubfolders: no
isUseGit: no
allowDifferentNoteFileName: no
activeNoteSubFolder name: empty
database file: L:\tmp\test\Data\Notes\notes.sqlite
ActiveNoteHistoryItem (NoteHistoryItem): <binary data>
Debug/fakeOldVersionNumber (QString): false
Debug/fileLogging (QString): false
Editor/CurrentSchemaKey (QString): EditorColorSchema-6033d61b-cb96-46d5-a3a8-20d5172017eb
Editor/autoBracketClosing (QString): true
Editor/autoBracketRemoval (QString): true
Editor/disableCursorBlinking (QString): false
Editor/editorWidthInDFMOnly (QString): true
Editor/highlightCurrentLine (QString): true
Editor/indentSize (QString): 4
Editor/removeTrailingSpaces (QString): false
Editor/showLineNumbers (QString): false
Editor/useTabIndent (QString): false
Editor/vimMode (QString): false
LastUpdateCheck (QDateTime): 2023-03-20T18:13:05.500
MainWindow/geometry (QByteArray): <binary data>
MainWindow/mainToolBar.iconSize (QString): 30
MainWindow/menuBarGeometry (QByteArray): <binary data>
MainWindow/noteTextEdit.code.font (QString): Courier New,9,-1,2,50,0,0,0,0,0
MainWindow/noteTextEdit.font (QString): MS Shell Dlg 2,7.8,-1,5,50,0,0,0,0,0
MainWindow/noteTextView.code.font (QString): Courier New,9,-1,2,50,0,0,0,0,0
MainWindow/noteTextView.font (QString): MS Shell Dlg 2,7.8,-1,5,50,0,0,0,0,0
MainWindow/noteTextView.ignoreCodeFontSize (QString): true
MainWindow/noteTextView.refreshDebounceTime (QString): 600
MainWindow/noteTextView.rtl (QString): false
MainWindow/noteTextView.underline (QString): true
MainWindow/noteTextView.useEditorStyles (QString): true
MainWindow/noteTextView.useInternalExportStyling (QString): true
MessageBoxOverride/windows-dark-mode (int): 65536
NoteFolder-1/NoteTabNameList (QString): Code Snippets
NoteFolder-1/NoteTabStickinessList: empty
NoteFolder-1/NoteTabSubFolderPathDataList (QString): empty
NoteHistory-1 (QVariantList): <variant list with 4 item(s)>
NoteHistoryCurrentIndex-1 (QString): 3
PiwikClientId (QString): <hidden>
ScriptRepositoryDialog/mainSplitterState (QByteArray): <binary data>
SearchEngineId (QString): 2
SettingsDialog/geometry (QByteArray): <binary data>
SettingsDialog/mainSplitterState (QByteArray): <binary data>
ShowSystemTray (QString): false
StartHidden (QString): false
WelcomeDialog/geometry (QByteArray): <binary data>
acceptAllExternalModifications (QString): false
allowNoteEditing (bool): true
allowOnlyOneAppInstance (QString): true
appMetrics/disableAppHeartbeat (QString): true
appMetrics/disableTracking (QString): true
appMetrics/notificationShown (QString): true
automaticNoteFolderDatabaseClosing (QString): false
checkSpelling (bool): false
closeTodoListAfterSave (QString): false
cryptoKey (QString): <hidden>
currentNoteFolderId (QString): 1
currentWorkspace (QString): initial
cursorWidth (QString): 1
darkMode (QString): false
darkModeColors (QString): true
darkModeIconTheme (QString): false
darkModeTrayIcon (QString): true
defaultNoteFileExtension (QString): md
demoNotesCreated (QString): true
disableAutomaticUpdateDialog (QString): true
disableSavedSearchesAutoCompletion (QString): false
dockWasInitializedOnce (QString): true
enableNoteTree (QString): false
enableSocketServer (QString): true
enableWebAppSupport (QString): false
externalEditorPath (QString): empty
fullyHighlightedBlockquotes (QString): false
gitCommitInterval (QString): 30
gitExecutablePath (QString): empty
gitLogCommand (QString): empty
guiFirstRunInit (bool): true
ignoreAllExternalModifications (QString): false
ignoreAllExternalNoteFolderChanges (QString): false
ignoreNoteSubFolders (QString): ^\.
ignoredNoteFiles (QString): empty
imageScaleDown (QString): false
imageScaleDownMaximumHeight (QString): 1024
imageScaleDownMaximumWidth (QString): 1024
initialLayoutIdentifier (QString): minimal
insertTimeFormat (QString): empty
interfaceFontSize (QString): 11
interfaceLanguage (QString): empty
internalIconTheme (QString): false
itemHeight (QString): 16
legacyLinking (QString): false
localTrash/autoCleanupDays (QString): 30
localTrash/autoCleanupEnabled (QString): true
localTrash/supportEnabled (QString): true
markdownHighlightingEnabled (QString): true
navigationPanelAutoSelect (QString): true
navigationPanelHideSearch (QString): false
networking/ignoreSSLErrors (QString): true
networking/proxyType (QString): 2
newNoteAskHeadline (QString): false
noteEditIsCentralWidget (QString): true
noteFileExtensionList (QStringList): md, txt
noteListPreview (QString): false
noteSaveIntervalTime (QString): 10
noteSubfoldersPanelDisplayAsFullTree (QString): true
noteSubfoldersPanelHideSearch (QString): false
noteSubfoldersPanelOrder (QString): 0
noteSubfoldersPanelShowFullPath (QString): false
noteSubfoldersPanelShowNotesRecursively (QString): false
noteSubfoldersPanelShowRootFolderName (QString): true
noteSubfoldersPanelSort (QString): 0
notesPanelOrder (QString): 0
notesPanelSort (QString): 1
notesPath (QString): Notes
notifyAllExternalModifications (QString): false
overrideInterfaceFontSize (QString): false
ownCloud/supportEnabled (QString): false
ownCloud/todoCalendarBackend (QString): 3
ownCloud/todoCalendarCalDAVPassword (QString): <hidden>
ownCloud/todoCalendarCalDAVServerUrl (QString): empty
ownCloud/todoCalendarCalDAVUsername (QString): empty
ownCloud/todoCalendarCloudConnectionId (QString): 1
ownCloud/todoCalendarDisplayNameList: empty
ownCloud/todoCalendarEnabledList: empty
ownCloud/todoCalendarEnabledUrlList: empty
ownCloud/todoCalendarUrlList: empty
ownCloudInfo/appIsValid (bool): false
ownCloudInfo/connectionErrorMessage (QString): empty
ownCloudInfo/notesPathExistsText (QString): empty
ownCloudInfo/serverVersion (QString): empty
restoreCursorPosition (QString): true
restoreLastNoteAtStartup (QString): true
restoreNoteTabs (QString): true
savedSearches/noteFolder-1 (QStringList): Snippet2
showMatches (QString): true
showMenuBar (bool): true
showStatusBar (bool): true
startInReadOnlyMode (QString): false
systemIconTheme (QString): false
taggingShowNotesRecursively (QString): false
tagsPanelHideNoteCount (QString): false
tagsPanelHideSearch (QString): false
tagsPanelOrder (QString): 0
tagsPanelSort (QString): 0
todoCalendarSupport (QString): true
toolbar/1/items (QStringList): action_New_note, action_Find_note, action_Remove_note, action_Open_note_in_external_editor, actionShow_local_trash, actionAllow_note_editing, , action_Back_in_note_history, action_Forward_in_note_history, , action_Find_text_in_note, actionReplace_in_current_note, , actionShow_versions, actionShow_trash, actionShare_note, , actionShow_Todo_List
toolbar/1/name (QString): mainToolBar
toolbar/1/title (QString): main toolbar
toolbar/2/items (QStringList): actionFormat_text_bold, actionFormat_text_italic, actionStrike_out_text, actionInsert_code_block, actionInsert_block_quote
toolbar/2/name (QString): formattingToolbar
toolbar/2/title (QString): formatting toolbar
toolbar/3/items (QStringList): actionInsert_text_link, actionInsert_image, actionInsert_current_time
toolbar/3/name (QString): insertingToolbar
toolbar/3/title (QString): inserting toolbar
toolbar/4/items (QStringList): action_Encrypt_note, actionEdit_encrypted_note, actionDecrypt_note
toolbar/4/name (QString): encryptionToolbar
toolbar/4/title (QString): encryption toolbar
toolbar/5/items (QStringList): actionWorkspaceComboBox, actionStore_as_new_workspace, actionRemove_current_workspace, actionRename_current_workspace, actionSwitch_to_previous_workspace, actionUnlock_panels, , actionToggle_distraction_free_mode, action_Increase_note_text_size, action_Decrease_note_text_size, action_Reset_note_text_size
toolbar/5/name (QString): windowToolbar
toolbar/5/title (QString): window toolbar
toolbar/6/items (QString): action_Quit
toolbar/6/name (QString): quitToolbar
toolbar/6/title (QString): quit toolbar
toolbar/size (QString): 6
useNoteFolderButtons (QString): false
useUNIXNewline (QString): false
webAppClientService/serverUrl (QString): wss://app.qownnotes.org
webAppClientService/token (QString): <hidden>
webSocketServerService/bookmarksNoteName (QString): Bookmarks
webSocketServerService/bookmarksTag (QString): bookmarks
webSocketServerService/commandSnippetsNoteName (QString): Commands
webSocketServerService/commandSnippetsTag (QString): commands
webSocketServerService/port (QString): 22222
workspace-initial/name (QString): Minimal
workspace-initial/noteSubFolderDockWidgetVisible (QString): true
workspace-initial/windowState (QByteArray): <binary data>
workspaces (QString): initial
</details>
I realized after re-reading this I left "js" for the syntax highlighting, but it doesn't matter; changing to "html" which would be proper, still demonstrates this.
For some reason, the preview is trying to actually render HTML in code blocks. I think it really shouldn't do that.
I can confirm. I am several versions behind: 22.12.3 build 1002 on Windows 10.
Testing:(no spaces) - works Testing:(one space) - works Testing:(two spaces) - the issue reported
"Unfortunately, in my real note (which is 641 lines), I haven't been able to get the actual blocks to properly show up in preview; there's a lot of non-code sections before it."
I pasted your example at the end of a note with other markdown and code blocks and so on. I scrolled to the middle of the note and placed two spaces after a line before another code block and the issue appeared in your code block.
I pasted your example at the end of a note with other markdown and code blocks and so on. I scrolled to the middle of the note and placed two spaces after a line before another code block and the issue appeared in your code block.
Hi; thanks for confirming this, so @pbek knows I'm not completely insane. When he tried to reproduce this before, he wasn't able to.
I want to try to clarify what I meant about my real note: This issue came to light because I have a very long note, with many different code snippets, as well as unrelated items (headers and notes). So the actual code snippet is well below some other junk; none of which is html in a code block. In the actual section with it, no matter what space I add or remove, or lines, I can't get it to properly render, within that note. So, I'm assuming one of the many many items above it must need modified somehow.
I wonder if this issue is how some regular expression is, that's getting messed up if there's an unexpected space or something.
I messed around a bit more with this, and it's just weird. Two spaces after "testing" and it doesn't render. Remove all spaces, it renders. Add a line break, renders, add another, renders, add a third or more, and it doesn't. However, simply adding the word "test" (no spaces or linebreak) after however many before, and it renders again.
It seems like there's something parsing that's really very, very particular in how it's presented with whitespace before a codeblock, and if it's not within some exact match, isn't properly terminating the markdown outside the block but passing it to be rendered as HTML?
Snippet2
I can confirm that behavior.
First I thought this is an issue of QTextBrowser again, like in #2755, because the HTML code from md4c seemed ok (with a <pre>
around and so on), but then I realized not even Firefox is able to render it "correctly". It seems to be an issue by md4c!
Testing:
<pre><code class="language-js"><input id="test" name="attribution" type="radio" value="fully">
<BR>
<span>Text</span></code></pre>
Firefox shows the radio!
Testing:
<pre><code class="language-js"><input id=<span class="code-string">"test"</span> name=<span class="code-string">"attribution"</span> type=<span class="code-string">"radio"</span> value=<span class="code-string">"fully"</span>>
<BR>
<span>Text</span></code></pre>
Note that there is escaped far more by md4c! Firefox shows the code.
@Waqar144, how do you see this?
Md4c doesn't touch our code blocks iirc. So must be an issue in the code Highlighter
Ah, highlightCode
is called on the Markdown, not the HTML. I removed all the code for testing that is executed on the Markdown before md4c. The HTML is still the same. The one difference tho is that now the the the code for the radio is missing all the time, like in my first screenshot. 😁 So the highlightCode
even helped a bit...
Yeah such simple stuff will not work.
It looks like highlightCode
is the only reason code blocks with html (regardless of highlighting) even work! Seems like md4c doesn't support that at all... highlightCode
is doing the escaping of the html code!
It does work without that, it fails because the md4c we use has a couple of patches that enforce this behavior.
So highlightCode
has an issue with those two trailing spaces after Testing:
?
Looking at the resulting html from both cases can help understand the issue. Ultimately without debugging its hard to tell what's going wrong
As long there are trailing spaces after Testing:
the method highlightCode
does nothing:
[10:37:56] [debug] textToMarkdownHtml - 'str1': "Code Block Test\n========\n\nTesting: \n\n\n```js\n<input id=\"test\" name=\"attribution\" type=\"radio\" value=\"fully\">\n<BR>\n<span>Text</span>\n```\n"
[10:37:56] [debug] textToMarkdownHtml - 'str2': "Code Block Test\n========\n\nTesting: \n\n\n```js\n<input id=\"test\" name=\"attribution\" type=\"radio\" value=\"fully\">\n<BR>\n<span>Text</span>\n```\n"
(str1: before highlightCode, str2: after highlightCode)
So it's just failing to grab the code block from text, That might be the reason
I found the issue, it's the c.isSpace()
in https://github.com/pbek/QOwnNotes/blob/a32cf69c6eefa5113cb3fd5f107370f09484477f/src/entities/note.cpp#L2272
It also counts newlines as "space". When I change it to c == QChar(' ')
it works.
@Waqar144, what is this check supposed to do?
There now is a new release, could you please test it and report if it works for you?
There now is a new release, could you please test it and report if it works for you?
Yes, it works splendidly, on the ones I knew were breaking. Most excellent; thank you for getting to the bottom of this and fixing it!
I'll try to stop doing things that apparently no one else is. (Now I'm sort of wondering what other people actually use to store code fragments...)
Thank you for testing. You are a good fuzzy test 😁
Expected behavior is that the only changes inside a code block/fence is that syntax highlighting will be applied.
Instead, pieces of HTML disappear completely from the preview/render view. Sometimes this is mitigated by making an edit within QOwnNotes, but it's imprecise.