HTML is parsed inside of code blocks.

[aberfan]

Expected behavior is that the only changes inside a code block/fence is that syntax highlighting will be applied.

Instead, pieces of HTML disappear completely from the preview/render view. Sometimes this is mitigated by making an edit within QOwnNotes, but it's imprecise.

Codetest 2
========
```html
class floatPanel extends HTMLElement {
   <div id="floatPanelCustom" class="noprint">
        <span class="labelText">Utility:</span>
        <BR>
        <BR>
        <button id="fixFormatBtn" type="button">Fix Formatting</button>
        <button id="saveDataBtn" type="button">Save Values</button>
        <BR>
        <BR>
        <span id="statusText">Status</span>
   </div>

![a4168c40a70422bfdc6d4fa8ab37e9ca](https://user-images.githubusercontent.com/72705523/218612173-e63b0b3a-9882-4a13-a510-d07b7c49a6df.png)
![18ecd52a40ffbab3f5b827c0d8a1f541](https://user-images.githubusercontent.com/72705523/218612211-2b3d97e6-bcde-4e04-a309-42c74fa8e0f3.png)

[pbek]

Are we talking about an highlighting issue or a preview issue?

[aberfan]

It's wholly a preview issue. In the editor, lines show up. The preview window simply deletes them. This is repeatable for me.

The expected behavior is that I'd actually see everything inside the code fence, not a bunch of empty lines. It appears that it's rendering the HTML; it's like what happens here on github if you don't use the backticks to escape the HTML.

And, to make sure it's not something stupid I'm doing, I extracted v23.3.7 to a new directory, didn't change any setting, ran as portable (in case that is the thing that's different) and pasted in one of the notes where this is happening.

Edit: Removed code and screenshots; see post below.

[aberfan]

This is really bugging me. Since I can't help with code, I've tried to come up with better information.

Here's a broken note:

Snippet2
========

Testing:  

```js
<input id="test" name="attribution" type="radio" value="fully">
<BR>
<span>Text</span>

![image](https://user-images.githubusercontent.com/72705523/226489638-9015b116-4f65-4a91-9e01-9ab54e963cc7.png)

The screenshot is how that's rendered.

But.

Here's the same exact thing as above: 
![image](https://user-images.githubusercontent.com/72705523/226489711-87043a5e-bd2f-4e29-9b07-6bc80adb884f.png)

There's only one difference between them: in the second, I removed a single space after the word "Testing:".

This is completely virgin. I did this wholly in QOwnNotes, on a different system, without changing any settings.  I'm curious if this is reproducible for anyone.  I mentioned something similar in the other (@ in links) thread where it suddenly rendered if I added a line break.  Unfortunately, in my real note (which is 641 lines), I haven't been able to get the actual blocks to properly show up in preview; there's a lot of non-code sections before it.

Here's some system information, and again, this is from simply extracting the newest QOwnNotes into a directory and running and entering the snippet above, so I'm not doing anything funky here.
<details>
  <summary>
QOwnNotes Debug Information
  </summary>

===========================

General Info

Current Date: Mon Mar 20 18:54:41 2023 Version: 23.3.7 Build date: Mar 20 2023 Build number: 1023 Platform: windows Operating System: Windows 10 Version 2009 Build architecture: x86_64 Current architecture: x86_64 Release: GitHub Actions Qt Version (build): 5.15.2 Qt Version (runtime): 5.15.2 Portable mode: yes Settings path / key: L:/tmp/test/Data/PBE/QOwnNotes.ini Application database path: L:\tmp\test\Data\QOwnNotes.sqlite Application arguments: QOwnNotes.exe, QOwnNotes.exe, --portable, --allow-multiple-instances Qt Debug: 0 Locale (system): en_US Locale (interface): empty Primary screen resolution: 1920x1080 Icon theme: breeze-qownnotes Notes in current note folder: 4 Calendar items: 0 Enabled scripts: 0

Spellchecking

Enabled: false Selected language: empty Language codes: empty Language names: empty Application dictionaries path: L:\tmp\test\Data\dicts

Note folders

currentNoteFolderId: 1

Note folder default

id: 1 isCurrent: yes activeTagId: 0 localPath: L:\tmp\test\Data\Notes remotePath: Notes cloudConnectionId: 1 isShowSubfolders: no isUseGit: no allowDifferentNoteFileName: no activeNoteSubFolder name: empty database file: L:\tmp\test\Data\Notes\notes.sqlite

Enabled scripts

Settings

ActiveNoteHistoryItem (NoteHistoryItem): <binary data> Debug/fakeOldVersionNumber (QString): false Debug/fileLogging (QString): false Editor/CurrentSchemaKey (QString): EditorColorSchema-6033d61b-cb96-46d5-a3a8-20d5172017eb Editor/autoBracketClosing (QString): true Editor/autoBracketRemoval (QString): true Editor/disableCursorBlinking (QString): false Editor/editorWidthInDFMOnly (QString): true Editor/highlightCurrentLine (QString): true Editor/indentSize (QString): 4 Editor/removeTrailingSpaces (QString): false Editor/showLineNumbers (QString): false Editor/useTabIndent (QString): false Editor/vimMode (QString): false LastUpdateCheck (QDateTime): 2023-03-20T18:13:05.500 MainWindow/geometry (QByteArray): <binary data> MainWindow/mainToolBar.iconSize (QString): 30 MainWindow/menuBarGeometry (QByteArray): <binary data> MainWindow/noteTextEdit.code.font (QString): Courier New,9,-1,2,50,0,0,0,0,0 MainWindow/noteTextEdit.font (QString): MS Shell Dlg 2,7.8,-1,5,50,0,0,0,0,0 MainWindow/noteTextView.code.font (QString): Courier New,9,-1,2,50,0,0,0,0,0 MainWindow/noteTextView.font (QString): MS Shell Dlg 2,7.8,-1,5,50,0,0,0,0,0 MainWindow/noteTextView.ignoreCodeFontSize (QString): true MainWindow/noteTextView.refreshDebounceTime (QString): 600 MainWindow/noteTextView.rtl (QString): false MainWindow/noteTextView.underline (QString): true MainWindow/noteTextView.useEditorStyles (QString): true MainWindow/noteTextView.useInternalExportStyling (QString): true MessageBoxOverride/windows-dark-mode (int): 65536 NoteFolder-1/NoteTabNameList (QString): Code Snippets NoteFolder-1/NoteTabStickinessList: empty NoteFolder-1/NoteTabSubFolderPathDataList (QString): empty NoteHistory-1 (QVariantList): <variant list with 4 item(s)> NoteHistoryCurrentIndex-1 (QString): 3 PiwikClientId (QString): <hidden> ScriptRepositoryDialog/mainSplitterState (QByteArray): <binary data> SearchEngineId (QString): 2 SettingsDialog/geometry (QByteArray): <binary data> SettingsDialog/mainSplitterState (QByteArray): <binary data> ShowSystemTray (QString): false StartHidden (QString): false WelcomeDialog/geometry (QByteArray): <binary data> acceptAllExternalModifications (QString): false allowNoteEditing (bool): true allowOnlyOneAppInstance (QString): true appMetrics/disableAppHeartbeat (QString): true appMetrics/disableTracking (QString): true appMetrics/notificationShown (QString): true automaticNoteFolderDatabaseClosing (QString): false checkSpelling (bool): false closeTodoListAfterSave (QString): false cryptoKey (QString): <hidden> currentNoteFolderId (QString): 1 currentWorkspace (QString): initial cursorWidth (QString): 1 darkMode (QString): false darkModeColors (QString): true darkModeIconTheme (QString): false darkModeTrayIcon (QString): true defaultNoteFileExtension (QString): md demoNotesCreated (QString): true disableAutomaticUpdateDialog (QString): true disableSavedSearchesAutoCompletion (QString): false dockWasInitializedOnce (QString): true enableNoteTree (QString): false enableSocketServer (QString): true enableWebAppSupport (QString): false externalEditorPath (QString): empty fullyHighlightedBlockquotes (QString): false gitCommitInterval (QString): 30 gitExecutablePath (QString): empty gitLogCommand (QString): empty guiFirstRunInit (bool): true ignoreAllExternalModifications (QString): false ignoreAllExternalNoteFolderChanges (QString): false ignoreNoteSubFolders (QString): ^\. ignoredNoteFiles (QString): empty imageScaleDown (QString): false imageScaleDownMaximumHeight (QString): 1024 imageScaleDownMaximumWidth (QString): 1024 initialLayoutIdentifier (QString): minimal insertTimeFormat (QString): empty interfaceFontSize (QString): 11 interfaceLanguage (QString): empty internalIconTheme (QString): false itemHeight (QString): 16 legacyLinking (QString): false localTrash/autoCleanupDays (QString): 30 localTrash/autoCleanupEnabled (QString): true localTrash/supportEnabled (QString): true markdownHighlightingEnabled (QString): true navigationPanelAutoSelect (QString): true navigationPanelHideSearch (QString): false networking/ignoreSSLErrors (QString): true networking/proxyType (QString): 2 newNoteAskHeadline (QString): false noteEditIsCentralWidget (QString): true noteFileExtensionList (QStringList): md, txt noteListPreview (QString): false noteSaveIntervalTime (QString): 10 noteSubfoldersPanelDisplayAsFullTree (QString): true noteSubfoldersPanelHideSearch (QString): false noteSubfoldersPanelOrder (QString): 0 noteSubfoldersPanelShowFullPath (QString): false noteSubfoldersPanelShowNotesRecursively (QString): false noteSubfoldersPanelShowRootFolderName (QString): true noteSubfoldersPanelSort (QString): 0 notesPanelOrder (QString): 0 notesPanelSort (QString): 1 notesPath (QString): Notes notifyAllExternalModifications (QString): false overrideInterfaceFontSize (QString): false ownCloud/supportEnabled (QString): false ownCloud/todoCalendarBackend (QString): 3 ownCloud/todoCalendarCalDAVPassword (QString): <hidden> ownCloud/todoCalendarCalDAVServerUrl (QString): empty ownCloud/todoCalendarCalDAVUsername (QString): empty ownCloud/todoCalendarCloudConnectionId (QString): 1 ownCloud/todoCalendarDisplayNameList: empty ownCloud/todoCalendarEnabledList: empty ownCloud/todoCalendarEnabledUrlList: empty ownCloud/todoCalendarUrlList: empty ownCloudInfo/appIsValid (bool): false ownCloudInfo/connectionErrorMessage (QString): empty ownCloudInfo/notesPathExistsText (QString): empty ownCloudInfo/serverVersion (QString): empty restoreCursorPosition (QString): true restoreLastNoteAtStartup (QString): true restoreNoteTabs (QString): true savedSearches/noteFolder-1 (QStringList): Snippet2 showMatches (QString): true showMenuBar (bool): true showStatusBar (bool): true startInReadOnlyMode (QString): false systemIconTheme (QString): false taggingShowNotesRecursively (QString): false tagsPanelHideNoteCount (QString): false tagsPanelHideSearch (QString): false tagsPanelOrder (QString): 0 tagsPanelSort (QString): 0 todoCalendarSupport (QString): true toolbar/1/items (QStringList): action_New_note, action_Find_note, action_Remove_note, action_Open_note_in_external_editor, actionShow_local_trash, actionAllow_note_editing, , action_Back_in_note_history, action_Forward_in_note_history, , action_Find_text_in_note, actionReplace_in_current_note, , actionShow_versions, actionShow_trash, actionShare_note, , actionShow_Todo_List toolbar/1/name (QString): mainToolBar toolbar/1/title (QString): main toolbar toolbar/2/items (QStringList): actionFormat_text_bold, actionFormat_text_italic, actionStrike_out_text, actionInsert_code_block, actionInsert_block_quote toolbar/2/name (QString): formattingToolbar toolbar/2/title (QString): formatting toolbar toolbar/3/items (QStringList): actionInsert_text_link, actionInsert_image, actionInsert_current_time toolbar/3/name (QString): insertingToolbar toolbar/3/title (QString): inserting toolbar toolbar/4/items (QStringList): action_Encrypt_note, actionEdit_encrypted_note, actionDecrypt_note toolbar/4/name (QString): encryptionToolbar toolbar/4/title (QString): encryption toolbar toolbar/5/items (QStringList): actionWorkspaceComboBox, actionStore_as_new_workspace, actionRemove_current_workspace, actionRename_current_workspace, actionSwitch_to_previous_workspace, actionUnlock_panels, , actionToggle_distraction_free_mode, action_Increase_note_text_size, action_Decrease_note_text_size, action_Reset_note_text_size toolbar/5/name (QString): windowToolbar toolbar/5/title (QString): window toolbar toolbar/6/items (QString): action_Quit toolbar/6/name (QString): quitToolbar toolbar/6/title (QString): quit toolbar toolbar/size (QString): 6 useNoteFolderButtons (QString): false useUNIXNewline (QString): false webAppClientService/serverUrl (QString): wss://app.qownnotes.org webAppClientService/token (QString): <hidden> webSocketServerService/bookmarksNoteName (QString): Bookmarks webSocketServerService/bookmarksTag (QString): bookmarks webSocketServerService/commandSnippetsNoteName (QString): Commands webSocketServerService/commandSnippetsTag (QString): commands webSocketServerService/port (QString): 22222 workspace-initial/name (QString): Minimal workspace-initial/noteSubFolderDockWidgetVisible (QString): true workspace-initial/windowState (QByteArray): <binary data> workspaces (QString): initial

</details>

I realized after re-reading this I left "js" for the syntax highlighting, but it doesn't matter; changing to "html" which would be proper, still demonstrates this.

For some reason, the preview is trying to actually render HTML in code blocks. I think it really shouldn't do that.

[Kvcd12]

I can confirm. I am several versions behind: 22.12.3 build 1002 on Windows 10.

Testing:(no spaces) - works Testing:(one space) - works Testing:(two spaces) - the issue reported

"Unfortunately, in my real note (which is 641 lines), I haven't been able to get the actual blocks to properly show up in preview; there's a lot of non-code sections before it."

I pasted your example at the end of a note with other markdown and code blocks and so on. I scrolled to the middle of the note and placed two spaces after a line before another code block and the issue appeared in your code block.

[aberfan]

I pasted your example at the end of a note with other markdown and code blocks and so on. I scrolled to the middle of the note and placed two spaces after a line before another code block and the issue appeared in your code block.

Hi; thanks for confirming this, so @pbek knows I'm not completely insane. When he tried to reproduce this before, he wasn't able to.

I want to try to clarify what I meant about my real note: This issue came to light because I have a very long note, with many different code snippets, as well as unrelated items (headers and notes). So the actual code snippet is well below some other junk; none of which is html in a code block. In the actual section with it, no matter what space I add or remove, or lines, I can't get it to properly render, within that note. So, I'm assuming one of the many many items above it must need modified somehow.

I wonder if this issue is how some regular expression is, that's getting messed up if there's an unexpected space or something.

I messed around a bit more with this, and it's just weird. Two spaces after "testing" and it doesn't render. Remove all spaces, it renders. Add a line break, renders, add another, renders, add a third or more, and it doesn't. However, simply adding the word "test" (no spaces or linebreak) after however many before, and it renders again.

It seems like there's something parsing that's really very, very particular in how it's presented with whitespace before a codeblock, and if it's not within some exact match, isn't properly terminating the markdown outside the block but passing it to be rendered as HTML?

[pbek]

Snippet2

I can confirm that behavior. First I thought this is an issue of QTextBrowser again, like in #2755, because the HTML code from md4c seemed ok (with a <pre> around and so on), but then I realized not even Firefox is able to render it "correctly". It seems to be an issue by md4c!

md4c code-block HTML output with two spaces after Testing:

<pre><code class="language-js"><input id="test" name="attribution" type="radio" value="fully">
<BR>
<span>Text</span></code></pre>

Firefox shows the radio!

md4c code-block HTML output with no spaces after Testing:

<pre><code class="language-js">&lt;input id=<span class="code-string">&quot;test&quot;</span> name=<span class="code-string">&quot;attribution&quot;</span> type=<span class="code-string">&quot;radio&quot;</span> value=<span class="code-string">&quot;fully&quot;</span>&gt;
&lt;BR&gt;
&lt;span&gt;Text&lt;&#47;span&gt;</code></pre>

Note that there is escaped far more by md4c! Firefox shows the code.

[pbek]

@Waqar144, how do you see this?

[Waqar144]

Md4c doesn't touch our code blocks iirc. So must be an issue in the code Highlighter

[pbek]

Ah, highlightCode is called on the Markdown, not the HTML. I removed all the code for testing that is executed on the Markdown before md4c. The HTML is still the same. The one difference tho is that now the the the code for the radio is missing all the time, like in my first screenshot. 😁 So the highlightCode even helped a bit...

[Waqar144]

Yeah such simple stuff will not work.

[pbek]

It looks like highlightCode is the only reason code blocks with html (regardless of highlighting) even work! Seems like md4c doesn't support that at all... highlightCode is doing the escaping of the html code!

[Waqar144]

It does work without that, it fails because the md4c we use has a couple of patches that enforce this behavior.

[pbek]

So highlightCode has an issue with those two trailing spaces after Testing:?

[Waqar144]

Looking at the resulting html from both cases can help understand the issue. Ultimately without debugging its hard to tell what's going wrong

[pbek]

As long there are trailing spaces after Testing: the method highlightCode does nothing:

[10:37:56] [debug] textToMarkdownHtml - 'str1': "Code Block Test\n========\n\nTesting: \n\n\n```js\n<input id=\"test\" name=\"attribution\" type=\"radio\" value=\"fully\">\n<BR>\n<span>Text</span>\n```\n"
[10:37:56] [debug] textToMarkdownHtml - 'str2': "Code Block Test\n========\n\nTesting: \n\n\n```js\n<input id=\"test\" name=\"attribution\" type=\"radio\" value=\"fully\">\n<BR>\n<span>Text</span>\n```\n"

(str1: before highlightCode, str2: after highlightCode)

[Waqar144]

So it's just failing to grab the code block from text, That might be the reason

[pbek]

I found the issue, it's the c.isSpace() in https://github.com/pbek/QOwnNotes/blob/a32cf69c6eefa5113cb3fd5f107370f09484477f/src/entities/note.cpp#L2272

It also counts newlines as "space". When I change it to c == QChar(' ') it works. @Waqar144, what is this check supposed to do?

https://github.com/pbek/QOwnNotes/blob/a32cf69c6eefa5113cb3fd5f107370f09484477f/src/entities/note.cpp#L2270-L2275

[pbek]

23.3.8

• an issue with html tags in code-blocks that where not previewed correctly was fixed

[pbek]

There now is a new release, could you please test it and report if it works for you?

[aberfan]

There now is a new release, could you please test it and report if it works for you?

Yes, it works splendidly, on the ones I knew were breaking. Most excellent; thank you for getting to the bottom of this and fixing it!

I'll try to stop doing things that apparently no one else is. (Now I'm sort of wondering what other people actually use to store code fragments...)

[pbek]

Thank you for testing. You are a good fuzzy test 😁