Highlight plugin doesn't work in Markdown tiddlers

[mardukbp]

In a standard tiddler I can write (I put the backslash in order to print the closing backticks)

```python
a = [i for i in range(10)]
\```

to get syntax highlighting. However, if I change the type of the tiddler to Markdown, the syntax highlighting is gone. I tried with both Gruber and Maruku variants. Is there a special syntax for getting syntax highlighting with Markdown in TiddlyWiki?

[pmario]

The highlight plugin adds a TW parser to recognice ```python in a special way. It adds this to the DOM

<pre class="python hljs ">

The markdown parser doesn't know this syntax element ```python

[Jermolene]

Hi @mardukbp I'm afraid that the highlight plugin is a feature of TiddlyWiki's native wikitext parser, and doesn't work with Markdown. I think making it work might require some changes to the Markdown parser that we use.

[mardukbp]

If TW uses markdown-js, the author is working on implementing GFM, which supports the triple backtick syntax. Hopefully when that is ready, TW will support it.

[Jermolene]

Thanks, that's great news: TiddlyWiki does indeed use https://github.com/evilstreak/markdown-js

[dimvic]

Right now the bundled markdown-js version TiddlyWiki uses (as of 5.1.13) supports the ``` operator. Any hope for highlight support in markdown?

[Jermolene]

Hi @dimvic I'm afraid the situation here hasn't changed; markdown.js is no longer actively maintained, and so there's little prospect of it supporting triple backticks. I'm open to adopting a different, suitable JS Markdown parser, though.

[dimvic]

chjj/marked and jonschlinkert/remarkable can be good options, they both support highlight.js. The former seems more stable and acknowledged, the latter more active.

I tried toying with the Tiddly source to see if it is a snap to integrate either, I kind of got lost somewhere in the middle for no reason and had to give up to leave the office, but I believe it should be very easy.

In general, I agree that the problem isn't much related to the notation used to identify code blocks, it is more about passing the code block identified by the md parser to highlight.js for processing. This would indeed require messing with markdown-js source as you said, but that is not the case for marked and remarkable.

Based on this, I believe switching to either of these might be a good choice after confirming that they are fully compatible with at least Gruber which is the default for Tiddly. Either that or introduce something like x-marked/x-remarkable content type (too messy?).

It's not like markdown code highlighting is that important, the default markup language provided is very good, it would be nice to be able to copy/paste markdown snippets with code and have them highlighted though.

[Jermolene]

Hi @dimvic great stuff, as I say I'd love to have a better Markdown plugin; for backwards compatibility we might as well keep the old plugin around.

If you're interested in taking this on I'd be very happy to help you as much as I can on the TiddlyWiki side.

[syrte]

I've tried to use marked instead of the default markdown-js parser inspired by the MarKed plugin written by Jeffrey Wikinson. It works well and supports the fenced code block and syntax highlighting. I post it below for your information in case anyone want to have a try. https://gist.github.com/syrte/8bd641a43f6b7e1c99f23e8c3ec82c4d

[tinyheero]

Brilliant @syrte! Just tried your gist and it works perfectly solving both the fenced code block and syntax highlighting problems for me. I hope something this could get incorporated into the official markdown plugin...

[dnebauer]

I envy @tinyhero as I followed the instructions from @syrte to switch to the marked parser and it did not fix my problems with code block syntax highlighting. I'm using TW@5.1.19 on nodejs, and perhaps the solution needs to be modified for that environment.

I was unable to follow the linked instructions for switching to the markdown-it parser. They are only an outline of the steps that need to be taken - while plugin devs would be able to "fill in the blanks," that is beyond a tyro like me.

[syrte]

@dnebauer or you can try this https://gist.github.com/syrte/8062e11d8fec6221820c5f998239af61 It's easier to use, you don't have to write files yourself. To use it, you can download and import markdown.tid therein to your TiddlyWiki. It works at least on 5.1.17 on nodejs. This plugin will override the official markdown plugin, use it with your care :)

[dnebauer]

I appreciate your help, @syrte, but the markdown-it instructions you linked to are the ones I referred to previously as being beyond my capabilities as a TW beginner. The tid file includes the "How to make the markdown-it plugin" instructions, and I assume I would have to follow them in order for the parser to work? Those instructions include a lot of steps which are probably very simple once you know how, including:

• Add files/tiddlywiki.files
• Add config.tid
• Edit wrapper.js
• Add EditorToolbar/mono-block.tid
• Add EditorToolbar/timestamp.tid
• Add EditorToolbar/link.tid
• Add Snippets/Table4x3.tid
• Edit readme.tid
• Edit usage.tid
• Add howto.tid

Unfortunately, I don't know how...

Have I misunderstood something?

[syrte]

@dnebauer You don't have to write these things yourself. Just download https://gist.githubusercontent.com/syrte/8062e11d8fec6221820c5f998239af61/raw/571cecc9b3a41a07868d5a8f29604a6340d823bf/markdown.tid as markdown.tid, then pull this file into your tiddlywiki to install.

[dnebauer]

Okay, tried to do that but it blocked the import with the message:

Blocked plugin (due to incoming 5.1.15 being older than existing 5.1.19)

Rather weirdly, if I manually edit markdown.tid so line 10 is version: 5.1.19, the import is still blocked with the message:

Blocked plugin (due to incoming 5.1.19 being older than existing 5.1.19)

Any idea what that's about?

[dnebauer]

@syrte Okay, I managed to install [https://gist.githubusercontent.com/syrte/8062e11d8fec6221820c5f998239af61/raw/571cecc9b3a41a07868d5a8f29604a6340d823bf/markdown.tid](). There were a couple of issues:

First, TW5 refused to import the tiddler until I edited the file and changed the version to 5.1.20 (see previous comment). Once imported, I edited the raw imported tiddler file in my repo ($__plugins_tiddlywiki_markdown.tid) and reset the version to 5.1.19.

Second, if you import the tiddler without first installing the KaTeX plugin it renders tiddlywiki unrunnable: it crashes on startup with the error:

Error executing boot module $:/plugins/tiddlywiki/markdown/markdown-it-katex.js: {"code":"MODULE_NOT_FOUND"}

module.js:549
    throw err;
    ^

Error: Cannot find module '$:/plugins/tiddlywiki/katex/katex.min.js'
    at Function.Module._resolveFilename (module.js:547:15)
    at Function.Module._load (module.js:474:25)
    at Module.require (module.js:596:17)
    at require (internal/module.js:11:18)
    at Object.$tw.modules.execute (/usr/local/lib/node_modules/tiddlywiki/boot/boot.js:720:11)
    at require (/usr/local/lib/node_modules/tiddlywiki/boot/boot.js:679:24)
    at $:/plugins/tiddlywiki/markdown/markdown-it-katex.js:14:13
    at ContextifyScript.Script.runInContext (vm.js:59:29)
    at ContextifyScript.Script.runInNewContext (vm.js:65:15)
    at Object.runInNewContext (vm.js:135:38)

Once I manually removed the $__plugins_tiddlywiki_markdown.tid file I was able to start TW5, install the KaTeX plugin, and copy back $__plugins_tiddlywiki_markdown.tid.

Then, I found that TW5 started successfully and my markdown plugins were rendering flawlessly! I've thrown some moderately complex markdown at it, including tables and images, and it has handled them without problem. Importantly, it formats fenced code blocks properly, which the standard markdown plugin does not. @syrte Thanks very much for your help with this.

[syrte]

@dnebauer Bravo! Many thanks for your feedback. The version number conflict is a known problem, your solution is the simplest one. I shouldn't use the same name as the official plugin at the begin probably... The error with katex is unexpected, I'll fix it sometime.

[tonywoode]

I was wondering if there is any plan to incorporate the above? (since I take the point made in https://github.com/Jermolene/TiddlyWiki5/issues/3667#issuecomment-450538321 that new technical users will probably have some amount of prior expectation that markdown code-highlighing will work)

[Jermolene]

Hi @tonywoode it would certainly be worth revisiting this. Since this ticket was made, we have switched to a different Markdown parser, and now do much more post-processing than before to enable other wikitext features.

[tonywoode]

Thank you Jeremy - I see. After installing that (browser), it appears to me that the standard remarkable markdown plugin well supports all manner of syntax highlighting (in the preview/viewing pane). Amazing! I'll certainly investigate the post-processing!

Potentially fixing this 404 might prevent others ending up cast on the shores of this ticket (mentioned because the results google returns me for "tiddlywiki markdown" don't seem terribly favourable at present)