Feature request: support other syntax highlight libraries

[curbengh]

This is a spin out of https://github.com/hexojs/hexo-util/issues/19#issuecomment-539356312.

The idea is to support different libraries other than the current highlight.js, e.g.

• atom's highlights
• prism

A major benefit is that theme dev can use/adapt from much more css. Some library may also be faster than another. https://github.com/hexojs/hexo-util/issues/39 can be addressed by atom and prism.

---

Currently, highlight.js is tightly integrated into hexo, so adding another library is expected not to be straightforward. One possible issue is configuration, which I think should be tackled first.

So, to change library, user can specify engine (I will refer both library and engine interchangeably for the rest of the comment),

highlight:
  engine: hljs|atom|prism

No problem, but here comes the tricky part, configuring each of them. While each engine may share similar options, I think ideally they should be separated, like so:

highlight:
  engine: ...
  hljs:
    gutter: true
  prism: ...

highlight.hljs is preferred over highlight.highlight for clarity. But currently the option is used to address https://github.com/hexojs/hexo-util/issues/19#issuecomment-267040668 using

highlight:
  hljs: true

Retaining compatibility is possible (like https://github.com/hexojs/hexo/pull/3675), i.e. can check highlight.hljs is a boolean or not. If boolean, parse it as highlight.hljs.classPrefix.

highlight:
  hljs:
    classPrefix: ''|'hljs-'

Likewise, the approach could also apply to retain compatibility with the current options:

highlight:
  auto_detect:
  line_number:
  tab_replace:

However, I suggest not to retain old options forever, they can be dropped in hexo@5 for instance, and permanently move to under highlight.hljs.

A side benefit of having hljs-specific option is that users familiar with the engine can use its option naming instead; hexo rename some option which confuse those users. See https://github.com/hexojs/hexo-util/issues/40.

---

Previous suggestion https://github.com/hexojs/hexo/issues/1300

[SukkaW]

In order to retain backward compatibility for all current hexo theme, I suggest to remain current config.highlight there, and add a new config.prismjs option.

Prism related utility has been added in hexojs/hexo-util#168.

Related code to be changed:

• backtick_code block filter
• include_code helper
• code helper
• marked highlight option for hexo-renderer-marked
• And test related to post rendering:
  • https://github.com/hexojs/hexo/blob/master/test/fixtures/post_render.js
  • https://github.com/hexojs/hexo/blob/master/test/scripts/hexo/post.js

[SukkaW]

During https://github.com/hexojs/hexo-util/pull/168, I have noticed PrismJS is mainly designed for working at browser side. Many features likes "highlight specific lines", "The first line number" only work in browser.

So if we are going to bring PrismJS to Hexo, IMHO we should address two options:

1. browser mode. In this mode Hexo will produce HTML snippet that will could be processed by prism.js & prism.css in browser. The full features of PrismJS could be used then, and it will be compatible with all options of current include_code() and code() helper.
2. preprocess mode. In this mode Hexo will produce prism-parsed HTML snippet. Then theme/user only need to add prism.css. However, many features will no longer works under preprocess mode thus will not be compatible with include_code() and code() helper.

---

Also I have noticed something strange:

When using Hexo, there are many ways to add syntax highlighted code to Hexo.

• Use helpers provided by Hexo (likes code()). In code() helper Hexo will call hexo-util.highlight to output highlightjs.css compatible HTML snippet.
• Use markdown syntax (backticks). And it is now much interesting:
  • Hexo has before_post_render filter. When config.highlight.enable is true, Hexo will call hexo-util.highlight to pre-process content wrapped inside backticks, even before hexo-renderer-marked and hexo-renderer-markdown-it.
  • When config.highlight.enable is false, before_post_render filter will do nothing (return directly).
  • In hexo-renderer-marked, The marked.setOptions will still call hexo-util.highlight again to process.

So it leads to a very strange result:

config.highlight.enable: true

By default hljs is disabled and wrap is enabled. Hexo will render something likes:

<figure class="highlight sh">
  <table>
    ...
  </table>
</figure>

config.highlight.enable: false with hexo-renderer-marked

Since highlight has been disabled, the before_post_render filter will do nothing and leave everything to helpers (powered by nunjucks) and marked (from hexo-renderer-marked).

code() helper is designed to leave everything it should be without applying any highlight.js specific options, so it will render something like:

<pre>
  <code>
    ...
  </code>
</pre>

But for code block wrapped in backticks, it will be rendered by hexo-rendered-marked. And hexo-renderer-marked will call hexo-util.highlight with only str and lang option is given. So it will be rendered to:

<pre>
  <code class="bash">
    ...
  </code>
</pre>

config.highlight.enable: false with hexo-rendered-markdown-it

For code() helper, the result is pretty much the same as with hexo-renderer-marked (just as I said, helpers will not be processed by markdown renderer).

For code block wrapped in backticks, since markdown-it itself has no marked.setOptions.highlight like marked, so the result is the same:

<pre>
  <code>
    ...
  </code>
</pre>

---

So to keep the result consistent when config.highlight.enable is set to false, we should remove marked.setOptions.highlight from hexo-renderer-marked.

[SukkaW]

backtick_code filter is important for Hexo. Hexo has extended the "markdown backticks syntax" by supporting caption and many other thing. backtick_code filter also enables the possibility of consistent code highlight across hexo-renderer-marked & hexo-renderer-markdown-it.

---

For PrismJS browser mode, marked.setOptions.highlight shouldn't be used as well: We already have backtick_code filter, and PrismJS in browser could support all the features brought by "Hexo extended markdown backticks syntax"

For PrismJS preprocess mode, backtick_code filter could be disabled as well, because it is quite "useless" ("Hexo extended markdown backticks syntax" related feature will no longer works). If we then use marked.setOptions.highlight to handle them, hexo-renderer-markdown-it will have no syntax highlight).

[SukkaW]

@hexojs/core @curbengh

I have transferred this issue to Hexo repo.

[stevenjoezhang]

Here is a feature request about adding filter event for backtick code block: #1891 This opens up more possibilities for users & plugin developers

[SukkaW]

@stevenjoezhang

What about bring up an new API hexo.extend.highlight? Plugin developer then could provide a function using lang, tab, mark, code to output the highlighted code?

[stevenjoezhang]

This will be a breaking change. Maybe we need to make prism highlight use the same API? https://github.com/hexojs/hexo/pull/4119

[SukkaW]

This will be a breaking change. Maybe we need to make prism highlight use the same API? #4119

I am considering about it.

I still have no idea about how the API should be designed.