Use shiki for code highlight

[felipecrs]

Have you read the Contributing Guidelines on issues?

• [X] I have read the Contributing Guidelines on issues.

Motivation

Have you guys ever considered switching to Shiki for code highlight, instead of Prism?

I find Shiki a lot nicer, with much better grammars that are able to highlight much more code than Prism does.

Also, Shiki is being actively maintained, while Prism seems a little bit abandoned. However, even if Shiki wasn't being maintained, its themes would never stop improving since they are taken from VS Code anyway.

Here is a quick comparison I did:

Prism (dracula - the default theme from Docusaurus):

Shiki (dracula):

PS: I'm aware of docusaurus-preset-shiki-twoslash, and that's what I used for the example above (https://github.com/felipecrs/shiki-docusaurus-test). This is to suggest a better default for the project as a whole.

Maybe it's a good breaking change candidate for Docusaurus v3?

Self-service

• [ ] I'd be willing to do some initial work on this proposal myself.

[Josh-Cena]

Several things, based on my experience with Shiki, do correct me if I'm wrong:

1. Shiki is server-side and generates static markup during Markdown parsing (using a Rehype plugin, IIRC), so there's no way to use it on a React page.
2. Shiki doesn't support as many languages as Prism, neither is it as easy to add them.
3. Shiki being purely server-side means it cannot be used in live code blocks, so there's inconsistency out-of-the-box.

What do you think?

[SuperManito]

Several things, based on my experience with Shiki, do correct me if I'm wrong:

1. Shiki is server-side and generates static markup during Markdown parsing (using a Rehype plugin, IIRC), so there's no way to use it on a React page.
2. Shiki doesn't support as many languages as Prism, neither is it as easy to add them.
3. Shiki being purely server-side means it cannot be used in live code blocks, so there's inconsistency out-of-the-box.

What do you think?

How about Highlight.js?

[felipecrs]

I'm actually confused.

PS: I know nothing about frontend nor React.

1. But looking at Shiki's documentation, I don't see any indicator that it runs on server-side. In fact, it can run on server-side with Node.js. But it also runs in the browser, including a WebAssembly library to perform language tokenization.

   About using with React, I believe this may help: https://www.npmjs.com/package/remark-shiki-twoslash

2. I think that's debatable. If there is a text grammar compatible with VS Code (and several other editors which uses the same standard), it's easy to port to Shiki. For example, I use a lot of Jenkinsfiles, and Prism's support for it is limited to Groovy. There are a few dark VSCode extensions that adds rich syntax highlight for declarative Jenkins pipelines that I could then easily port to my website. I have a feeling that the list of languages supported by Shiki tends to increase more than Prism does, also, maybe the languages not supported by Shiki but supported by Prism aren't that meaningful anwyay.

3. I think that's not actually applicable anymore. See the reference below:

• Apparently Shiki used to be server-side only, that must be what you remember of it: https://github.com/shikijs/shiki/issues/91

How about Highlight.js?

I don't know, but at least it seems more under development than Prism is.

[Josh-Cena]

You mentioned docusaurus-preset-shiki-twoslash. It works by injecting a remark-shiki-twoslash plugin into our content plugin's config. This plugin is a Remark plugin, so it runs on server side. Basically, it takes a Markdown string and spits out a bunch of JSX. However, this transformation is done once and only once when the Markdown is compiled to JSX, but not if your code was authored in JSX to start with.

Now there could be a way to migrate all this logic to client side, sure; but that's not how any existing solution in the wild works, and it will need a lot of prototyping in the community before we can consider its feasibility to be recommended to everyone. There's nothing stopping you from implementing it yourself, or asking the Shiki folks to try it.

[felipecrs]

Alright. That makes sense!

Right now I will not be working on implementing it myself, and I believe Shiki folks are busy working towards Shiki 1.0. Maybe I can follow up with them after.

Thanks for the feedback, I'll close this issue for now (and reopen if something changes).

@SuperManito, I'd recommend you open a separate issue to discuss/propose Highlight.js.

[slorber]

There's also https://bright.codehike.org/

We'll figure out what is the best solution after migrating to server components.

For now the current solution is good enough and the system is flexible enough for you to use Shiki if you want.

[felipecrs]

There's also https://bright.codehike.org/

OMG that's amazing!

[ryanhamilton]

May I suggest you make a library agnostic way of overriding the code highlighter. For different languages people will always want different highlighters. If you give them an extensible method of simply plugging in their own, they will do it. My personal preference is codemirror :)

[slorber]

@ryanhamilton we have 4 ways of doing that:

• implementing your own theme
• providing a remark plugin that process code blocks in Markdown AST, possibly highlighting code at build time instead of runtime
• providing your own @theme/MDXComponents renderer for code blocks
• swizzling the built-in @theme/CodeBlock and modifying it to not use Prism for some languages

That's probably largely enough extension points, we are not coupled to Prism, it is only used in one theme, but you can swap it for something else at multiple levels

[felipecrs]

It looks like Shiki got a complete revamp for 1.0: https://github.com/shikijs/shiki/releases

Apparently things that were previously only server side can now run client side.

[SuperManito]

Shiki may become the industry's best solution for code highlighting.

[johnnyreilly]

I'm actually really keen on Shiki, but the Docusaurus integration of https://github.com/shikijs/twoslash/tree/main/packages/docusaurus-preset-shiki-twoslash unfortunately has issues that mean (much as I'd like to) I've been unable to make use of it on my own site at present:

https://github.com/shikijs/twoslash/issues/120

[felipecrs]

I don't think docusaurus-preset-shiki-twoslash was updated to work with Shiki 1.0 yet, nor Docusaurus 3.0.

[Josh-Cena]

I'm +1 on investigating integration, but I think it would greatly help us if there's a community proof-of-concept. I'm mainly interested because Shiki can do other fun things like showing type/linting errors.

[lachieh]

@Josh-Cena I currently have the slightly older shikiji-rehype plugin implemented on cosmonic.com/docs (example here with wit highlighting) and am in the process of adding the newer shiki-rehype for wasmcloud.com. It took a little wrangling, so if there's interest I can make it into a docusaurus preset with more easily configurable options.

[slorber]

It would be hard to make Shiki the default option of Docusaurus considering we probably want by default to be consistent between the regular code block and the live code block (both using Prism) and afaik live code block is not really an option (nor a good idea?) with Shiki.

---

We usually don't create official plugins for things we don't use ourselves.

However, it's probably worth it to explore how to use Shiki/Twoslash to enhance the experience on our own website, and create an opt-in plugin that users can turn on with some code block metastring on Docusaurus v3.x?

```ts shiki
const hello = "World"

// @module: esnext
// @filename: maths.ts
export function absolute(num: number) {
  if (num < 0) return num * -1;
  return num;
}
// @filename: hello.ts
const hello = "World"
//    ^?

If this opt-in syntax highlighter becomes successful, we could see how to add an option to make it the default in Docusaurus v4. 
(in v3, you could write a remark plugin that adds that meta string for you, although it would be less convenient it's still possible)

---

Note:
- Shiki recently released v1.0 (after merging back the Shikiji fork): https://github.com/shikijs/shiki/releases/tag/v1.0.0
- Twoslash also has a new repo/org/docs and is now integrated as a Shiki transformer: https://github.com/twoslashes/twoslash
- Both tools are in active development
- Shiki now provides a rehype plugin instead of a remark one like previously

---

@lachieh thanks for your proposal, I'm curious to see how you implement such preset. 

To be honest, I think Docusaurus is missing a few primitives to make it possible to implement this as a robust plugin or preset, so it's probably better if we work on it in the main repo, but I'd be curious to look at your solution.

[lachieh]

Thanks, @slorber. That all makes sense. I'll hold off on making the preset for now but if anyone finds their way here from a search, the PR below shows one way to integrate the @shiki/rehype plugin with Docusaurus 3.

https://github.com/wasmCloud/wasmcloud.com/pull/297/files

[userdocs]

@lachieh thanks, I got it working on my install as I wanted to use shiki over prism. Prism is just not as good as shiki when showing more complex snippets.

Here is an example of a 2k line bash script that would have multiple cosmetic issues with prism due to the bash parser getting confused as well as inconsistent highlighting but works fine with shiki.

https://userdocs.github.io/qbittorrent-nox-static/docs/bash/

Now, I understand this is not a normal use case posting a 2k lines of code but i tried many things/langs/examples with prism, I made my own themes and the outcome was never as good as it is with shiki. It's just better.

You can use any vscode json theme and it will look exactly like it does in vscode in docusaurus. I'm using this https://github.com/daltonmenezes/aura-theme/tree/main/packages/vscode

This is one reason I wanted to jump over to astro/starlight which is using this https://github.com/expressive-code/expressive-code + shiki but i'd really like to see it implemented here.

This vscode engine idea is doing something right. It looks amazing.

[slorber]

TIL about https://expressive-code.com, thanks that looks interesting. I'll investigate and try to figure out if we should use this instead of Shiki directly

[kubukoz]

TIL about expressive-code.com, thanks that looks interesting. I'll investigate and try to figure out if we should use this instead of Shiki directly

Hi, @slorber, any success with this?