shikijs / shiki

A beautiful yet powerful syntax highlighter
http://shiki.style/
MIT License
9.91k stars 361 forks source link

Option to exclude themeMatches from explanations #736

Closed MichaelMakesGames closed 1 month ago

MichaelMakesGames commented 1 month ago

Clear and concise description of the problem

I want an option to exclude themeMatches from token explanations, but still include scopeName, so that plugins that need scope information can work without the slowdown of including all the themeMatches too

Suggested solution

In @shiki/core codeToTokensBase could check an additional option to add explanations but skip themeMatches

Alternative

No response

Additional context

I wrote a shiki transformer for VSCode-style bracket colorization. It uses the textmate scopes for avoid matching some potential brackets (eg > when used as "less than") and to detect embedded languages (eg ts in a svelte file). It does not use the themeMatches on the explanations. I did some benchmarking and noticed that shiki is ~4.7x slower when includeExplanation is true, but when I manually edit the code to skip adding themeMatches it is only ~2x slower.

Validations

Contributes

antfu commented 1 month ago

I think we could have includeExplanation: 'scope-only' as an option to do so