I want an option to exclude themeMatches from token explanations, but still include scopeName, so that plugins that need scope information can work without the slowdown of including all the themeMatches too
Suggested solution
In @shiki/core codeToTokensBase could check an additional option to add explanations but skip themeMatches
Alternative
No response
Additional context
I wrote a shiki transformer for VSCode-style bracket colorization. It uses the textmate scopes for avoid matching some potential brackets (eg > when used as "less than") and to detect embedded languages (eg ts in a svelte file). It does not use the themeMatches on the explanations. I did some benchmarking and noticed that shiki is ~4.7x slower when includeExplanation is true, but when I manually edit the code to skip adding themeMatches it is only ~2x slower.
Clear and concise description of the problem
I want an option to exclude
themeMatches
from token explanations, but still includescopeName
, so that plugins that need scope information can work without the slowdown of including all the themeMatches tooSuggested solution
In @shiki/core codeToTokensBase could check an additional option to add explanations but skip
themeMatches
Alternative
No response
Additional context
I wrote a shiki transformer for VSCode-style bracket colorization. It uses the textmate scopes for avoid matching some potential brackets (eg
>
when used as "less than") and to detect embedded languages (egts
in asvelte
file). It does not use thethemeMatches
on the explanations. I did some benchmarking and noticed that shiki is ~4.7x slower whenincludeExplanation
is true, but when I manually edit the code to skip addingthemeMatches
it is only ~2x slower.Validations
Contributes