trishume / syntect

Rust library for syntax highlighting using Sublime Text syntax definitions.
https://docs.rs/syntect
MIT License
1.89k stars 132 forks source link

Consider delaying `parse_regex`'s transformations #541

Open CosmicHorrorDev opened 5 months ago

CosmicHorrorDev commented 5 months ago

syntect performs a few transformations to syntax definition's regexes while loading everything. The most notable one in terms of the final syntax set's size being the expansion of the various variables that can be included with a syntax definition. This expansion can be delayed to when the regex is compiled instead which can significantly reduce the embedded syntax definitions (my notes from testing with generating a syntax set without applying the transformations to regexes say that it took the total size of all of the LazyContexts that are stored in bat's syntax set from 720 to 673 KiB)