Some time ago I added a fast-path for ASCII characters in the lexer, which I remember making a huge difference for performance. Apparently, unicode-segmentation has since added a similar fast-path (https://github.com/unicode-rs/unicode-segmentation/pull/79). I should benchmark again, and see if my fast-path still makes much of a difference. If it does, maybe I could make a pull request to unicode-segmentation.
Some time ago I added a fast-path for ASCII characters in the lexer, which I remember making a huge difference for performance. Apparently, unicode-segmentation has since added a similar fast-path (https://github.com/unicode-rs/unicode-segmentation/pull/79). I should benchmark again, and see if my fast-path still makes much of a difference. If it does, maybe I could make a pull request to unicode-segmentation.