List of CSS Tokenizers - Githubissues

stylelint / css-parser

Universal CSS parser

MIT License

3 stars 0 forks source link

List of CSS Tokenizers #1

Open romainmenke opened 1 year ago

romainmenke commented 1 year ago

This is a list of CSS Tokenizers. This issue is not intended for in depth discussion about any individual tokenizer or any aspect of CSS tokenizing.

scripthunter7 commented 10 months ago

Hi @romainmenke, thank you for this compilation! We also created a tokenizer for CSS & Extended CSS: https://www.npmjs.com/package/@adguard/css-tokenizer

A detailed benchmark has also been prepared for the tokenizers listed here. Results can be found at the following link: https://github.com/AdguardTeam/tsurlfilter/blob/master/packages/css-tokenizer/benchmark/RESULTS.md

[!NOTE] Our API and token list is also compatible with the CSSTree's tokenizer API, and in the long term, we plan to integrate the tokenizer library into CSSTree via our ECSSTree library. See this issue for more details.

romainmenke commented 10 months ago

Hi @scripthunter7,

Interesting, added to the list! How does it score on these tests : https://github.com/romainmenke/css-tokenizer-tests

romainmenke commented 10 months ago

test results for @adguard/css-tokenizer https://www.romainmenke.com/css-tokenizer-tests

perfect score for widely used CSS in the community (bootstrap, tailwind, ...)
a few issues in more exotic CSS
it's faster than any other tokenizer currently tested

Mouvedia commented 4 months ago

Rome used to have a tokenizer: rome/tools#707