search

uben0 / tree-sitter-typst

Tree Sitter grammar for Typst
MIT License
112 stars 10 forks source link

0.10.0 #16

Closed brettgilio closed 10 months ago

brettgilio commented 10 months ago

Does this project need an update to support 0.10.0?

uben0 commented 10 months ago

Yes. I read through the version changes and there is one breaking change in the syntax:

The _ and * are considered regular text characters when directly surrounded by alphanumeric characters. But with the 0.10 release, the CJK (Chinese - Japanese - Korean) alphanumeric characters won't be part anymore of this exception.

I have to check the Typst implementation to see what exactly is considered "CJK". Anyway. This parsing will have to be done in the C scanner instead of the grammar.js.

Another change to the syntax is the support for other types than strings as dictionary keys. But the grammar already supports any type as dictionary key, so nothing to do for that.

uben0 commented 10 months ago

I found it lexer.rs

uben0 commented 10 months ago

I going to make a program that automatically fetches the files from unicode.org and produces C code with functions testing that a character is part of some unicode set.