search

tree-sitter / node-tree-sitter

Node.js bindings for tree-sitter
https://www.npmjs.com/package/tree-sitter
MIT License
624 stars 107 forks source link

Special characters in rule names cause Uncaught SyntaxError #173

Open rien opened 10 months ago

rien commented 10 months ago

I've encountered this issue when trying to load the node bindings of the tree-sitter-r parser with their 'next' branch. This grammar contains a lot of special symbols and as the node bindings generate a new class for each rule with its name, this causes an Uncaught SyntaxError when the class definition is evaled.

Now I've tried to fix this by creating a PR (https://github.com/r-lib/tree-sitter-r/pull/64) renaming all rules with special characters, but this is not as desired since it renames most of the original grammar.

What do you recommend a a good solution here? Should rule names with special characters be disallowed in grammars? Or should the node bindings be able to cope with these special characters as token names?

verhovsky commented 10 months ago

of course the Node bindings should be able to cope with arbitrary text in token names, whatever is supported by the regular library.

amaanq commented 10 months ago

I partially disagree - rules shouldn't be named like that, it makes no sense and it would be correct to abstract those rules to a node containing the special character text. I would argue R's grammar is not exactly written in the "standard" tree-sitter way of writing grammars