Ensure _non_special_token always gives us a token

tree-sitter / tree-sitter-rust

Rust grammar for tree-sitter

MIT License

341 stars 97 forks source link

Ensure _non_special_token always gives us a token #125

Closed Wilfred closed 5 months ago

Wilfred commented 2 years ago

Previously, this parser would discard tokens inside macro invocation. As suggested in https://github.com/tree-sitter/tree-sitter/issues/1156, use a named rule to ensure a token is returned.

Closes #119

aryx commented 2 years ago

@dcreager is there someone at github in charge of maintaining tree-sitter-rust? I see a few PRs without answers.

Wilfred commented 2 years ago

@aryx Looks like the corpus text files were generated with an older version of tree-sitter, but I'm not sure which. I've manually merged my changes in.

aryx commented 2 years ago

BTW @Wilfred did you receive the email I've sent you separately? (from pad@r2c.dev)

Wilfred commented 2 years ago

OK, I've replaced the regex with an explicit list of tokens. I've kept the named node, so it's available in the parse tree.

Please take a look and let me know what you think :)

amaanq commented 5 months ago

closing as stale