tree-sitter / tree-sitter-julia

Julia grammar for Tree-sitter
MIT License
93 stars 31 forks source link

Reduce size and compile time of the parser #136

Closed maxbrunsfeld closed 4 months ago

maxbrunsfeld commented 4 months ago

Fixes https://github.com/tree-sitter/tree-sitter-julia/issues/124 Depends on https://github.com/tree-sitter/tree-sitter/pull/3234

On my M3, compiling the parser to aarch64 now takes about 4 seconds. Compiling to WASM takes 3 minutes, which is still terrible, but better than before.

/cc @savq

savq commented 4 months ago
  • In the grammar, avoid single tokens that match multiple keywords e.g. primitive type, abstract type, as these don't work with keyword extraction, making the main lex function more complex.
  • In the grammar, simplify the argument_list rule, replacing a long highly-specific sequence with a more generic repetitive structure (reduces the parser size by ~6500 states)

I didn't know either of those things 😬

@maxbrunsfeld, could we move the argument_list update to #135? Most of that PR is around removing the conflicts between signatures/parameters and calls/arguments.

savq commented 4 months ago

I'll merge this now. We can regenerate later with a tree-sitter version that includes https://github.com/tree-sitter/tree-sitter/pull/3234/

maxbrunsfeld commented 4 months ago

Thanks, sorry for not responding @savq - https://github.com/tree-sitter/tree-sitter/pull/3234 is almost done.

savq commented 4 months ago

No problem.

I didn't update parser.h in the last generate, so the header still has the set_contains function.