search

jamesrhester / Lerche.jl

A Julia port of the Lark parser
MIT License
45 stars 3 forks source link

Feature request: underscore filter #13

Closed guyvdbroeck closed 3 years ago

guyvdbroeck commented 3 years ago

I noticed that Lark has a way to filter out tokens when their name starts with an underscore. I'm hoping that the same functionality in Lerche could speed up my parser.

For context, I am trying to replace my hacky string-splitting parsers, and started with this simple one: https://github.com/Juice-jl/LogicCircuits.jl/blob/parser/src/LoadSave/vtree_loaders.jl Unfortunately the grammar is whitespace-sensitive, so I end up with a lot of NEWLINE and SPACE in the tree which I suppose slows things down.

PS: Lerche is already pretty awesome; I've been waiting for a fast but easy to use parser in Julia for years...

jamesrhester commented 3 years ago

While I haven't tested this functionality specifically, it should work out of the box. test_ignore in test/test_parser.jl I think tests this functionality (also by dropping whitespace). I doubt that it will speed up the parse process that much as the regex matching is a significant part of the parse time now. It might speed up the parse tree transform, worth a try of course.

Glad that Lerche might be useful to you!

guyvdbroeck commented 3 years ago

It works, thanks! (although indeed no significant runtime improvement)