search

no-context / moo

Optimised tokenizer/lexer generator! 🐄 Uses /y for performance. Moo.
BSD 3-Clause "New" or "Revised" License
821 stars 65 forks source link

* Lexer.has: return true for error token, if any (fixes #76). #77

Closed JoshuaGrams closed 6 years ago

tjvr commented 6 years ago

Thanks for spotting this!

Please can you add a test for this? :-)

JoshuaGrams commented 6 years ago

Ooops. Added tests.

tjvr commented 6 years ago

I just realised you're the guy who wrote Pint-sized Earley Parser :D I really liked that article, wrote a couple of (unshared) parsers based on your ideas, and indeed I'm slowly trying to work them into Nearley... :-)

JoshuaGrams commented 6 years ago

Heh. Thanks. Not my ideas, though. That's almost straight out of Elizabeth Scott's paper. Loup Vaillant (Earley Parsing Explained) said that he "completely failed to understand" her paper. He presents a more complicated (and slower?) algorithm to construct the parse trees. We had a conversation on the FoNC mailing list and were just talking past each other, so I wrote an implementation to demonstrate how simple it actually is once you strip away all the obfuscating formalisms. Still not sure I actually addressed his objections, but I had fun writing it. :-)