search

no-context / moo

Optimised tokenizer/lexer generator! 🐄 Uses /y for performance. Moo.
BSD 3-Clause "New" or "Revised" License
821 stars 65 forks source link

Skip tokens #71

Closed ghost closed 7 years ago

ghost commented 7 years ago

During iteration, it would be great to be able to skip certain tokens, such as white space and comments.

In ANTLR, this is done using "hidden" channels. Maybe something like a "skip" attribute?

{match: /\s+/, lineBreaks: true; skip: true}
kach commented 7 years ago

~@tjvr this sounds like a moo question.~

EDIT: Err, it is. I thought this was the nearley repo — I guess I get email notifications for this one, too now?

tjvr commented 7 years ago

See #24; you can do this yourself.

ghost commented 7 years ago

@tjvr I don't like that solution. A skip property would be very useful.

tjvr commented 7 years ago

As I explained above, we've decided that such a feature would be out of scope for moo. If you have any particular arguments against this, perhaps #24 would be the place to add them.

ghost commented 7 years ago

@tjvr no problem, I forked moo and implemented it. Calling this feature out of scope is ridiculous, of course.