osa1 / lexgen

A fully-featured lexer generator, implemented as a proc macro
MIT License
63 stars 7 forks source link

Consider generating a warning (with an option to suppress) for rules that accept empty string #65

Open osa1 opened 1 year ago

osa1 commented 1 year ago

Recently I debugged a lexer with this rule:

("0b" | "0o" | "0x")? ($digit | '_')* $id? = ...,

This regex accepts empty string, so the lexgen state never failed and instead looped when it's supposed to fail.

Now, I've never needed this in a lexer, but this kind of rule can be used to switch to another state on failure, so I guess it's not completely useless. However I suspect for the majority of the use cases it actually means a bug in the lexer, so perhaps it makes sense to generate a warning for rules that accept empty strings, with a pragma to explicitly suppress the warning.