icerpc / slicec

The Slice compiler library
Apache License 2.0
13 stars 5 forks source link

Make our Lexers Infallible #649

Open InsertCreativityHere opened 1 year ago

InsertCreativityHere commented 1 year ago

LALRPOP's error recovery logic only allows it to recover from errors originating in LALRPOP. So, if the lexer returns an error (say because a user typed '%', an unknown symbol in Slice) we would be incapable of recovering from it.

Instead, we should add an extra 'error' token to the lexers, and return Ok(error_token) to signal a lexer error, rather than Err(...). This will allow LALRPOP to gracefully recover from such lexer failures.