Create Lexer safe point, so illegal tokens, that only contain legal characters throw the error at the safe point.
The safe point gets reset once a legal token is created.
If EOF is reached while the safepoint is not reset, an error is thrown with the safe point position.
Create Lexer safe point, so illegal tokens, that only contain legal characters throw the error at the safe point. The safe point gets reset once a legal token is created. If EOF is reached while the safepoint is not reset, an error is thrown with the safe point position.