Attention
This implementation is a work in progress and only contains the Lexer and initial REPL
Any constructive criticism is greatly appreciated! Thank you for your time.
Approach
In general, I tried to use more of the standard library where it helps:
Peekable<> and Chars<> iterators remove the need for storing position information
Delegate to Token::from(...) instead of the Lexer type knowing what literals a token can/should be (I was trying to think of a better way to serve this word salad)
Benefits
Peekable<> cleans up the peek_char() implementation
Using char and Chars<> (Instead of u8 and Bytes<>) enables Unicode/UTF-8 usage if I can change identifiers to allow more than ([a-z][A-Z]_)*.
Any constructive criticism is greatly appreciated! Thank you for your time.
Approach
In general, I tried to use more of the standard library where it helps:
Peekable<>andChars<>iterators remove the need for storing position informationToken::from(...)instead of theLexertype knowing what literals a token can/should be (I was trying to think of a better way to serve this word salad)Benefits
Peekable<>cleans up thepeek_char()implementationcharandChars<>(Instead ofu8andBytes<>) enables Unicode/UTF-8 usage if I can change identifiers to allow more than([a-z][A-Z]_)*.