Completes the Tokenizer in the cpp directory, the parser and the evaluation.
Also adds support for hexadecimal integer literals, bitwise operators & (and), | (or) ^ (xor) and ~ (negation) for integers, as well aas && and || for booleans (short circuit logic).
Also includes several other changes to the original code:
Keyword token types clashed with C++ keywords, so capitalized all token types for uniformity
Avoiding copy into Lexer is a good idea, but having it rely on a string_view with the contents possibly being deallocated elsewhere is potentially hazardous. Instead, move the string into the lexer.
Conversely, tokens where overengineered using variants for literals: Made using them more dfficult, and special case for char optimizes the one case where the literal is not even neeed (and not present in eg. Rust enum). Replaced variant for literals with string_view (but remember, now the literal does not live beyond the lexer's lifetime)
Added constructor to token for easier coding.
replaced position based indexing into string with an iterator. Much more C++ style.
Lexer no longer throws exception on empty input, just returns an EOF token.
Reduced C++ standard needed to c++20 as the only c++23 feature used was not worth it (see block inside "#if __cplusplus < 202302L" in test.cc
Extensive suite of builtins
repl or script execution, command line arguments.
Now also fully implements "Writing a Compiler in Go" with all abive mentioned extensions.
Completes the Tokenizer in the cpp directory, the parser and the evaluation. Also adds support for hexadecimal integer literals, bitwise operators & (and), | (or) ^ (xor) and ~ (negation) for integers, as well aas && and || for booleans (short circuit logic).
Also includes several other changes to the original code:
Now also fully implements "Writing a Compiler in Go" with all abive mentioned extensions.