This PR removes a lot of the previous regex-based lexer implementation and implement a more proper lexer. In the process it resolves #15. The lexer is rather rudimentary because it only needs to do a rather rudimentary tokenization of C++. For the most part if there is an error it tries to continue as best it can.
This PR removes a lot of the previous regex-based lexer implementation and implement a more proper lexer. In the process it resolves #15. The lexer is rather rudimentary because it only needs to do a rather rudimentary tokenization of C++. For the most part if there is an error it tries to continue as best it can.