Aareon
commented
5 years ago A number of TODOs can be found in the lexer.py file. These will need to be completed before the lexer can be made functional.
At this stage, there is a 99% chance that the lexer wont work even with those TODOs completed. A lot of work will still be needed to make it functional, but it gives me a good basis on which to work.
After its functional I can work on creating tests. When all tests are green, everything lexer related will be pushed to master (under normal circumstances, it would be pushed to the "lexer-impl" branch to be tested against master, but theres nothing to test against.). From there, ill work on optimizations, code reusability, Pythonic improvements, and documentation. We'll make more tests for the new stuff, and so long as all tests stay green, we'll push the "final" product to master so that we can then move on to other portions of the compiler.
Task
Implement the Nim lexer in Python
Notes
The Nim lexer (and most of the rest of the Nim compiler) is almost a 1 to 1 translation to Nim from Pascal. This means that the official lexer isn't an example of good Nim code style. While it's still extremely efficient, this means that the side effects aren't always easy to determine.
Also, because we're implementing the Nim language in Python, I'd prefer to design the library in a Pythonic way. Nim allows one to define functions in a way that lets a user pass multiple combinations of valid arguments. In Python, the closest thing to this is keyword arguments, which can lead to function signatures looking really messy. In Python, there should only be one right way to do something, and that will be the ideal that we will strive for in this library.