Figure out how to break up features into assignments

[rawrgulmuffins]

Expected Features

1. Tokens
2. Tokens __repr__ and __str__
3. EOF token lexes
4. - token lexes
5. + token lexes
6. Single digit integer token lexes
7. consume validates expected token
8. consume errors on unexpected token
9. consume produces EOF if no more tokens
10. 1+1 lexes and parsers to 2
11. Blank Program Parsers
12. multi-digit integers are lexed correctly
13. whitespace is ignored until new token found.
14. * lexes and parsers
15. / lexes and parsers
16. Arbitrary number of operations lex and parse correctly.

    Extra Credit

17. () lexes and parsers
18. parens around signle statement lexes and parsers
19. parens can arbitrarily nest
20. Parens change the order of operations.

    Initial proposal

    Assignment 1

21. Tokens
22. Tokens __repr__ and __str__

    Justification

Very simple starting assignment. This is here mainly as a means of testing the level of the class. If they struggle with basic methods and python idioms then we'll need to slow the class down. If they blaze through it all the better.

Assignment 2

1. EOF token lexes (in next_token)
2. - token lexes (in next_token)
3. + token lexes (in next_token)
4. Single digit integer token lexes

   Justification

So intially I wanted assignment 2 to be a fully working "Calculator" that did addition and subtraction. When I listed out all of the steps I noticed how many it was to go from tokens to a working REPL.

As such I started looking for a good break point and completing the _next_token method felt like a good one to me.

Assignment 3

1. consume validates expected token
2. consume errors on unexpected token
3. consume produces EOF if no more tokens
4. 1+1 lexes and parsers to 2; 1-1 parsers to 0
5. Blank Program Parsers

   Justification

consume_token is a small but interesting method. It's too small to be it's own assignment and parse pairs with it fairly well (since parse starts out by just calling consume three times with an expected pattern). So this felt like a good break point.

It also brings students to the point where they can run calc.py with sample input that generates real results.

Assignment 4

1. multi-digit integers are lexed correctly

   Justification

This requires a while True loop that keeps a first position and a current position. In the past I've found this hard for students (especially new students) to wrap their head around this idea. As such I wanted to break this into it's own assignment with it's own set of tests.

Additionally I had a decent number of tests so it just felt like a natural break point.

Assignment 5

1. whitespace is ignored until new token found.

   Justification

Same as 4.

Assignment 6

1. * lexes and parsers
2. / lexes and parsers

   Justification

Last break point before assignment 7 which is hard. This is also kind of a review for how to find and generate tokens.

Assignment 7

1. Arbitrary number of operations lex and parse correctly.

   Justification

Stretch goal. This is the last feature you can add without having to include some more complicated data structure like a stack.

This requires you to modify parse to use a While loop that identifies tokens and expects different Tokens based on the previously found ones.

[reillysiemens]

This is going to require a lot more review. Perhaps we should polish the earlier assignments before focusing on any of the later ones.