I've migrated the old Token and Tokenizer classes. From all the testing that occurred last semester, I believe they're minimally viable for supporting Scheme.
The two major changes I've made are to the unit tests and to the #!<directive> rule. Usually, Scheme accepts just #!no-fold-case and #!fold-casebut I've made it so that any other string, even the empty one, is not an error. This is so that we can use#!quit` to end the REPL. It's very useful.
Please check that the test cases look sane.Please check that the REPL code looks not too complex.Please check that the Doxygen-formatted comments for the Tokenizer class are well-written.
Note: The documentation for Tokenzier is unfinished because it's just too big. I'm putting of the work for it and adding a @todo marker to explicitly mark the documentation as unfinished.
I've migrated the old
TokenandTokenizerclasses. From all the testing that occurred last semester, I believe they're minimally viable for supporting Scheme.The two major changes I've made are to the unit tests and to the
#!<directive>rule. Usually, Scheme accepts just#!no-fold-caseand #!fold-casebut I've made it so that any other string, even the empty one, is not an error. This is so that we can use#!quit` to end the REPL. It's very useful.Please check that the test cases look sane. Please check that the REPL code looks not too complex. Please check that the Doxygen-formatted comments for the
Tokenizerclass are well-written.Note: The documentation for
Tokenzieris unfinished because it's just too big. I'm putting of the work for it and adding a@todomarker to explicitly mark the documentation as unfinished.