kjosib / booze-tools

Booze Tools will become the complete programming-language development workbench, all written in Python 3.9 (for now).
MIT License
14 stars 1 forks source link

MacroParse fails to compile a scanner-only document #32

Open kjosib opened 4 years ago

kjosib commented 4 years ago

Near the end of macroparse.compiler.compile_string(...) is a call to ebnf.sugarless_form() which may return None if no production rules are defined (and thus there's no start symbol) but the caller assumes a ContextFreeGrammar comes back and tries to make a table of it.

An automatic test for this would consist of compiling a simple scanner-only document (in-line) and asserting the result has a scanner table. Probably there should be no parse table, but scanner-only applications do exist and they should be supported in this manner. Accordingly, the test might continue on to apply the generated scanner-table in a "typical application" scenario, scan a few tokens, assert they are as expected, and then pass.