Location Tracking and Error Reporting

[kjosib]

A good strategy for reporting scan- and parse-errors needs to be part of the base model of a successful language development kit.

[kjosib]

The main hang-up with the present design is that the parser is "running the show" but it doesn't know/care where tokens come from, yet the scanner state is invisible to (e.g.) callers of runtime.the_simple_case(...) and so a parse exception cannot be located to the offending text. That deliberate-ignorance represents a separation-of-concerns, which I would like to preserve. That leaves roughly two and one half options:

1. Adjust so that parse(...) turns into an object that takes a method call for each token, and then another special call for the end-of-text. This has a clear translation path to a decent variety of other languages.
2. Adjust so that parse(...) is done as a co-routine: This is trendy and uses the latest Python gizmos, and there may some small performance advantage. Technically this is a very-special case of option 1 with the correspondence left as an exercise for the reader.
3. Leave parse(...) in charge of running the show, but make it demand a true "scanner interface" so that when an error is detected, the parse algorithm can include location information.

I'm leaning towards option 1:

• the_simple_case(...) can centralize exception handling, including for driver fails.
• During semantic analysis, you'll still want to be able to illustrate failure points in input text; this should therefore be factored away from the scanner and parser into something that knows how to format error messages independent of how they are noticed.
• Eventually, an interface for error reporting can be defined. This facilitates bringing these tools into something like an IDE with zero further impact on the way the generic scan/parse algorithms function.

[kjosib]

The new failureprone.py module attacks three sub-problems:

• Formatting error reports for the console half-decently, built atop bits for...
• Translating plain-integer file positions to row/column addresses, and
• Locating the bounds of a given line of text.

[kjosib]

the_simple_case(...) is now factored into three parts so that a workable solution for error reporting and lexical tie-ins became clear without changing any major architecture: the scanner is visible as an attribute of the generated parser function.

Remaining to add is a good example of how to use it. Probably that should go in the calculator example.

[kjosib]

Just pushed a working demo calculator with error reporting example as described above.