This PR adds a small Python script that, when run, spits out an EBNF grammar for Pop-11's itemiser, which in other programming languages would be called the tokeniser or lexer. The reason for using a script is to cope with the rules around non-decimal numbers. Poplog supports numbers written in base 2 through to base 36, which does create a challenge if you want the grammar to be reasonably accurate.
I have made two simplifications in the grammar. Firstly I do not try to capture the rules around special VED character or attributes. Secondly, I have not tried to capture the restrictions on numerical escape sequences properly - mainly because the documentation is not entirely accurate. I have included with this PR the reference document itemise in the background/ folder.
I have made a couple of adjustments to the Makefile, so that it targets the _build/ folder rather than dumps build products in the current directory.
This PR adds a small Python script that, when run, spits out an EBNF grammar for Pop-11's itemiser, which in other programming languages would be called the tokeniser or lexer. The reason for using a script is to cope with the rules around non-decimal numbers. Poplog supports numbers written in base 2 through to base 36, which does create a challenge if you want the grammar to be reasonably accurate.
I have made two simplifications in the grammar. Firstly I do not try to capture the rules around special VED character or attributes. Secondly, I have not tried to capture the restrictions on numerical escape sequences properly - mainly because the documentation is not entirely accurate. I have included with this PR the reference document
itemisein thebackground/folder.I have made a couple of adjustments to the Makefile, so that it targets the
_build/folder rather than dumps build products in the current directory.