inexorabletash
commented
7 years ago Should probably just match UCBLogo:
A word not after a quotation mark or inside square brackets is delimited by a space, a bracket, a parenthesis, or an infix operator +-*/=<>. Note that words following colons are in this category. Note that quote and colon are not delimiters.
I wrote the core tokenization code before I settled on trying to match UCBLogo as a baseline, so my choices were arbitrary.
In UCBLogo reference, it says that variable name must be a "word". It is not clear whether it means a Logo word or a more generic idea about word (a-z, A-Z, 0-9, _ ).
It appears we can make a variable name as below: make "1 2 make "$1 345 make "_1a "Hello
It is pretty much every character can be made variable names, except a few get into trouble in UCBLogo:
make "( 24 make "* 100
In JSLogo, I can: make "_a 456 print names to see a variable was created, but the parsing and codemirror mode will both reject a variable name starts with underscore or numbers.
I think the variable name should be clearly defined as to what characters and rules are legal to use.