rlwhitcomb / utilities

Some of my personal utility programs
MIT License
2 stars 0 forks source link

Eliminate lexical ambiguity in Calc grammar with directives #603

Closed rlwhitcomb closed 1 year ago

rlwhitcomb commented 1 year ago

For instance:

var exit = true
{ a:exit }

will currently generate a syntax error, because :exit is identified as a directive by the lever, instead of the two tokens : and exit within the context of a map / object declaration.

In the Antlr 4 book he has examples of lexer modes, useful for XML parsing (for instance), but this requires unique tokens (in the case of XML, like < and >) to enter and leave the mode. We don't have that, since mostly we need it for "object" parsing, but the { signaling entry into an object is also used for entry into a statement block, where directives are (or should be) still available. So, the only way we would know that we're parsing an object is at the parser level, which happens in a whole separate pass than lexical analysis, so we're kinda screwed...

rlwhitcomb commented 1 year ago

We might have to recast directives using different syntax, that can't be confused with the member:value syntax inside objects... Originally (if I remember correctly) we used a different character to introduce them... but I can't remember which now.

Looking around inside "Calc.g4" it looks like $ might be a good candidate.

rlwhitcomb commented 1 year ago

Unless there are some other scripts I don't know about that need the directive changes, we're done with the.