Allow optional tokens in rules

fubuloubu commented 4 years ago

I've spent a bit of time working with a project that uses SLY, and one thing I think could be a huge benefit would be to allow optionals (0 or 1 occurances) in rules.

For example, I've found myself having to define something like the following quite often:

@_('TOKEN')
@_('')
def maybe_token(self, p):
    pass

@_('... maybe_token ...')
def other_rule(self, p):
    ...

Where I don't really care about TOKEN but I do need to account for it because it is contextually useful (such as in a language like Python where whitespace is important... sometimes).

It would be much simpler and easier to read to be able to do the following:

@_('... [TOKEN] ...')
def other_rule(self, p):
    ...

Where [TOKEN] defines that it is optional to the production rule, expanding into a rule with and rule without.

dabeaz commented 4 years ago

This is already supported in the current version. See CHANGES. Also the "EBNF Features" section of the documentation. Note: you need to include spaces around the "[" and "]" in SLY.

fubuloubu commented 4 years ago

Thanks! I missed that!

dabeaz / sly

Allow optional tokens in rules #49