Passing options to rules

[mitranim]

Currently, literals accept options such as scope names, but rules don't. Seems like an omission. It would be very useful to define a rule and instantiate it with different scopes in different places:

ident = '\b[[:alpha:]_][[:alnum:]_]*\b';

keyword_define =
  '\bdefine\b'{storage.type.function}
  ident{storage.type}
  ident{entity.name.function};

Currently, this can be done only by parametrizing the rule:

ident[scope]{#[scope]} = '\b[[:alpha:]_][[:alnum:]_]*\b';

keyword_define =
  '\bdefine\b'{storage.type.function}
  ident['storage.type']
  ident['entity.name.function}'];

We might be able to agree that the first example involves less typing, looks cleaner, and is more flexible than the second. I also find the first one more intuitive. Would be nice if this was supported!

[BenjaminSchaaf]

The first example is indeed more intuitive, but what happens in the following case:

ident{keyword} = '\b[[:alpha:]_][[:alnum:]_]*\b';

keyword_define =
  '\bdefine\b'{storage.type.function}
  ident{storage.type}
  ident{entity.name.function};

Does it append, prepend, override or error? I don't think any one of those answers is intuitive.

[mitranim]

Was hoping I wouldn't have to answer this question, but you got me there.

I would expect it to simply override the scope specified before =. For example:

rule0{scope0} = 'pattern'{scope1};
rule1 = rule0{scope2};

Would generate scopes similar to:

rule0{scope2} = 'pattern'{scope1};

The writer has the freedom to specify "default" non-overridable scopes to the right of =, and there are already rules for how that interacts with scopes to the left of =. (I'm actually unsure how.)

[BenjaminSchaaf]

My primary usage for something like that would actually be to append the scope, so that's what I'd expect it to do. I don't think the small gain in brevity this feature offers is worth the ambiguity, whereas parameterization will always be explicit.

[mitranim]

I'm in favor of making the SBNF syntax simple and consistent. Whichever way we declare scopes, it should be the same for literals and rules. The current approach involves different delimiters, and the scope is sometimes in quotes and sometimes not, needlessly implying different semantics for the same thing. Any newcomer, like me, would be right to raise an eyebrow. 🤨

I would argue that the benefit is more than a few keystrokes. It's simpler semantics, simpler syntax, and a lower learning curve. As I mentioned in the topic of interpolation, having rule interpolation and rule options would allow most syntaxes to be written just using these basics. At least personally to me, this seems preferable and worth pursuing.

[BenjaminSchaaf]

The scope is always the first argument to the options, which are always inside {}. Parameterization doesn't change that, nor is it required to achieve the same effect. You can already write most syntaxes using just the basics, parameterization just allows for simplification. A language is simplest when there is no ambiguity in how things work, no implicit assumptions you need to make. How a rule scope overrides/appends/prepends is always going to be ambiguous, something that will always need to be looked up and remembered. Even if you're explicit about it using something like rule{override: scope} that's still a much worse learning curve than "you can't specify options for rules".

[mitranim]

Duplicating this from #12 to continue the topic.

Realized that in my head, passing options to a rule is a shortcut to the following:

outer = 'start' inner{scope_median} 'end';
inner{scope_inner} = 'pattern';

outer = 'start' median 'end';
median{scope_median} = inner;
inner{scope_inner} = 'pattern';

The second one is already possible, and combines the scopes from both. I don't know if this equivalence would be intuitive for you or anyone else. But it seems like an easy logical step to me. This does make the order of scopes non-obvious, but that problem already exists for the pattern in the second example which already works.

[mitranim]

I should also add from #12 that if we end up with the ability to declare global variables and pass options to them, then passing options to rules becomes less essential; see https://github.com/BenjaminSchaaf/sbnf/issues/12#issuecomment-719427736. But that very selfsame example also passes options to a rule. So, see above. 🙂

[BenjaminSchaaf]

Thinking back over this I think the only intuitive behavior is to have the following be essentially equivalent:

rule1 = 'foo' ;
rule2 = rule1{scope} ;

rule1 = 'foo' ;
tmp{scope} = rule1 ;
rule2 = tmp ;

With this behavior the scope is prepended. The tmp rule is optimized out.

[BenjaminSchaaf]

So from #42 there's also a desire to append the scope in this case, so I'm going to address that here. I don't see how that's tenable with rules that are more complicated, for example:

a : b{append} ;
b{b} : 'b'{b.inner} c ;
c{c} : 'c'{c.inner} ;

other : b ;

Would need to transform into:

a : 'b'{b b.inner append} 'c'{c c.inner append} ;
b{b} : 'b'{b.inner} c ;
c{c} : 'c'{c.inner} ;

other : b ;

Since entirely new terminals would need to be constructed and expanded in order to properly append. I think the only reasonable thing to do in the case that you need append is to use a function:

b[APPEND]{b} : 'b'{b.inner #[APPEND]} ;

[mitranim]

One idea that comes to mind is being able to refer to "super" options / scopes. In the following example, rule_two reuses rule_one, both appending and prepending to its scopes. If rule_one had multiple variants / clauses, "super" would refer to the scopes of the specific resolved variant (forgot the proper lexicon). The actual syntax of <super> does not currently come to mind. We can't just reserve a keyword here (unless it's upper-case like SUPER I suppose...).

rule_one{scope_one} : 'match_one'{scope_two} ;
rule_two : rule_one{scope_three <super> scope_five} ;

[mitranim]

If when passing options to a rule, super-options were opt-in, the default behavior would be to completely replace the options, which arguably makes the results more "unsurprising" / predictable, compared to the default prepend discussed earlier. This also makes me wonder if the same could / should be done to the workaround that already exists:

rule_one = 'match_one'{scope_one} ;
rule_two{scope_two} = rule_one ;

According to an earlier comment, currently this generates an equivalent of:

rule_two = 'match_one'{scope_two scope_one} ;

But perhaps it should override the options, and combining them would be opt-in:

rule_one = 'match_one'{scope_one} ;
rule_two{SUPER scope_two} = rule_one ;