Silver precendence problems

[tedinski]

Two immediate surprising bugs in Silver:

• !1 < 2 parses as !(1 < 2)
• -x % 2 parses as -(x % 2)

Precedence guidance

There are a few rules for choosing precedence:

• Behave in an expected manner, first and foremost.
  • People expect whitespace to denote looser precedence, so something typically written 1 + x.y should naturally parse as 1 + (x.y) and never, ever (1 + x).y.
  • Be consistent, both within the language, and across languages.
• Behave in a useful fashion, minimizing unnecessary parenthesis.

C has an incredible influence on expectations, and while some of C's choice are, in simple fact, wrong (bitwise operator precedence: just wrong wrong wrong!), most of C's choices are now just universally conventional.

One of C's rules for precedence is:

1. The tightest precedence is postfix operators. *a++ is *(a++).
2. Next tightest precedence is prefix operators. *x + 1 is (*x) + 1.

What does Silver get wrong?

• [ ] We give a bizarre precedence to ! which gives rise to the above unexpected parse.
• [ ] We casually slapped one precedence on - for both subtraction and negation, which is incorrect, these should have different precedence.

How should we fix this?

First, where?

1. In Silver.
2. We should add numeric negation to the Simple tutorial, and demonstrate how to fix it there, too. After all, this slipped by us for this long, obviously it should be pointed out and the best solution demonstrated. So first, what's the best solution?

Not thinking clearly at first, I thought this would be a problem. After all, how do we give two different precedences to -? But this is silly: we can apply operators to productions. What we're really suffering from here is over-abuse of the precedence/associativity capabilities.

We should adopt a new convention of limiting direct application of precedence/association to binary operators only, and instead develop a careful discipline for applying different rules for other operators (or the stray keyword, like else).

I think we can solve this problem by introducing two new pseudo-terminals:

terminal PostfixOperatorPrecedence // precedence = 24;
terminal PrefixOperatorPrecedence // precedence = 23;

• Feel free to bikeshed on different name choices

And the idea is to, on appropriate productions, apply an operator choice explicitly, rather than relying on the implicit operator choice (for everything except normal binary operators.)

So for example:

concrete production neg
top::Expr ::= '-' e::Expr
operator = PrefixOperatorPrecedence
{...}

This gives prefix negation a higher precedence, so -x%2 will now parse more tightly: (-x) % 2, as has become expected due to C's conventions.

Fun Fact

I just discovered that neg already looks like:

concrete production neg
top::Expr ::= '-' e::Expr
precedence = 13
{...}

It seems someone in history thought precedence on productions was the same mechanism as precedence on terminals, and had tried to fix this issue. That's not true! This is perhaps evidence we should rename this mechanism?

So additional TODO items:

• [ ] Remove these actually quite useless annotations.
• [ ] Consider renaming "reduce/reduce production precedences" to something different so they're not confused with "shift/reduce operator precedences." Name suggestions? "priority?" @schwerdf do you have thoughts on this? We could do this in just Silver, but perhaps Copper would want to use different terminology, too.

[schwerdf]

I think it would be more confusing to use two different terms for precedence.

[tedinski]

Ok, I'll defer to you on that one. I checked git blame for those erroneous lines, and they go all the way back to Derek's initial SVN commit transitioning from CVS. We could chalk the confusion up to documentation back then being poor to non-existent.

[schwerdf]

If the lines are from Derek's time, then they are likely from when Silver was still using Bison or Happy as the parsing backend, so it might just have been overlooked in the parser transition.