Investigate How Changing Space Operator Precedence For Maths

[jdunkerley]

Want to evaluate the complexity of making it so whitespace not affect the precedence of mathematical operators (arithmetic, bitwise and logical ops).

We would keep the space precedence for the other operators like . and ->.

a + b>4 . floor == ((a+b)>4).floor NOT (a+(b>4)).floor a-> b-> a+b

Goal is to understand how hard to change on the engine/parser side and how hard to update all the library code needed.

[wdanilo]

My idea would be to introduce another precedence level in parser. I don't think this involves any part of engine beside parser (please correct me if I'm wrong). Now, we hierarchical precedence discovery - first level is "spaced" and "non-spaced". Then in these groups we apply precedences of operators. We could add another group to the top level, so we will have "spaced", "non-spaced", "math" and then just resolve precedences in these groups. Just my idea, might not be the best one, but I shared it because maybe it will be useful to someone :)

[kazcw]

Expression	Interpretation (current)	Interpretation (Solution 1)	Interpretation (Solution 2)
`a * b+c . floor`	`a * ((b + c) . floor)`	`((a * b) + c) . floor`[^1]
`a * b+c.floor`	`a * (b + (c . floor))`	`((a * b) + (c . floor))`[^1]
`x+y * z`	`(x + y) * z`	`x + (y * z)`[^1]
`f x+y * z`	`(f (x + y)) * z`	`f (x + (y * z))`[^1]	*unchanged*
`f x + y`	`(f x) + y`	`f (x + y)`[^2]	*unchanged*
`f +y`	`f (+y)`	*unchanged*
`f x +y`	`((f x) (+y))`	*unchanged*

[^1]: In this case a warning may be emitted that spacing is inconsistent with effective precedence. [^2]: The interpretation of this expression has changed, but no warning is emitted. This may take some getting used to for library developers.

[kazcw]

Solution 0: Math-consistent spacing

Maintain the uniform application of space-precedence rules, but warn if the result is inconsistent with mathematical precedence.

Solution 1: High precedence math

Place math operators in a precedence level between unspaced non-math and spaced non-math.

Current logic:

• Reduce each sequence of terms that contains no spaces.
• Reduce the remainder.

New logic:

• Reduce each sequence of terms that contains no spaces or symmetrically-unspaced binary mathematical operators (but if a binary math operator has an unspaced term on exactly one side, reduce it to an operator section).
• Reduce each sequence consisting of X (F X)+ (regex notation), where X is a non-operator term and F is a binary mathematical operator.
• Reduce the remainder.

Solution 2: Space-invariant math precedence

Redefine the effect of spacing in terms of precedence-modification.

Current logic (after refactoring):

• When an operator is unspaced, its precedence is modified to be higher than any spaced operator.

New logic:

• When an operator is unspaced, its precedence is modified to be higher than any spaced operator, but when comparing precedence of mathematical operators unmodified precedence is used.

Discussion

Complexity for language users:

• Solution 0 allows the simplest language rules, but users unfamiliar with space-precedence may be surprised by a warning if they use spacing inconsistent with mathematical precedence.
• Solution 1 can be explained in terms of a precedence hierarchy, but it would no longer be possible for any operators (or function application) to have a higher precedence than mathematical operators.
• Solution 2 achieves space-invariant precedence within mathematical subexpressions without losing the high precedence of function application, though it requires a "modified precedence" concept that may be less intuitive than the precedence hierarchy concept of Solution 1.

Implementation

Parsing changes

• Solution 1: 3 points
• Solution 2: 5 points

Warnings

• Parser: Each solution would require different logic for detecting unexpected spacing. The parser has no mechanism for emitting warnings; this would need this to be added to the JNI and JS bindings.
  • Solution 0/1: 2 points
  • Solution 2: 1 point
• Compiler (? points): The compiler would receive a span->warnings map from the parser. I don't know whether it can make use of this data structure directly, or each warning needs to be attached to the IR with the corresponding span--the latter may take some work. The IDE doesn't need to receive these warnings from the compiler, since the IDE parses too; but the compiler should print them when run standalone, and maybe have a mode to treat parser warnings as errors when running tests etc.

Library changes

parse_all_enso_files.sh can be used to detect expressions whose interpretation have been changed. If there are few they could be updated by hand; if there are many, I could add refactoring suggestions to the warnings, and create a tool that can apply them (2 points).

Total (excluding compiler changes to print warnings)

• Solution 0: 2 points
• Solution 1: 7 points, assuming automated refactoring is needed
• Solution 2: 6-8 points, depending on whether automated refactoring turns out to be needed