The relative precedence of `::` and `where` is strangely context dependent

[LilithHafner]

From conversation with @c42f, the relative precedence of :: and where is strangely context dependent:

julia> using JuliaSyntax

julia> parsestmt(SyntaxNode, "function f()::T where T 0 end")
line:col│ tree                                   │ file_name
   1:1  │[function]
   1:10 │  [where]
   1:10 │    [::-i]
   1:10 │      [call]
   1:10 │        f
   1:15 │      T
   1:23 │    T
   1:24 │  [block]
   1:25 │    0

julia> parsestmt(SyntaxNode, "f()::T where T = 0")
line:col│ tree                                   │ file_name
   1:1  │[=]
   1:1  │  [::-i]
   1:1  │    [call]
   1:1  │      f
   1:6  │    [where]
   1:6  │      T
   1:14 │      T
   1:18 │  0

This also reproduces on pre-JS versions, but is less unambiguously visible

julia> :(function f()::T where T 0 end)
:(function (f()::T) where T
      #= REPL[26]:1 =#
      #= REPL[26]:1 =#
      0
  end)

julia> :(f()::T where T = 0)
:(f()::(T where T) = begin
          #= REPL[27]:1 =#
          0
      end)

[c42f]

Yes, there's a special case of where - vs :: in both parsers to do this. Presumably because the precedence for the function version really needs to be different from normal UnionAll expressions? @JeffBezanson would have to comment on the history of these constructs, though.

The fact long form vs short form function definitions are different is quite troubling. We could possibly change the precedence for the short form as well (by detecting the trailing = and hacking something in). It'd be a mess, but probably doable in JuliaSyntax ... Really such parsing is outside the capabilities of a strict recursive descent parser.

[ararslan]

Relevant: https://github.com/JuliaLang/julia/issues/21847

[c42f]

Thanks. I knew I'd seen Jeff comment about this somewhere (See https://github.com/JuliaLang/julia/issues/21847#issuecomment-301260055)

That old issue was closed because changing parsing is technically breaking. But if we had the mechanism which is being discussed in https://github.com/JuliaLang/julia/issues/54903 we might be able to fix some things which are "technically breaking" but not actually breaking in practice. And even things which are actually breaking (in a minor way), but we'd like to revisit for clarity.

I'm willing to bet that - if it was practical to change short-form parsing of function definitions - this would not actually break anything. Because people always want the function-like precedence of where in those cases. And in the cases they want to do it, they've already been forced to use parentheses.

[JeffBezanson]

Yes, as it is this was intentional; there is no single ideal option. So I don't consider this a bug but we can consider it a point of language evolution.

[c42f]

https://github.com/JuliaLang/JuliaSyntax.jl/pull/466 takes a related step toward doing this by changing the parsing of short form functions in the JuliaSyntax AST (this has no affect on Expr, it's just a parsing clarification upstream).