`bump_glue` `next_byte` disregard `num_tokens`

[sunxd3]

MWE:

julia> ps = ParseStream("a + b - c * d"); peek(ps);

julia> bump_glue(ps, K"Identifier", EMPTY_FLAGS, 4)
JuliaSyntax.ParseStreamPosition(0x00000002, 0x00000000)

julia> ps.tokens[end] |> dump
JuliaSyntax.SyntaxToken
  head: JuliaSyntax.SyntaxHead
    kind: JuliaSyntax.Kind K"Identifier"
    flags: UInt16 0x0000
  orig_kind: JuliaSyntax.Kind K"Identifier"
  preceding_whitespace: Bool false
  next_byte: UInt32 0x00000003

If I understand correctly, I would expect the next_byte to be 0x00000006 as,

julia> JuliaSyntax.peek_token(ps)
Identifier      |6

Maybe at https://github.com/JuliaLang/JuliaSyntax.jl/blob/fc572f95c250f802c6721f90061125b05422965e/src/parse_stream.jl#L738 should be

                                     stream.lookahead[i+num_tokens].next_byte))

and maybe bound checking?

This effects build_tree as https://github.com/JuliaLang/JuliaSyntax.jl/blob/fc572f95c250f802c6721f90061125b05422965e/src/parse_stream.jl#L1075

[c42f]

bump_glue is a bit of an unfortunate anomaly that I wish I could remove; it only needs to exist to resolve a lexing ambiguity with signed numbers:

• -2 means the literal integer -2
• -2*x means the infix call (call-i -2 x)
• However -2^x means (call-pre - (call-i 2 ^ x)) due to precedence (See #julialang/julia/18851)

Anyway, so it seems the easiest way forward here is to just remove the num_tokens parameter entirely, as we only ever use bump_glue with num_tokens == 2.

[sunxd3]

That sounds reasonable to me. PR #338