This pull request implements two fixes to the parser:
parsing of x!=1 (used to be parsed as x!(1) instead of x != 1)
handling of line breaks after a hash-key symbol, for example
foo x:
"some text"
should be parsed as foo(x: "some text") but it was parsed as foo(x:); "some text" because the parser incorrectly interpreted the line break after x: as a statement terminator.
The first problem is addressed by letting the scanner handle identifiers that end with !, so it can do a negative lookahead test for a = character.
The second problem is addressed by adding a fake "no_line_break" token in the grammar right after the : symbol in a pair and change the scanner so it won't produce "line_break" symbols if a "no_line_break" symbol is expected. This solution feels a bit "hacky" to me, but it could not think of a better way. @maxbrunsfeld any ideas?
Checklist:
[x] All tests pass in CI.
[x] There are sufficient tests for the new fix/feature.
[x] Grammar rules have not been renamed unless absolutely necessary.
[x] The conflicts section hasn't grown too much.
[x] The parser size hasn't grown too much (check the value of STATE_COUNT in src/parser.c).
This pull request implements two fixes to the parser:
x!=1(used to be parsed asx!(1)instead ofx != 1)should be parsed as
foo(x: "some text")but it was parsed asfoo(x:); "some text"because the parser incorrectly interpreted the line break afterx:as a statement terminator.The first problem is addressed by letting the scanner handle identifiers that end with
!, so it can do a negative lookahead test for a=character.The second problem is addressed by adding a fake "no_line_break" token in the grammar right after the
:symbol in a pair and change the scanner so it won't produce "line_break" symbols if a "no_line_break" symbol is expected. This solution feels a bit "hacky" to me, but it could not think of a better way. @maxbrunsfeld any ideas?Checklist: