tree-sitter / tree-sitter-python

Python grammar for tree-sitter
MIT License
360 stars 132 forks source link

bug: Special case not managed by the parser, when an expression is split without a backslash and the second line is dedented. #256

Open martinett opened 6 months ago

martinett commented 6 months ago

Did you check existing issues?

Tree-Sitter CLI Version, if relevant (output of tree-sitter --version)

No response

Describe the bug

If an expression is split into 2 lines without a backslash and not after a comma, and the second line is de-indented making it go outside of the instruction block (outside of an if, for example), the parser thinks there is an error, while python correctly interprets it. Example of minimal code :

if True:
  print(1+
1)

But it also works with a multiplication star or any operator before the split. It also works with a subscript statement, or any parenthesized which can be split as we want, according to Python. And it also works inside a loop or any statement with an indented block.

It may works with other syntactic exceptions, but I can't find a more general pattern.

Steps To Reproduce/Bad Parse Tree

This produce the following error :

(module
  (if_statement condition: (true)
    (ERROR (integer))
    consequence: (block))
  (ERROR (integer)))

Expected Behavior/Parse Tree

(module
  (if_statement condition: (true)
    consequence: (block
      (expression_statement
        (call function: (identifier)
        arguments:
          (argument_list
            (binary_operator left: (integer) right: (integer))))))))

Repro

if True:
  print(1+
1)