tree-sitter / tree-sitter-go

Go grammar for tree-sitter
MIT License
310 stars 65 forks source link

non-declaration statements are permitted at top level #63

Closed adonovan closed 2 years ago

adonovan commented 2 years ago

A Go source file always starts with a package declaration, but the Tree Sitter grammar for Go permits statements at top level:

x = 1

[source_file](https://tree-sitter.github.io/tree-sitter/playground#) [0, 0] - [1, 0]
  [assignment_statement](https://tree-sitter.github.io/tree-sitter/playground#) [0, 0] - [0, 5]
    left: [expression_list](https://tree-sitter.github.io/tree-sitter/playground#) [0, 0] - [0, 1]
      [identifier](https://tree-sitter.github.io/tree-sitter/playground#) [0, 0] - [0, 1]
    right: [expression_list](https://tree-sitter.github.io/tree-sitter/playground#) [0, 4] - [0, 5]
      [int_literal](https://tree-sitter.github.io/tree-sitter/playground#) [0, 4] - [0, 5]

x := <-ch

[source_file](https://tree-sitter.github.io/tree-sitter/playground#) [0, 0] - [1, 0]
  [short_var_declaration](https://tree-sitter.github.io/tree-sitter/playground#) [0, 0] - [0, 9]
    left: [expression_list](https://tree-sitter.github.io/tree-sitter/playground#) [0, 0] - [0, 1]
      [identifier](https://tree-sitter.github.io/tree-sitter/playground#) [0, 0] - [0, 1]
    right: [expression_list](https://tree-sitter.github.io/tree-sitter/playground#) [0, 5] - [0, 9]
      [unary_expression](https://tree-sitter.github.io/tree-sitter/playground#) [0, 5] - [0, 9]
        operand: [identifier](https://tree-sitter.github.io/tree-sitter/playground#) [0, 7] - [0, 9]

The cause is this production:

https://github.com/tree-sitter/tree-sitter-go/blob/07d722831382a043b16547b6d9202f3da07f3cb3/grammar.js#L105-L108

The fix is to remove the reference to _statement here and to add _declaration (var/const/type) to the choices for _top_level_declaration:

https://github.com/tree-sitter/tree-sitter-go/blob/07d722831382a043b16547b6d9202f3da07f3cb3/grammar.js#L110-L115

I wonder whether this was done intentionally because some tools want to process a chunk of statements instead of a complete file.

maxbrunsfeld commented 2 years ago

This is intentional, so that Tree-sitter can parse things like code snippets inside of markdown files.