bug: invalid query fontifying region in emacs with LSP server

[cowboyd]

Did you check existing issues?

• [ ] I have read all the tree-sitter docs if it relates to using the parser
• [X] I have searched the existing issues of tree-sitter-typescript

Tree-Sitter CLI Version, if relevant (output of tree-sitter --version)

No response

Describe the bug

After upgrading from version 0.20.3, my typescript buffers in emacs started continually raising the following error:

error: (treesit-query-error "Node type error at" 2  "(function name: (identifier) @font-lock-function-name-face) (function_declaration name: (identifier) @font-lock-function-name-face) (function_signature name: (identifier) @font-lock-function-name-face) (method_definition name: (property_identifier) @font-lock-function-name-face) (method_signature name: (property_identifier) @font-lock-function-name-face) (required_parameter (identifier) @font-lock-variable-name-face) (optional_parameter (identifier) @font-lock-variable-name-face) (variable_declarator name: (identifier) @font-lock-function-name-face value: [(function) (arrow_function)]) (variable_declarator name: (identifier) @font-lock-variable-name-face) (enum_declaration (identifier) @font-lock-type-face) (extends_clause value: (identifier) @font-lock-type-face) (extends_clause value: (member_expression object: (identifier) @font-lock-type-face property: (property_identifier) @font-lock-type-face)) (arrow_function parameter: (identifier) @font-lock-variable-name-face) (variable_declarator name: (array_pattern (identifier) (identifier) @font-lock-function-name-face) value: (array (number) (function))) (catch_clause parameter: (identifier) @font-lock-variable-name-face) (import_clause (identifier) @font-lock-variable-name-face) (import_clause (named_imports (import_specifier alias: (identifier) @font-lock-variable-name-face))) (import_clause (named_imports (import_specifier !alias name: (identifier) @font-lock-variable-name-face))) (import_clause (namespace_import (identifier) @font-lock-variable-name-face))" "Debug the query with `treesit-query-validate'")
  treesit-query-capture(#<treesit-node program in 1-23> #<treesit-compiled-query> 1 23)
  treesit-font-lock-fontify-region(1 23 nil)
  font-lock-fontify-syntactically-region(1 23 nil)
  font-lock-default-fontify-region(1 23 nil)
  font-lock-fontify-region(1 23)

Steps To Reproduce/Bad Parse Tree

1. Using vanilla emacs@29 with no customizations
2. install tree-sitter-typescript using the tag v0.20.4 (v0.20.5 and master will also reproduce the bug)
3. create hello.ts file with the following content:

export function hello() {
  console.log("Hello World");
}

4. eval (font-lock-fontify-region 1 23) which is called by the language server mode.

Expected Behavior/Parse Tree

for v0.20.3 and before, this succeeds. Past that point it fails. If this is a bug in eglot or emacs-lsp mode, then happy to report those there.

Repro

Apologies for requiring an external tool/IDE. I've spent about 90 minutes trying to track this down to be as reproducible as possible so that I could put it on somebody's radar, but without really understanding it, I'm not sure how to reproduce it without Emacs. It might be problem with emacs, it might be a problem with eglot (the emacs LSP client), or it might be a problem with tree sitter-typescript. The only thing I know that is that it has worked flawlessly up until this point, and that the tree-sitter-typescript version is the only variable I can manipulate to cause it to break.

My current workaround is to pin tree-sitter-typescript to v0.20.3

[ajyoon]

I also ran into this today, and can confirm pinning to v0.20.3 fixed it

[amaanq]

I don't think this belongs here, there's no parse errors and we're not going to debug an issue inside emacs. File this with the appropriate emacs repository responsible for tree-sitter integration.

[ajyoon]

@cowboyd and anyone else finding this later - this is confirmed to be an Emacs bug, and it was fixed in Emacs 29.2, which is already out. I just updated to 29.2 and confirmed the fix worked :)

https://lists.gnu.org/archive/html/bug-gnu-emacs/2023-10/msg01335.html

[cowboyd]

@ajyoon I am still experiencing this in emacs 29.2

Apart from re-compiling tree-sitter-typescript and upgrading to emacs 29.2, was there anything else you had to do?

[ajyoon]

I also removed the tree-sitter-typescript version pin from my .emacs, maybe double check that? But otherwise no, that's all I did and it works for me now.