[Question] am I expected to provide the values of treesit-language-source-alist?

[acdoussan]

I am currently trying to get tree sitter working with typescript, but I have been having some problems.

My emacs is built with a bunch of tree sitter languages available, however, the default treesit-language-source-alist variable does not include most of them, including typescript / TSX. This leads to them not showing as an install option when running treesit-install-language-grammar and I don't get prompted to install them when opening a file. Am I expected to provide mappings for these, or should these be part of the variable by default?

$ emacs --version
GNU Emacs 29.0.60
Copyright (C) 2023 Free Software Foundation, Inc.
GNU Emacs comes with ABSOLUTELY NO WARRANTY.
You may redistribute copies of GNU Emacs
under the terms of the GNU General Public License.
For more information about these matters, see the file named COPYING.

$ emacs -nw -q --batch --eval '(message system-configuration-options)'
--prefix=/nix/store/674fmg6rmzqcays9x0bk8gm5x0135im1-emacs-head --disable-build-details --with-modules --with-x-toolkit=gtk3 --with-xft --with-cairo --with-xwidgets --with-json --with-tree-sitter --with-dbus --with-gif --with-jpeg --with-png --with-rsvg --with-tiff --with-xpm -with-gpm=no --with-modules --with-harfbuzz

Mode when I open a ts file (want this to be typescript-ts-mode)

C-h v major-mode
major-mode is a buffer-local variable defined in buffer.c.

Value in #<buffer selectors.test.ts>
typescript-mode

Original Value
fundamental-mode

nix script used for building emacs: https://github.com/acdoussan/env-setup/blob/emacs-29/files/nix/emacs-override.nix

langs provided by the tree sitter dependency: https://github.com/NixOS/nixpkgs/tree/nixos-22.11/pkgs/development/tools/parsing/tree-sitter/grammars https://github.com/NixOS/nixpkgs/blob/nixos-22.11/pkgs/development/tools/parsing/tree-sitter/default.nix#L69

C-h v treesit-language-source-alist

treesit-language-source-alist is a variable defined in treesit.el.gz.

Value
((yaml "https://github.com/ikatyang/tree-sitter-yaml" nil nil nil nil)
 (toml "https://github.com/tree-sitter/tree-sitter-toml" nil nil nil nil)
 (rust "https://github.com/tree-sitter/tree-sitter-rust" nil nil nil nil)
 (ruby "https://github.com/tree-sitter/tree-sitter-ruby" nil nil nil nil)
 (python "https://github.com/tree-sitter/tree-sitter-python" nil nil nil nil)
 (protobuf "https://github.com/mitchellh/tree-sitter-proto" nil nil nil nil)
 (markdown "https://github.com/ikatyang/tree-sitter-markdown" nil nil nil nil)
 (make "https://github.com/alemuller/tree-sitter-make" nil nil nil nil)
 (lua "https://github.com/Azganoth/tree-sitter-lua" nil nil nil nil)
 (latex "https://github.com/latex-lsp/tree-sitter-latex" nil nil nil nil)
 (julia "https://github.com/tree-sitter/tree-sitter-julia" nil nil nil nil)
 (json "https://github.com/tree-sitter/tree-sitter-json" nil nil nil nil)
 (javascript "https://github.com/tree-sitter/tree-sitter-javascript" "master" "src" nil nil)
 (java "https://github.com/tree-sitter/tree-sitter-java" nil nil nil nil)
 (html "https://github.com/tree-sitter/tree-sitter-html" nil nil nil nil)
 (go "https://github.com/tree-sitter/tree-sitter-go" nil nil nil nil)
 (elixir "https://github.com/elixir-lang/tree-sitter-elixir" nil nil nil nil)
 (dockerfile "https://github.com/camdencheek/tree-sitter-dockerfile" nil nil nil nil)
 (css "https://github.com/tree-sitter/tree-sitter-css" nil nil nil nil)
 (cpp "https://github.com/tree-sitter/tree-sitter-cpp" nil nil nil nil)
 (commonlisp "https://github.com/theHamsta/tree-sitter-commonlisp" nil nil nil nil)
 (cmake "https://github.com/uyha/tree-sitter-cmake" nil nil nil nil)
 (c-sharp "https://github.com/tree-sitter/tree-sitter-c-sharp" nil nil nil nil)
 (c "https://github.com/tree-sitter/tree-sitter-c" nil nil nil nil)
 (bibtex "https://github.com/latex-lsp/tree-sitter-bibtex" nil nil nil nil)
 (bash "https://github.com/tree-sitter/tree-sitter-bash" nil nil nil nil))

[renzmann]

To answer the most immediate question: no, you are not expected to provide values to treesit-language-source-alist. The real bug here is because in the transition to 0.5 I forgot to add the :remap keywords for those two languages in their recipes https://github.com/renzmann/treesit-auto/blob/70bc85b7b907e2b5d4d692d263c3153068a1065e/treesit-auto.el#L245

This is a simple fix, so give it roughly a day for a patch to go through along with the MELPA build. Thanks for pointing out the bug!

[acdoussan]

Gotcha, no problem, thanks!

Would be cool to see some of the other languages like nix / svelte / vue added as well while you're at it, maybe there is a way for you to download / parse those json files and add them to the list? Kinda surprised there isn't just a source of truth for these from tree-sitter, but I guess that is part of the point.

[renzmann]

We do have a slight issue though - typescript-mode is used for both TypeScript and TSX. Remapping major modes needs to be a true function (in the mathematical sense). That is, I can't have typescript-mode remapping to both tsx-ts-mode and typescript-ts-mode, so for now, I've just given :remap typescript-mode to typescript-ts-mode.

What you can do for tsx-ts-mode, though, is make sure the .tsx extension is on auto-mode-alist

(add-to-list 'auto-mode-alist '("\\.tsx\\'" . tsx-ts-mode))

Then you should get installation prompts and automatic switching to tsx-ts-mode when opening a .tsx file. If it seems appropriate, we could add an :mode or similar keyword like how use-package does it, but I worry about conflicts with user config or other packages by doing that.

[renzmann]

I am not a JavaScript dev, so I would have to rely on a PR from someone more knowledgeable about what the best source repo and recipe configuration for those frameworks would look like!

[acdoussan]

So, think something else is not quite right here, looks like typescript-ts-mode and tsx-ts-mode might be tied together? I get this error after choosing yes when prompted to install typescript-ts-mode when opening a .ts file.

⛔ Warning (treesit): Cannot activate tree-sitter, because language grammar for tsx is unavailable (not-found): (libtree-sitter-tsx libtree-sitter-tsx.so) No such file or directory

The file then fails to open. Open the file again and typescript-ts-mode is not enabled.

C-h v major-mode

major-mode is a buffer-local variable defined in buffer.c.

Value in #<buffer index.ts>
typescript-mode

Original Value
fundamental-mode

[acdoussan]

I get the same error when trying to open a .tsx file after also setting (add-to-list 'auto-mode-alist '("\\.tsx\\'" . tsx-ts-mode))

⛔ Warning (treesit): Cannot activate tree-sitter, because language grammar for tsx is unavailable (not-found): (libtree-sitter-tsx libtree-sitter-tsx.so) No such file or directory

the .tsx file then opens in fundamental mode, rather than tsx-ts-mode or something else

[renzmann]

Oh, interesting. This seems related to some of the work on #31, which would add support for these modes/grammars that are tied together. If we can break that PR down to just the new :requires keyword, then we can use this issue as one of the ways to test the resolution

[renzmann]

This should be fixed in #33 - let me know if you're still having any trouble!

[acdoussan]

Alright, got there but a bit weirdly. So I now got prompted to install typescript, then got the same error presented, then got prompted to install tsx, then still ended up in typescript-mode instead of typescript-ts-mode. Closed and reopened the file, still typescript-mode.

Closed and reopened emacs, then reopened the file, and I've got typescript-ts-mode!

Is it normal to have to restart emacs for installing to take effect?

[acdoussan]

Seems like after installing any language I have to restart emacs to get the file to open in tree sitter mode. I just had the same thing happen for a yml file where I was prompted to install it, but the file wouldn't use the tree sitter major mode until i restarted emacs.

[renzmann]

Do you have a minimal configuration that could reproduce this behavior? I have yet to run into this, but do want to squash the bug if it's there. Here's an example configuration that works as intended:

;; ~/tmp/init.el
(use-package typescript-mode
  :ensure t)

(use-package treesit-auto
  :ensure t
  :custom
  (treesit-auto-install t)
  :config
  (global-treesit-auto-mode))

Then launching Emacs as emacs --init-directory=~/tmp, I visit any .ts file, it installs the grammars, and C-h m displays

The major mode is TypeScript mode defined in typescript-ts-mode.el:

If there is a variation on this that causes the behavior you're seeing, then I'm happy to take a look into it

[renzmann]

Since it's been a few months, I'll close this as stale for now. Feel free to reopen if the problem persists or if anyone else is facing similar issues.