Treesitter not working

[kflak]

Operating System

Linux/BSD

Additional Operating System information

Arch Linux 6.9.8

Emacs Version

29

Emacs Configuration Details

• [X] Native Compilation
• [X] pGTK
• [ ] alternative package manager (use-package, straight.el, ...)

Anything else that may be related to the issue you're having?

Treesitter-git installed on my system

What happened?

First of all, thanks for a great starter kit! I am just getting into emacs after nvimming for a decade, and this kit made it very easy to make the switch...

Now: treesitter seems not to be happening. In C++ files there is no highlighting at all, and emacs doesn't respond to any of the treesit commands.

I have this in my init.el:

(setq custom-file (expand-file-name "custom.el" user-emacs-directory))
(when (and custom-file
           (file-exists-p custom-file))
  (load custom-file nil :nomessage))
(load "~/crafted-emacs/modules/crafted-init-config.el")
(defun crafted-emacs-load-modules (modules)
  "Initialize crafted-emacs modules.

MODULES is a list of module names without the -packages or
-config suffixes.  Note that any user-provided packages should be
added to `package-selected-packages' before invoking this
function."
  (dolist (m modules)
    (require (intern (format "crafted-%s-packages" m)) nil :noerror))
  (package-install-selected-packages :noconfirm)
  (dolist (m modules)
    (require (intern (format "crafted-%s-config" m)) nil :noerror)))

;; Any extra packages in addition to the ones added by crafted-emacs
;; modules go here before we call `crafted-emacs-load-modules'.
(customize-set-variable 'package-selected-packages '(ef-themes magit))
(add-to-list 'package-selected-packages 'ef-themes)
(crafted-emacs-load-modules '(defaults completion ui ide org writing lisp))

(crafted-ide-configure-tree-sitter '(css html bash cpp glsl json markdown org toml yaml))
(crafted-ide-eglot-auto-ensure-all)

and the output of c-h m when I'm in a C++ buffer is:

Minor modes enabled in this buffer: Auto-Save Breadcrumb-Local Corfu
Display-Line-Numbers Eglot--Managed Eglot-Inlay-Hints Eldoc Flymake
Font-Lock Undo-Fu-Session Yas

The major mode is C++// mode defined in c-ts-mode.el:

Major mode for editing C++, powered by tree-sitter.

This mode is independent from the classic cc-mode.el based
‘c++-mode’, so configuration variables of that mode, like
‘c-basic-offset’, don’t affect this mode.

What should have happened?

Treesitter should work..

[sthesing]

@kflak I run into the following error message when I reproduce this with your provided init.el (Arch with almost identical settings; I use tree-sitter from extra, you are using tree-sitter-git from AUR):

Warning (treesit): Cannot activate tree-sitter, because language grammar for c is unavailable (not-found): (libtree-sitter-c libtree-sitter-c.0 libtree-sitter-c.0.0 libtree-sitter-c.so libtree-sitter-c.so.0 libtree-sitter-c.so.0.0) No such file or directory

Do you see that, too? It makes sense since c++-ts-mode is derived from c-ts-mode, as your help buffer suggests.

For me, this was fixed by adding c to the list of configured languages:

(crafted-ide-configure-tree-sitter '(css html bash c cpp glsl json markdown org toml yaml))

Does that help for you, too?

[kflak]

Hmmm... strange, I don't get any warnings like yours. And the weird thing is that some treesitter functions do actually work, so it's not entirely dead. Only syntax highlighting seems to be broken.

[sthesing]

Did you try adding "c" to the list of languages for crafted-ide-configure-tree-sitter?

[kflak]

Yes, I did, but for some reason no C parser has been installed into my ~/.config/emacs/tree-sitter directory. I'm probably missing some basic bit on how to install them. C++ is there...

[sthesing]

Hm... is the init.el you posted your whole config? Emacs should ask you whether you want to "Install missing grammars? (y or n)". For some reason you seem to be neither seeing that question nor the error message. Can you spot something relevant in the Warnings oder Messages buffers?

Also, you can try to build the grammar for C manually by running treesit-install-language-grammar.

[kflak]

The plot thickens...

Yes, I've been running with the stripped-down init.el I posted above.

When I disable tree-sitter for c++ I get syntax highlighting. When I enable it and open f.x. ofApp.h I get the same highlighting, running in cc-mode.el, not in c++-ts-mode. Partial output from C-h m:

The major mode is C++//l mode defined in cc-mode.el:

The syntax highlighting looks like this:

When I switch to the implemtation file (ofApp.cpp) the mode switches over to c-ts-mode. Output of C-h m:

The major mode is C++// mode defined in c-ts-mode.el:

Major mode for editing C++, powered by tree-sitter.

In this mode the syntax highlighting looks like this:

I also tried building both c and c++ grammars manually. They are both in the ~/.config/emacs/tree-sitter dir:

  /home/kf/.config/emacs/tree-sitter:
  drwxr-xr-x  2 kf kf 4.0K Jul 17 17:28 .
  drwxr-xr-x 14 kf kf 4.0K Jul 17 17:29 ..
  -rwxr-xr-x  1 kf kf 3.1M Jul 17 16:52 libtree-sitter-cpp.so
  -rwxr-xr-x  1 kf kf 588K Jul 17 17:28 libtree-sitter-c.so
  -rwxr-xr-x  1 kf kf  93K Jul 10 18:50 libtree-sitter-css.so
  -rwxr-xr-x  1 kf kf  60K Jul 17 16:51 libtree-sitter-elisp.so
  -rwxr-xr-x  1 kf kf 816K Jul 10 18:51 libtree-sitter-glsl.so
  -rwxr-xr-x  1 kf kf  38K Jul 10 18:50 libtree-sitter-html.so
  -rwxr-xr-x  1 kf kf  16K Jul 10 18:51 libtree-sitter-json.so
  -rwxr-xr-x  1 kf kf 502K Jul 10 18:51 libtree-sitter-org.so
  -rwxr-xr-x  1 kf kf  88K Jul 17 16:52 libtree-sitter-scheme.so
  -rwxr-xr-x  1 kf kf  40K Jul 10 18:51 libtree-sitter-toml.so
  -rwxr-xr-x  1 kf kf 211K Jul 10 18:51 libtree-sitter-yaml.so

[jeffbowman]

There are differences between language-mode and language-ts-mode (in this case c/c++). The ts modes usually have fewer features. Navigation and parsing are obviously provided, but fontification is a bit hit-and-miss in my experience. I'll take a look at the mode and see if there is anything enlightening there.

[sthesing]

Thanks for looking into this, @jeffbowman! This issue might be related: tree-sitter/tree-sitter-cpp#274

But unless you unearth something, am I right in assuming that - as far as Crafted Emacs is concerned - the c++-ts-mode does work, it only isn't as feature rich as @kflak expected?

[jeffbowman]

Yes, that seems like the case (as it is with most ts modes).

Also, I don't see anything in the code and in fact it looks like fontification "should" be happening, at least as far as I can see.

However, the bug you refer to is worth watching as well as it might be the root cause and not a problem with Crafted Emacs.

[sthesing]

Thanks, @jeffbowman.

@kflak It seems there's not much more we can do for you. I'm closing this issue, if that's OK. Feel free to reopen if there is still something we can help you with.

[kflak]

Ok, thanks for looking into it! Am I right in assuming that tree-sitter on emacs is not quite ready for showtime yet?

[jeffbowman]

That's a harder question to answer. Personally, I feel I get more value from the normal modes than the ts modes. In theory, using ts modes would be "faster" because they should "fix" fontification - the canonical example of typing opening quotes to start a string and the rest of the file gets fontified as a string, but with a ts mode, that doesn't happen, so rendering the buffer is quicker. In theory, navigation is much better with ts modes as well. I think it boils down to personal experience and expectations. If you heavily use features of a particular mode (c/c++ in your case) then using tsmodes will be a step backwards. If you don't, and if you want improved navigation, then ts modes might be ok for you. I do think it is early in the ts lifespan, so there will be rough bits and bumps. But ready for showtime? I mean, the treesit executable works, Emacs integration with it works, so... yeah - and at the same time not so much given the above comments.

[kflak]

Sorry to keep nagging about this, but I still suspect this has something to do with the configuration... If I open a C++ file and engage c++-ts-mode the highlighting disappears. So far old news. However, if I run tree-sitter-hl-mode I get the message:

No language registered for major mode ‘c++-ts-mode’

Here comes the interesting twist, though: If I go back to "regular" c++-mode and then run tree-sitter-hl-mode I get the beautiful treesitter syntax highlighting! However, I lose the treesit-* functions. Any way to make sense of this? Can anybody confirm the same behavior on their systems?

[RogerWWilliams]

FWIW I'm seeing the same behavior as kflak on both a Debian Linux system running Testing and also a FreeBSD system running STABLE. I also found that M-x c-ts-mode will get the highlighting back. I don't see M-x c++-ts-mode as an option currently although the language grammar has been installed without issue.

[kflak]

Ah, interesting! If I activate c-ts-mode I don't get any highlighting... It seems like it's all very much connected to emacs' own treesitter-implementation, but I'm not quite sure where to file a bug report for that.

[jeffbowman]

The "right" place would be submitting a bug on Emacs itself. Things to do to be helpful in that case:

1. Create a minimal configuration that demonstrates the problem. You can do this quite easily by using the --init-directory command line option. For example, /usr/bin/emacs --init-directory treesit-c-bug. In the treesit-c-bug directory put a init.el with only the treesitter configuration for c/c++ files. Make sure this minimal configuration exhibits the problem you are seeing.
2. Run the M-x report-emacs-bug command and fill in the details. (Read https://www.gnu.org/software/emacs/manual/html_node/emacs/Bugs.html for more guidance)

You'll want to provide a clear description of the problem, the steps to reproduce the problem, the minimal configuration that exhibits the problem, etc.

Do make sure you have read through the manual chapter linked above (or by using C-h i in Emacs) .

Hope that helps.

Caveat emptor - if you cannot reproduce the problem in this way (ie outside of Crafted Emacs) - reopen this bug and I'll try to assist again.

[jeffbowman]

Here is a minimal configuration which exhibits the issue you mention:

(treesit-install-language-grammar 'cpp)
(treesit-install-language-grammar 'c)

(add-hook 'cpp-mode-hook (lambda () (c++-ts-mode) (electric-pair-mode)))
(add-to-list 'major-mode-remap-alist '(c-mode . c-ts-mode))
(add-to-list 'major-mode-remap-alist '(c++-mode . c++-ts-mode))
(add-to-list 'major-mode-remap-alist
             '(c-or-c++-mode . c-or-c++-ts-mode))

And here is a simple c++ file as well:

#include <iostream>

class Person {
private:
  std::string first_name;
  std::string last_name;

public:
  Person(std::string f, std::string l) {
    first_name = f;
    last_name = l;
  }

  std::string name() {
    return last_name + ", " + first_name;
  }
};

int main(int argc, char ** argv) {
  std::cout << "Hello, world" << std::endl;
  Person p = Person("James", "Gosling");
  std::cout << "And hello to " << p.name() << std::endl;
  return 0;
}

Hope that helps.

[kflak]

Awesome, thank you so much, @jeffbowman! I'll submit the bug report as soon as I have a bit more time on my hands.

[kflak]

News update: I came across a post on Mastodon from the developer of foot (the wayland terminal emulator) that he had the same behavior with c++-tsmode, but whatever it was went away as soon as he updated to emacs 30. So I guess I'll just calmly wait for that to be released :-)

[RogerWWilliams]

Nice! Looks like it may have already been addressed in that case by the Emacs team. Thank you @kflak for digging into this issue with the Emacs team and thank you @jeffbowman for taking the time to look at this and supplying the code snippets!