emacsPgtk fails to find tree-sitter grammars

[liff]

To reproduce run this flake and attempt to load c-ts-mode:

nix run --no-write-lock-file git+https://gist.github.com/f302bd2cc086f30092f803b2fed5aaf8.git -- \
    --eval "(require 'c-ts-mode)"

Results in this warning:

⛔ Warning (treesit): Cannot activate tree-sitter, because language grammar for cpp is unavailable (not-found): (libtree-sitter-cpp libtree-sitter-cpp.so) No such file or directory

This seems to be caused by the recent addition of wrapGAppsHook when withPgtk is enabled. This causes the RPATH patch to be applied to the wrapper instead of the actual Emacs binary.

[bhankas]

Can confirm. So since the issue is upstream (nixpkgs), the issue/fix should also ideally be upstream, no?

[liff]

I don’t think it’s necessarily an upstream issue. Using the wrapper is only a problem in this sort of case where the overlay wants to fix up the binary. The wrapper change originated from this overlay, by the way, see #309 :)

Suppose it would be convenient, though, if upstream directly supported building with the grammars somehow.

[realcomplex]

Any way of easily overriding the native build inputs for emacs in a home-manager setup? This issue is kind of a show-stopper for me so for now I just downgraded by pinning to an earlier commit of this overlay. I'm however not up to the task of patching of the overlay in the right way...

[liff]

A quick and very dirty workaround using overrideAttrs:

emacsPgtk.overrideAttrs (prev: {
  postFixup = builtins.replaceStrings [ "/bin/emacs" ] [ "/bin/.emacs-*-wrapped" ] prev.postFixup;
})

The idea is to change the patchelf command to modify the wrapped binary’s RPATH.

[DamienCassou]

I recommend against the emacs-overlay way of adding grammars and suggest instead to follow the path of nipkgs: https://github.com/nix-community/emacs-overlay/issues/323.

[idlip]

Not sure if I'm not understanding this, even after the pr https://github.com/NixOS/nixpkgs/pull/230751, emacsPgtk fails to find tree-sitter grammar.

Emacs path: /nix/store/svha8kgvinvp52sz7ss24jqdm3q13z75-emacs-pgtk-with-packages-20230518.0/bin/emacs Hydra build: https://hydra.nix-community.org/eval/304764

But it works both in emacsGtk and emacsUnstable.

I know emacs 29 stable release is near, and may launch in nixos unstable with time, Can we expect emacsPgtk as a package variant for emacs in nixpkgs?

I have a really slow hardware, so apart from building the package locally, how can i use emacsPgtk right now, with binary cache available?

[DamienCassou]

@idlip how did you install emacsPgtk with the plugins?

[idlip]

@DamienCassou, Does it work fine with emacsPgtk for you? If yes, i may try some changes.

Note: I am not compiling emacs, I got emacsPgtk from nix-community cachix I have this block for emacs;

# Symlinking emacs configs, so we can edit it in realtime and have immediate effect without requiring a rebuild.

home.file.".config/emacs/early-init.el".source = config.lib.file.mkOutOfStoreSymlink "<<my-config-path>>/emacs/early-init.el";
home.file.".config/emacs/init.el".source = config.lib.file.mkOutOfStoreSymlink "<<my-config-path>>/emacs/init.el";
home.file.".config/emacs/elfeed.org".source = config.lib.file.mkOutOfStoreSymlink "<<my-git-path>>/d-rss.org";

programs.emacs = {
  enable = true;
  package = pkgs.emacsPgtk;
  extraPackages = (epkgs: (with epkgs; [
    vterm multi-vterm vundo undo-fu-session flycheck helpful ox-pandoc
    no-littering rainbow-delimiters rainbow-mode vertico 
    orderless consult marginalia embark org olivetti org-modern corfu
    embark-consult consult-eglot consult-flycheck
    cape markdown-mode nix-mode
    nerd-icons async dirvish
    reddigg hnreader mingus which-key magit webpaste org-present
    pdf-tools nov shrface shr-tag-pre-highlight gcmh
    org-mime corfu-terminal beframe denote tempel tempel-collection
    sdcv elfeed elfeed-org link-hint powerthesaurus jinx meow
    doom-modeline hide-mode-line el-fetch ox-hugo htmlize
    ement kind-icon speed-type
  ])
  );
};

Apart from this, I have emacs config here repo

With emacs -Q i get same error

[DamienCassou]

In extraPackages, you need to add epkgs.treesit-grammars.with-all-grammars. You may also want to get only a subset of the grammars with epkgs.treesit-grammars.with-grammars (grammars: with grammars; [tree-sitter-bash]).

As @wentasah mentioned in https://github.com/NixOS/nixpkgs/pull/230751#issuecomment-1550009697, you also need an override.

All in all, something like that has greater chances of success (untested):

{
  programs.emacs = {
    enable = true;
    package = pkgs.emacsPgtk.overrideAttrs (old: {        # ⇐ added
      passthru = old.passthru // {
        treeSitter = true;
      };
    });
    extraPackages = (epkgs: (with epkgs; [
      treesit-grammars.with-all-grammars                  # ⇐ added
      vterm multi-vterm vundo undo-fu-session flycheck helpful ox-pandoc
      no-littering rainbow-delimiters rainbow-mode vertico 
      orderless consult marginalia embark org olivetti org-modern corfu
      embark-consult consult-eglot consult-flycheck
      cape markdown-mode nix-mode
      nerd-icons async dirvish
      reddigg hnreader mingus which-key magit webpaste org-present
      pdf-tools nov shrface shr-tag-pre-highlight gcmh
      org-mime corfu-terminal beframe denote tempel tempel-collection
      sdcv elfeed elfeed-org link-hint powerthesaurus jinx meow
      doom-modeline hide-mode-line el-fetch ox-hugo htmlize
      ement kind-icon speed-type
  ])
  );
};
}

[idlip]

Splendid, i was scared if the override had to compile locally and i cannot use cachix, but it was not the case.

But it was weird, it did not work with emacs -Q, but works when i create empty ~/.emacs.d directory and also works as expected with my emacs config too. Probably it needs to load something?

Thank you @DamienCassou for your support to both Nix and Emacs world! (such a rabbit hole ;0 )

@liff You can try latest package with above mentioned changes, it works for me!

I had this only issue in emacs, which was bothering for some time, and boom, its gone now. Emacs never disappoints lol

[idlip]

@DamienCassou Then, passthru attribute should be set for treeSitter=true; by default in emacs-overlay only?

wentasah says it needs to be set for emacsGit and emacsUnstable, but treesitter works there without setting passthru when i tested, the issue was only with emacsPgtk.

You can test on your end if you will, but as far as i can tell, can you please make a pr to set passthru by default here in emacs-overlay for emacsPgtk ?

Thank you again!

[wentasah]

The passthru thing seems to be tricky. As far as I understand, emacs-overlay reuses emacs Nix expression from nixpkgs and applies a few overrides on it. The passtrhu.treeSitter is set by the nixpkgs expression if it is called with version >= 29. However, emacs-overlay overrides the nixpkgs Emacs expression after it has been called with nixpkgs Emacs version 28 and therefore, it has passthru.treeSitter false.

Treesitter in emacs-overlay works, because it does not rely on passthru.treeSitter, but it is somehow hardcoded in the overlay. However, @DamienCassou's treesit-grammars relies on nixpkgs way of treesitter integration, which needs passthru.treeSitter.

When I tried something like in https://github.com/nix-community/emacs-overlay/issues/312#issuecomment-1554066326, the passthrou override was not applied (but I've not tested that particular example). I don't really know why. You can see whether it was applied by checking the value of treesit-extra-load-path emacs variable. If passthru.treeSitter is true, the variable contains a path like "/nix/store/6bq92lvsvi5s3l5hn2k3y36is70wmd9j-emacs-packages-deps/lib/", otherwise, it's nil.

[liff]

@idlip, @DamienCassou treesit-grammars.with-all-grammars and passthru works nicely, thanks!