NixOS / nixpkgs

Nix Packages collection & NixOS
MIT License
17.81k stars 13.91k forks source link

emacsPackages.treesit-grammars.with-all-grammars doesn't seem to build/link in all grammars #348933

Open diegs opened 8 hours ago

diegs commented 8 hours ago

Describe the bug

This may be user error, but I am trying to understand why not all of the treesitter grammars in https://github.com/NixOS/nixpkgs/blob/nixos-unstable/pkgs/development/tools/parsing/tree-sitter/default.nix#L54 are being pulled into https://github.com/NixOS/nixpkgs/blob/nixos-unstable/pkgs/applications/editors/emacs/elisp-packages/manual-packages/treesit-grammars/package.nix#L14

In particular, libtree-sitter-markdown-inline, which is an extra "sub-grammar" (https://github.com/NixOS/nixpkgs/blob/nixos-unstable/pkgs/development/tools/parsing/tree-sitter/default.nix#L73) doesn't appear to get linked properly.

I don't know if this is a bug in the way the extra grammars are being merged into grammars' or another issue.

Steps To Reproduce

Steps to reproduce the behavior:

  1. Add something like the following to your emacs config:
    extraEmacsPackages = epkgs: [
          epkgs.treesit-grammars.with-all-grammars
        ];
  2. Build emacs
  3. Inspect the generated emacs-packages-deps/lib directory
❯ ls /nix/store/i5a8nl1wm90qpi3ddlp6pk3zqhi448h5-emacs-packages-deps/lib 
emacs                   libmu_mbox.la           libtree-sitter-beancount.so          libtree-sitter-glimmer.so         libtree-sitter-nickel.so           libtree-sitter-sql.so
libmailutils.la         libmu_mbox.so           libtree-sitter-bibtex.so             libtree-sitter-glsl.so            libtree-sitter-nix.so              libtree-sitter-supercollider.so
libmailutils.so         libmu_mbox.so.9         libtree-sitter-bitbake.so            libtree-sitter-go.so              libtree-sitter-norg-meta.so        libtree-sitter-surface.so
libmailutils.so.9       libmu_mbox.so.9.0.0     libtree-sitter-bqn.so                libtree-sitter-godot-resource.so  libtree-sitter-norg.so             libtree-sitter-svelte.so
libmailutils.so.9.0.0   libmu_mh.la             libtree-sitter-c-sharp.so            libtree-sitter-gomod.so           libtree-sitter-nu.so               libtree-sitter-talon.so
libmarkdown_inline.so   libmu_mh.so             libtree-sitter-c.so                  libtree-sitter-gowork.so          libtree-sitter-ocaml-interface.so  libtree-sitter-templ.so
libmu_auth.la           libmu_mh.so.9           libtree-sitter-clojure.so            libtree-sitter-graphql.so         libtree-sitter-ocaml.so            libtree-sitter-tiger.so
libmu_auth.so           libmu_mh.so.9.0.0       libtree-sitter-cmake.so              libtree-sitter-haskell.so         libtree-sitter-perl.so             libtree-sitter-tlaplus.so
libmu_auth.so.9         libmu_pop.la            libtree-sitter-comment.so            libtree-sitter-hcl.so             libtree-sitter-pgn.so              libtree-sitter-toml.so
libmu_auth.so.9.0.0     libmu_pop.so            libtree-sitter-commonlisp.so         libtree-sitter-heex.so            libtree-sitter-php.so              libtree-sitter-tsq.so
libmu_dbm.la            libmu_pop.so.9          libtree-sitter-cpp.so                libtree-sitter-hjson.so           libtree-sitter-pioasm.so           libtree-sitter-tsx.so
libmu_dbm.so            libmu_pop.so.9.0.0      libtree-sitter-css.so                libtree-sitter-html.so            libtree-sitter-prisma.so           libtree-sitter-turtle.so
libmu_dbm.so.9          libmu_py.la             libtree-sitter-cuda.so               libtree-sitter-http.so            libtree-sitter-proto.so            libtree-sitter-typescript.so
libmu_dbm.so.9.0.0      libmu_py.so             libtree-sitter-cue.so                libtree-sitter-janet-simple.so    libtree-sitter-pug.so              libtree-sitter-typst.so
libmu_dotmail.la        libmu_py.so.9           libtree-sitter-dart.so               libtree-sitter-java.so            libtree-sitter-python.so           libtree-sitter-uiua.so
libmu_dotmail.so        libmu_py.so.9.0.0       libtree-sitter-devicetree.so         libtree-sitter-javascript.so      libtree-sitter-ql-dbscheme.so      libtree-sitter-verilog.so
libmu_dotmail.so.9      libmu_scm.la            libtree-sitter-dockerfile.so         libtree-sitter-jsdoc.so           libtree-sitter-ql.so               libtree-sitter-vim.so
libmu_dotmail.so.9.0.0  libmu_scm.so            libtree-sitter-dot.so                libtree-sitter-json.so            libtree-sitter-query.so            libtree-sitter-vue.so
libmu_imap.la           libmu_scm.so.9          libtree-sitter-earthfile.so          libtree-sitter-json5.so           libtree-sitter-r.so                libtree-sitter-wgsl.so
libmu_imap.so           libmu_scm.so.9.0.0      libtree-sitter-eex.so                libtree-sitter-jsonnet.so         libtree-sitter-regex.so            libtree-sitter-wing.so
libmu_imap.so.9         libmu_sieve.la          libtree-sitter-elisp.so              libtree-sitter-julia.so           libtree-sitter-rego.so             libtree-sitter-yaml.so
libmu_imap.so.9.0.0     libmu_sieve.so          libtree-sitter-elixir.so             libtree-sitter-just.so            libtree-sitter-rst.so              libtree-sitter-yang.so
libmu_maildir.la        libmu_sieve.so.9        libtree-sitter-elm.so                libtree-sitter-koka.so            libtree-sitter-ruby.so             libtree-sitter-zig.so
libmu_maildir.so        libmu_sieve.so.9.0.0    libtree-sitter-embedded-template.so  libtree-sitter-kotlin.so          libtree-sitter-rust.so             mailutils
libmu_maildir.so.9      libmuaux.la             libtree-sitter-erlang.so             libtree-sitter-latex.so           libtree-sitter-scala.so            python3.12
libmu_maildir.so.9.0.0  libmuaux.so             libtree-sitter-fennel.so             libtree-sitter-ledger.so          libtree-sitter-scheme.so           systemd
libmu_mailer.la         libmuaux.so.9           libtree-sitter-fish.so               libtree-sitter-llvm.so            libtree-sitter-scss.so             
libmu_mailer.so         libmuaux.so.9.0.0       libtree-sitter-fortran.so            libtree-sitter-lua.so             libtree-sitter-smithy.so           
libmu_mailer.so.9       liborg.so               libtree-sitter-gdscript.so           libtree-sitter-make.so            libtree-sitter-solidity.so         
libmu_mailer.so.9.0.0   libtree-sitter-bash.so  libtree-sitter-gleam.so              libtree-sitter-markdown.so        libtree-sitter-sparql.so   

Expected behavior

All of the grammars in buildGrammars (https://github.com/NixOS/nixpkgs/blob/nixos-unstable/pkgs/development/tools/parsing/tree-sitter/default.nix#L54) should be linked into that directory.

However, some are missing, in particular libtree-sitter-markdown-inline.so does not appear.

johnhamelink commented 8 hours ago

I've also experienced this issue, particularly with markdown-inline. I was able to unstick myself by manually building and loading markdown-inline: https://git.sr.ht/~johnhamelink/nix/tree/master/item/home/modules/emacs/emacs.nix#L36 and in particular https://git.sr.ht/~johnhamelink/nix/tree/master/item/home/modules/emacs/src/treesitter-grammars/markdown-inline.nix#L17