Package request: py-tree-sitter-languages

[samuela]

Project description

py-tree-sitter-languages provides binary Python wheels for all tree sitter languages. The binary wheels remove the need to download and compile support for individual languages.

Metadata

• homepage URL: https://github.com/grantjenks/py-tree-sitter-languages
• source URL: https://github.com/grantjenks/py-tree-sitter-languages
• license: asl2
• platforms: unix, linux, darwin, etc

nixpkgs has already packaged tree-sitter and py-tree-sitter, so this ought to be relatively straightforward. The only trickiness might be in the way that language parsers are fetched/built. Perhaps the already-packaged tree-sitter-grammars could be helpful here?

cc @marsam @Profpatsch

---

Add a :+1: reaction to issues you find important.

[teto]

what problem does it solve ? treesitter grammars are already available in nixpkgs and trivial to build. The nixpkgs+python ecosystem is painful to maintain so I would tend to use less python. Except if the idea is to provide a python package that should work outside of nix maybe ?

[samuela]

@teto py-tree-sitter-languages simplifies the process of using grammars -- downloading, building, linking, and so forth. I would be keen to take a nix-first approach to this if possible, but I was not able to find such an example. Is there a convenient way to use tree-sitter-grammars grammars with py-tree-sitter or the tree-sitter CLI?

There are also a handful of packages that rely on py-tree-sitter-languages as a dependency including https://aider.chat/, so this would open up the possibility of packaging those as well.

[doronbehar]

Is there a convenient way to use tree-sitter-grammars grammars with py-tree-sitter or the tree-sitter CLI?

What do you mean exactly by the tree-sitter CLI?

There are also a handful of packages that rely on py-tree-sitter-languages as a dependency including aider.chat, so this would open up the possibility of packaging those as well.

Finally I hear a good reason to package this python package :) See effort to fix this issue at https://github.com/NixOS/nixpkgs/pull/320783 .

[samuela]

What do you mean exactly by the tree-sitter CLI?

The tree-sitter package comes with a CLI binary:

❯ ns -p tree-sitter

[nix-shell:~/dev/SWE-agent]$ tree-sitter --help
tree-sitter 0.20.8
Max Brunsfeld <maxbrunsfeld@gmail.com>
Generates and tests parsers

USAGE:
    tree-sitter <SUBCOMMAND>

FLAGS:
    -h, --help       Prints help information
    -V, --version    Prints version information

SUBCOMMANDS:
    init-config       Generate a default config file
    generate          Generate a parser
    parse             Parse files
    query             Search files using a syntax tree query
    tags              Generate a list of tags
    test              Run a parser's tests
    highlight         Highlight a file
    build-wasm        Compile a parser to WASM
    playground        Start local playground for a parser in the browser
    dump-languages    Print info about all known language parsers

[nix-shell:~/dev/SWE-agent]$ 

Finally I hear a good reason to package this python package :)

😎

[doronbehar]

The tree-sitter package comes with a CLI binary:

Yes but since this one is packaged in Nixpkgs, and python3.pkgs.tree-sitter is also packaged, what exactly do you want to do with the tree-sitter-grammars which are also packaged in their own way? I mean, what benefit do you get with py-tree-sitter-languages that you can't get by tree-sitter-grammars?

[samuela]

what exactly do you want to do with the tree-sitter-grammars which are also packaged in their own way? I mean, what benefit do you get with py-tree-sitter-languages that you can't get by tree-sitter-grammars?

My personal reasons:

1. py-tree-sitter-languages is a dependency for other interesting packages like aider.
2. I have no idea how to use tree-sitter-grammars and I have not been able to find documentation explaining how to use them thus far.

A possible third reason that I had not previously considered is that py-tree-sitter-languages allows loading languages dynamically at runtime. tree-sitter-grammars requires specifying languages statically at build time.

[taha-yassine]

py-tree-sitter-languages is already packaged. See here: https://github.com/NixOS/nixpkgs/blob/nixos-unstable/pkgs/development/python-modules/tree-sitter-languages/default.nix#L63

[doronbehar]

Haha it is me who packaged this! I think though that the hard part is the python package named tree-sitter-grammers, that is attempted to be packaged here:

https://github.com/NixOS/nixpkgs/pull/320783

Perhaps @samuela would like to change the title and description of the issue.