search

grantjenks / py-tree-sitter-languages

Binary Python wheels for all tree sitter languages.
Other
170 stars 49 forks source link

Add more parsers #53

Open Freed-Wu opened 9 months ago

Freed-Wu commented 9 months ago

Use parsers.json and lockfile.json to get information required by Language.build_library() Add .github/workflows/update.yml to update parsers.json and lockfile.json

Continue #33 @sansmoraxz lockfile.json lock the commit

Freed-Wu commented 9 months ago

Also a lot of them are missing src/parser.c. (Written in JS)

Those will fail to build.

In github action, tree-sitter and npm will be installed to fix this problem.

Freed-Wu commented 9 months ago

https://github.com/tree-sitter/py-tree-sitter/issues/189 affect many parsers.

Freed-Wu commented 9 months ago

Now it can work

heijligers commented 9 months ago

Big BUMP to have this PR integrated. Aider with Swift support would rock.

heijligers commented 9 months ago

Any updated what would be required to merge/approve this PR? An eager Aider user. Thanks!

cmosguy commented 8 months ago

@Freed-Wu Something is wrong now, when I checkout your parsers branch and try to do an install I am getting an error during the python build.py process:

Initialized empty Git repository in /workdir/py-tree-sitter-languages/vendor/wing/.git/
remote: Enumerating objects: 3043, done.
remote: Counting objects: 100% (3043/3043), done.
remote: Compressing objects: 100% (2235/2235), done.
remote: Total 3043 (delta 924), reused 1813 (delta 671), pack-reused 0
Receiving objects: 100% (3043/3043), 23.79 MiB | 7.44 MiB/s, done.
Resolving deltas: 100% (924/924), done.
From https://github.com/winglang/wing
 * branch            992e76b445311e13ff18470542f5ca972fb28567 -> FETCH_HEAD
Note: switching to '992e76b445311e13ff18470542f5ca972fb28567'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:

  git switch -c <new-branch-name>

Or undo this operation with:

  git switch -

Turn off this advice by setting config variable advice.detachedHead to false

HEAD is now at 992e76b feat(sdk): implement `entries()` method inside Map (#5475)
Error processing rule _escape_sequence

Caused by:
    regex parse error:
        u{[0-9a-fA-F]+}
          ^
    error: repetition quantifier expects a valid decimal
Traceback (most recent call last):
  File "/workdir/py-tree-sitter-languages/build.py", line 56, in <module>
    subprocess.check_call(["tree-sitter", "generate"], cwd=vendor)
  File "/usr/lib/python3.10/subprocess.py", line 369, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['tree-sitter', 'generate']' returned non-zero exit status 1.

Is this because something is wrong with the downstream of the wing language parser?

Freed-Wu commented 8 months ago

Can you disable wing then try again?

sansmoraxz commented 8 months ago

Whatever git commit you locking down the build step to should be tested across multiple archs and os.

Maybe try to build through gh actions in a separate branch.

ret2libc commented 8 months ago

I think I saw a similar error with newer tree sitter on another project

Freed-Wu commented 8 months ago

a similar error

I have updated and now I haven't found any error.

Any updated what would be required to merge/approve this PR?

Wait @grantjenks on #54

cmosguy commented 7 months ago

@Freed-Wu can you clarify something for me?

Can you adjust this project to run the python build.py and compile everything in as a github action? Only you can enable because you own the repo.

That way we can just include this repo in the requirements.txt file and the languages.so library will be included.

sansmoraxz commented 7 months ago

@cmosguy I believe that is out of the scope of this project.

From README:

py-tree-sitter-languages provides binary Python wheels for all tree sitter languages. The binary wheels remove the need to download and compile support for individual languages.

This project aims to provide precompiled tree sitter packages so you don't need to depend on a C compiler.

cmosguy commented 7 months ago

Thanks @sansmoraxz - I think I am confused then. The pipeline for this project compiles and generates .whl files that are used during compilation. If you look at this workflow.yaml and the setup.py it builds multiple .whl files that should be used for installation.

My point is that because @grantjenks has absconded from this project, we need someone to basically take over the reigns now and create a new project repo and PiPy project so that others can use these python bindings. A github action workflow will build the .whl files and deploy them to PiPy, but we need to transfer ownership to @Freed-Wu probably since I think he can manage this better.

What do you say @Freed-Wu ? Maybe create a whole new repo since @grantjenks is not interested in this anymore?

FeepingCreature commented 1 month ago

Okay, on the basis of "if nobody does anything then nothing will happen" (and also cause I really wanted aider D support), I've created a new Python package called 'tree-sitter-languages-freed-wu-pr' https://pypi.org/project/tree-sitter-languages-freed-wu-pr/ https://github.com/FeepingCreature/py-tree-sitter-languages

It changes the package name to tree_sitter_languages_freed_wu_pr to avoid collisions with this repo.

I don't know what @grantjenks or @Freed-Wu are up to, or if they're going to come back to this, but that should at least let other projects move forward for now.

I'm not committing to maintaining it. I'm especially not committing to maintaining it after the forty attempts to get the Github release action to work. But hopefully it should work at least as well as the last official release.

Changes from this PR:

But otherwise, ie. on Linux and Mac, all the languages from this PR should be in there and working.

If you want, you can look at the failed build actions here.