search

alex-pinkus / tree-sitter-swift

A tree-sitter grammar for the Swift programming language.
MIT License
143 stars 39 forks source link

Failing to install in python #432

Open yaroslavyaroslav opened 2 months ago

yaroslavyaroslav commented 2 months ago

In both pip install git+https://github.com/alex-pinkus/tree-sitter-swift.git and poetry add git+https://github.com/alex-pinkus/tree-sitter-swift.git installation fails on a building stage because of missed src/parser.c artefact.

➜  swift4lmm python3 -m venv ~/.venv/swift4llm                                                                                                                                           
➜  swift4lmm venv swift4llm                                                                                                                                                              
(swift4llm) ➜  swift4lmm pip install -r requirements.txt                                                                                                                                 
Collecting openai (from -r requirements.txt (line 1))
  Downloading openai-1.43.0-py3-none-any.whl.metadata (22 kB)
Collecting tree-sitter (from -r requirements.txt (line 2))
  Using cached tree_sitter-0.23.0-cp312-cp312-macosx_11_0_arm64.whl.metadata (10 kB)
Collecting mlx-lm (from -r requirements.txt (line 3))
  Downloading mlx_lm-0.18.1-py3-none-any.whl.metadata (6.9 kB)
ERROR: Ignored the following versions that require a different python version: 0.21.0 Requires-Python <3.12,>=3.8
ERROR: Could not find a version that satisfies the requirement tree-sitter-swift (from versions: none)
ERROR: No matching distribution found for tree-sitter-swift
(swift4llm) ➜  swift4lmm python3 --version                                                                                                                                               
Python 3.12.5
(swift4llm) ➜  swift4lmm python3 --version                                                                                                                                               
Python 3.12.5
(swift4llm) ➜  swift4lmm pip list                                                                                                                                                        
Package Version
------- -------
pip     24.2
(swift4llm) ➜  swift4lmm pip install -r requirements.txt                                                                                                                                 
ERROR: Invalid requirement: 'git+git@github.com:alex-pinkus/tree-sitter-swift.git': Expected end or semicolon (after name and no valid version specifier)
    git+git@github.com:alex-pinkus/tree-sitter-swift.git
       ^ (from line 4 of requirements.txt)
Hint: It looks like a path. File 'git+git@github.com:alex-pinkus/tree-sitter-swift.git' does not exist.
(swift4llm) ➜  swift4lmm pip install -r requirements.txt                                                                                                            ➜  swift4lmm python3 -m venv ~/.venv/swift4llm                                                                                                                                           
➜  swift4lmm venv swift4llm                                                                                                                                                              
(swift4llm) ➜  swift4lmm pip install -r requirements.txt                                                                                                                                 
Collecting openai (from -r requirements.txt (line 1))
  Downloading openai-1.43.0-py3-none-any.whl.metadata (22 kB)
Collecting tree-sitter (from -r requirements.txt (line 2))
  Using cached tree_sitter-0.23.0-cp312-cp312-macosx_11_0_arm64.whl.metadata (10 kB)
Collecting mlx-lm (from -r requirements.txt (line 3))
  Downloading mlx_lm-0.18.1-py3-none-any.whl.metadata (6.9 kB)
ERROR: Ignored the following versions that require a different python version: 0.21.0 Requires-Python <3.12,>=3.8
ERROR: Could not find a version that satisfies the requirement tree-sitter-swift (from versions: none)
ERROR: No matching distribution found for tree-sitter-swift
(swift4llm) ➜  swift4lmm python3 --version                                                                                                                                               
Python 3.12.5
(swift4llm) ➜  swift4lmm python3 --version                                                                                                                                               
Python 3.12.5
(swift4llm) ➜  swift4lmm pip list                                                                                                                                                        
Package Version
------- -------
pip     24.2
(swift4llm) ➜  swift4lmm pip install -r requirements.txt                                                                                                                                 
ERROR: Invalid requirement: 'git+git@github.com:alex-pinkus/tree-sitter-swift.git': Expected end or semicolon (after name and no valid version specifier)
    git+git@github.com:alex-pinkus/tree-sitter-swift.git
       ^ (from line 4 of requirements.txt)
Hint: It looks like a path. File 'git+git@github.com:alex-pinkus/tree-sitter-swift.git' does not exist.
(swift4llm) ➜  swift4lmm pip install -r requirements.txt                                                                                                                                 
Collecting git+https://github.com/alex-pinkus/tree-sitter-swift.git (from -r requirements.txt (line 4))
  Cloning https://github.com/alex-pinkus/tree-sitter-swift.git to /private/var/folders/gc/v8tx0lzx4qg7tt1rl88wzgwr0000gn/T/pip-req-build-qv1hf486
  Running command git clone --filter=blob:none --quiet https://github.com/alex-pinkus/tree-sitter-swift.git /private/var/folders/gc/v8tx0lzx4qg7tt1rl88wzgwr0000gn/T/pip-req-build-qv1hf486
  Resolved https://github.com/alex-pinkus/tree-sitter-swift.git to commit d627a54d7266f0a8ee61b447b34d0ad07dadf1d6
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Collecting openai (from -r requirements.txt (line 1))
  Using cached openai-1.43.0-py3-none-any.whl.metadata (22 kB)
Collecting tree-sitter (from -r requirements.txt (line 2))
  Using cached tree_sitter-0.23.0-cp312-cp312-macosx_11_0_arm64.whl.metadata (10 kB)
Collecting mlx-lm (from -r requirements.txt (line 3))
  Using cached mlx_lm-0.18.1-py3-none-any.whl.metadata (6.9 kB)
Collecting anyio<5,>=3.5.0 (from openai->-r requirements.txt (line 1))
  Using cached anyio-4.4.0-py3-none-any.whl.metadata (4.6 kB)
Collecting distro<2,>=1.7.0 (from openai->-r requirements.txt (line 1))
  Using cached distro-1.9.0-py3-none-any.whl.metadata (6.8 kB)
Collecting httpx<1,>=0.23.0 (from openai->-r requirements.txt (line 1))
  Using cached httpx-0.27.2-py3-none-any.whl.metadata (7.1 kB)
Collecting jiter<1,>=0.4.0 (from openai->-r requirements.txt (line 1))
  Using cached jiter-0.5.0-cp312-cp312-macosx_11_0_arm64.whl.metadata (3.6 kB)
Collecting pydantic<3,>=1.9.0 (from openai->-r requirements.txt (line 1))
  Using cached pydantic-2.8.2-py3-none-any.whl.metadata (125 kB)
Collecting sniffio (from openai->-r requirements.txt (line 1))
  Using cached sniffio-1.3.1-py3-none-any.whl.metadata (3.9 kB)
Collecting tqdm>4 (from openai->-r requirements.txt (line 1))
  Using cached tqdm-4.66.5-py3-none-any.whl.metadata (57 kB)
Collecting typing-extensions<5,>=4.11 (from openai->-r requirements.txt (line 1))
  Using cached typing_extensions-4.12.2-py3-none-any.whl.metadata (3.0 kB)
Collecting mlx>=0.17.0 (from mlx-lm->-r requirements.txt (line 3))
  Downloading mlx-0.17.2-cp312-cp312-macosx_14_0_arm64.whl.metadata (5.1 kB)
Collecting numpy (from mlx-lm->-r requirements.txt (line 3))
  Using cached numpy-2.1.0-cp312-cp312-macosx_14_0_arm64.whl.metadata (60 kB)
Collecting transformers>=4.39.3 (from transformers[sentencepiece]>=4.39.3->mlx-lm->-r requirements.txt (line 3))
  Using cached transformers-4.44.2-py3-none-any.whl.metadata (43 kB)
Collecting protobuf (from mlx-lm->-r requirements.txt (line 3))
  Using cached protobuf-5.28.0-cp38-abi3-macosx_10_9_universal2.whl.metadata (592 bytes)
Collecting pyyaml (from mlx-lm->-r requirements.txt (line 3))
  Using cached PyYAML-6.0.2-cp312-cp312-macosx_11_0_arm64.whl.metadata (2.1 kB)
Collecting jinja2 (from mlx-lm->-r requirements.txt (line 3))
  Using cached jinja2-3.1.4-py3-none-any.whl.metadata (2.6 kB)
Collecting idna>=2.8 (from anyio<5,>=3.5.0->openai->-r requirements.txt (line 1))
  Using cached idna-3.8-py3-none-any.whl.metadata (9.9 kB)
Collecting certifi (from httpx<1,>=0.23.0->openai->-r requirements.txt (line 1))
  Downloading certifi-2024.8.30-py3-none-any.whl.metadata (2.2 kB)
Collecting httpcore==1.* (from httpx<1,>=0.23.0->openai->-r requirements.txt (line 1))
  Using cached httpcore-1.0.5-py3-none-any.whl.metadata (20 kB)
Collecting h11<0.15,>=0.13 (from httpcore==1.*->httpx<1,>=0.23.0->openai->-r requirements.txt (line 1))
  Using cached h11-0.14.0-py3-none-any.whl.metadata (8.2 kB)
Collecting annotated-types>=0.4.0 (from pydantic<3,>=1.9.0->openai->-r requirements.txt (line 1))
  Using cached annotated_types-0.7.0-py3-none-any.whl.metadata (15 kB)
Collecting pydantic-core==2.20.1 (from pydantic<3,>=1.9.0->openai->-r requirements.txt (line 1))
  Using cached pydantic_core-2.20.1-cp312-cp312-macosx_11_0_arm64.whl.metadata (6.6 kB)
Collecting filelock (from transformers>=4.39.3->transformers[sentencepiece]>=4.39.3->mlx-lm->-r requirements.txt (line 3))
  Using cached filelock-3.15.4-py3-none-any.whl.metadata (2.9 kB)
Collecting huggingface-hub<1.0,>=0.23.2 (from transformers>=4.39.3->transformers[sentencepiece]>=4.39.3->mlx-lm->-r requirements.txt (line 3))
  Using cached huggingface_hub-0.24.6-py3-none-any.whl.metadata (13 kB)
Collecting packaging>=20.0 (from transformers>=4.39.3->transformers[sentencepiece]>=4.39.3->mlx-lm->-r requirements.txt (line 3))
  Using cached packaging-24.1-py3-none-any.whl.metadata (3.2 kB)
Collecting regex!=2019.12.17 (from transformers>=4.39.3->transformers[sentencepiece]>=4.39.3->mlx-lm->-r requirements.txt (line 3))
  Using cached regex-2024.7.24-cp312-cp312-macosx_11_0_arm64.whl.metadata (40 kB)
Collecting requests (from transformers>=4.39.3->transformers[sentencepiece]>=4.39.3->mlx-lm->-r requirements.txt (line 3))
  Using cached requests-2.32.3-py3-none-any.whl.metadata (4.6 kB)
Collecting safetensors>=0.4.1 (from transformers>=4.39.3->transformers[sentencepiece]>=4.39.3->mlx-lm->-r requirements.txt (line 3))
  Using cached safetensors-0.4.4-cp312-cp312-macosx_11_0_arm64.whl.metadata (3.8 kB)
Collecting tokenizers<0.20,>=0.19 (from transformers>=4.39.3->transformers[sentencepiece]>=4.39.3->mlx-lm->-r requirements.txt (line 3))
  Using cached tokenizers-0.19.1-cp312-cp312-macosx_11_0_arm64.whl.metadata (6.7 kB)
Collecting sentencepiece!=0.1.92,>=0.1.91 (from transformers[sentencepiece]>=4.39.3->mlx-lm->-r requirements.txt (line 3))
  Using cached sentencepiece-0.2.0-cp312-cp312-macosx_11_0_arm64.whl.metadata (7.7 kB)
Collecting MarkupSafe>=2.0 (from jinja2->mlx-lm->-r requirements.txt (line 3))
  Using cached MarkupSafe-2.1.5-cp312-cp312-macosx_10_9_universal2.whl.metadata (3.0 kB)
Collecting fsspec>=2023.5.0 (from huggingface-hub<1.0,>=0.23.2->transformers>=4.39.3->transformers[sentencepiece]>=4.39.3->mlx-lm->-r requirements.txt (line 3))
  Using cached fsspec-2024.6.1-py3-none-any.whl.metadata (11 kB)
Collecting charset-normalizer<4,>=2 (from requests->transformers>=4.39.3->transformers[sentencepiece]>=4.39.3->mlx-lm->-r requirements.txt (line 3))
  Using cached charset_normalizer-3.3.2-cp312-cp312-macosx_11_0_arm64.whl.metadata (33 kB)
Collecting urllib3<3,>=1.21.1 (from requests->transformers>=4.39.3->transformers[sentencepiece]>=4.39.3->mlx-lm->-r requirements.txt (line 3))
  Using cached urllib3-2.2.2-py3-none-any.whl.metadata (6.4 kB)
Downloading openai-1.43.0-py3-none-any.whl (365 kB)
Using cached tree_sitter-0.23.0-cp312-cp312-macosx_11_0_arm64.whl (129 kB)
Downloading mlx_lm-0.18.1-py3-none-any.whl (109 kB)
Using cached anyio-4.4.0-py3-none-any.whl (86 kB)
Using cached distro-1.9.0-py3-none-any.whl (20 kB)
Using cached httpx-0.27.2-py3-none-any.whl (76 kB)
Using cached httpcore-1.0.5-py3-none-any.whl (77 kB)
Using cached jiter-0.5.0-cp312-cp312-macosx_11_0_arm64.whl (296 kB)
Downloading mlx-0.17.2-cp312-cp312-macosx_14_0_arm64.whl (21.4 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 21.4/21.4 MB 9.4 MB/s eta 0:00:00                                                                                                            
Using cached pydantic-2.8.2-py3-none-any.whl (423 kB)
Using cached pydantic_core-2.20.1-cp312-cp312-macosx_11_0_arm64.whl (1.8 MB)
Using cached sniffio-1.3.1-py3-none-any.whl (10 kB)
Using cached tqdm-4.66.5-py3-none-any.whl (78 kB)
Using cached transformers-4.44.2-py3-none-any.whl (9.5 MB)
Using cached numpy-2.1.0-cp312-cp312-macosx_14_0_arm64.whl (5.1 MB)
Using cached PyYAML-6.0.2-cp312-cp312-macosx_11_0_arm64.whl (173 kB)
Using cached typing_extensions-4.12.2-py3-none-any.whl (37 kB)
Using cached jinja2-3.1.4-py3-none-any.whl (133 kB)
Using cached protobuf-5.28.0-cp38-abi3-macosx_10_9_universal2.whl (414 kB)
Using cached annotated_types-0.7.0-py3-none-any.whl (13 kB)
Using cached huggingface_hub-0.24.6-py3-none-any.whl (417 kB)
Using cached idna-3.8-py3-none-any.whl (66 kB)
Using cached MarkupSafe-2.1.5-cp312-cp312-macosx_10_9_universal2.whl (18 kB)
Using cached packaging-24.1-py3-none-any.whl (53 kB)
Using cached regex-2024.7.24-cp312-cp312-macosx_11_0_arm64.whl (279 kB)
Using cached safetensors-0.4.4-cp312-cp312-macosx_11_0_arm64.whl (381 kB)
Using cached sentencepiece-0.2.0-cp312-cp312-macosx_11_0_arm64.whl (1.2 MB)
Using cached tokenizers-0.19.1-cp312-cp312-macosx_11_0_arm64.whl (2.4 MB)
Downloading certifi-2024.8.30-py3-none-any.whl (167 kB)
Using cached filelock-3.15.4-py3-none-any.whl (16 kB)
Using cached requests-2.32.3-py3-none-any.whl (64 kB)
Using cached charset_normalizer-3.3.2-cp312-cp312-macosx_11_0_arm64.whl (119 kB)
Using cached fsspec-2024.6.1-py3-none-any.whl (177 kB)
Using cached h11-0.14.0-py3-none-any.whl (58 kB)
Using cached urllib3-2.2.2-py3-none-any.whl (121 kB)
Building wheels for collected packages: tree-sitter-swift
  Building wheel for tree-sitter-swift (pyproject.toml) ... error
  error: subprocess-exited-with-error

  × Building wheel for tree-sitter-swift (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [38 lines of output]
      running bdist_wheel
      running build
      creating build
      creating build/lib.macosx-14.0-arm64-cpython-312
      creating build/lib.macosx-14.0-arm64-cpython-312/tree_sitter_swift
      creating build/lib.macosx-14.0-arm64-cpython-312/tree_sitter_swift/queries
      copying queries/tags.scm -> build/lib.macosx-14.0-arm64-cpython-312/tree_sitter_swift/queries
      copying queries/outline.scm -> build/lib.macosx-14.0-arm64-cpython-312/tree_sitter_swift/queries
      copying queries/locals.scm -> build/lib.macosx-14.0-arm64-cpython-312/tree_sitter_swift/queries
      copying queries/indents.scm -> build/lib.macosx-14.0-arm64-cpython-312/tree_sitter_swift/queries
      copying queries/highlights.scm -> build/lib.macosx-14.0-arm64-cpython-312/tree_sitter_swift/queries
      copying queries/textobjects.scm -> build/lib.macosx-14.0-arm64-cpython-312/tree_sitter_swift/queries
      copying queries/injections.scm -> build/lib.macosx-14.0-arm64-cpython-312/tree_sitter_swift/queries
      running build_py
      copying bindings/python/tree_sitter_swift/__init__.py -> build/lib.macosx-14.0-arm64-cpython-312/tree_sitter_swift
      running egg_info
      writing bindings/python/tree_sitter_swift.egg-info/PKG-INFO
      writing dependency_links to bindings/python/tree_sitter_swift.egg-info/dependency_links.txt
      writing requirements to bindings/python/tree_sitter_swift.egg-info/requires.txt
      writing top-level names to bindings/python/tree_sitter_swift.egg-info/top_level.txt
      reading manifest file 'bindings/python/tree_sitter_swift.egg-info/SOURCES.txt'
      adding license file 'LICENSE'
      writing manifest file 'bindings/python/tree_sitter_swift.egg-info/SOURCES.txt'
      copying bindings/python/tree_sitter_swift/__init__.pyi -> build/lib.macosx-14.0-arm64-cpython-312/tree_sitter_swift
      copying bindings/python/tree_sitter_swift/binding.c -> build/lib.macosx-14.0-arm64-cpython-312/tree_sitter_swift
      copying bindings/python/tree_sitter_swift/py.typed -> build/lib.macosx-14.0-arm64-cpython-312/tree_sitter_swift
      running build_ext
      building '_binding' extension
      creating build/temp.macosx-14.0-arm64-cpython-312
      creating build/temp.macosx-14.0-arm64-cpython-312/bindings
      creating build/temp.macosx-14.0-arm64-cpython-312/bindings/python
      creating build/temp.macosx-14.0-arm64-cpython-312/bindings/python/tree_sitter_swift
      creating build/temp.macosx-14.0-arm64-cpython-312/src
      clang -fno-strict-overflow -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -O3 -Wall -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX14.sdk -DPy_LIMITED_API=0x03080000 -DPY_SSIZE_T_CLEAN -Isrc -I/venvs/include -I/opt/homebrew/opt/python@3.12/Frameworks/Python.framework/Versions/3.12/include/python3.12 -c bindings/python/tree_sitter_swift/binding.c -o build/temp.macosx-14.0-arm64-cpython-312/bindings/python/tree_sitter_swift/binding.o -std=c11
      clang -fno-strict-overflow -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -O3 -Wall -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX14.sdk -DPy_LIMITED_API=0x03080000 -DPY_SSIZE_T_CLEAN -Isrc -I/venvs/include -I/opt/homebrew/opt/python@3.12/Frameworks/Python.framework/Versions/3.12/include/python3.12 -c src/parser.c -o build/temp.macosx-14.0-arm64-cpython-312/src/parser.o -std=c11
      clang: error: no such file or directory: 'src/parser.c'
      clang: error: no input files
      error: command '/usr/bin/clang' failed with exit code 1
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for tree-sitter-swift
Failed to build tree-sitter-swift
ERROR: ERROR: Failed to build installable wheels for some pyproject.toml based projects (tree-sitter-swift)
(swift4llm) ➜  swift4lmm
alex-pinkus commented 2 months ago

There’s a special branch for this. I don’t like it and I discourage its use (checking these artifacts into Git is a bad practice) so I’ll let you figure out the details from here ;). If that branch is missing anything, happy to update the script.

yaroslavyaroslav commented 2 months ago

I suggested PR to solve this just yet. It's perfectly fine (AFAIK) for a python package to take some building steps during installation, it's just that they have to be declared in setup.py to let the tooling know how to handle this.