search

tree-sitter / py-tree-sitter

Python bindings to the Tree-sitter parsing library
https://tree-sitter.github.io/py-tree-sitter/
MIT License
817 stars 96 forks source link

Windows ARM64 wheel not provided? #281

Closed vintagedave closed 2 weeks ago

vintagedave commented 3 weeks ago

Hello. The main docs say that wheels are provided for 'all major platforms', but it appears that this does not include Windows ARM (which is fair, it may not count as a major platform yet.) Could one be provided please?

I am using v0.21 (due to this issue which is a real problem.) Installing tree-sitter from the MSVC build tools v14 command prompt gives the following. This means it appears to be building the ARM64 wheel itself (that's why I installed the build tools), ie, it's not a prebuilt wheel. The fact that it does not link is something I have not yet figured out a reason or solution for.

C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools>py -m pip install --force-reinstall -v "tree-sitter==0.21
"
Using pip 24.2 from C:\Users\work\AppData\Local\Programs\Python\Python311-arm64\Lib\site-packages\pip (python 3.11)
Collecting tree-sitter==0.21
  Using cached tree-sitter-0.21.0.tar.gz (155 kB)
  Running command pip subprocess to install build dependencies
  Using pip 24.2 from C:\Users\work\AppData\Local\Programs\Python\Python311-arm64\Lib\site-packages\pip (python 3.11)
  Collecting setuptools>=43
    Obtaining dependency information for setuptools>=43 from https://files.pythonhosted.org/packages/07/6a/0270e295bf30c37567736b7fca10167640898214ff911273af37ddb95770/setuptools-73.0.1-py3-none-any.whl.metadata
    Using cached setuptools-73.0.1-py3-none-any.whl.metadata (6.6 kB)
  Using cached setuptools-73.0.1-py3-none-any.whl (2.3 MB)
  Installing collected packages: setuptools
  Successfully installed setuptools-73.0.1
  Installing build dependencies ... done
  Running command Getting requirements to build wheel
  running egg_info
  writing tree_sitter.egg-info\PKG-INFO
  writing dependency_links to tree_sitter.egg-info\dependency_links.txt
  writing top-level names to tree_sitter.egg-info\top_level.txt
  reading manifest file 'tree_sitter.egg-info\SOURCES.txt'
  reading manifest template 'MANIFEST.in'
  no previously-included directories found matching 'tree_sitter\core\lib\src\wasm'
  adding license file 'LICENSE'
  writing manifest file 'tree_sitter.egg-info\SOURCES.txt'
  Getting requirements to build wheel ... done
  Running command Preparing metadata (pyproject.toml)
  running dist_info
  creating C:\Users\work\AppData\Local\Temp\pip-modern-metadata-r0woe_nh\tree_sitter.egg-info
  writing C:\Users\work\AppData\Local\Temp\pip-modern-metadata-r0woe_nh\tree_sitter.egg-info\PKG-INFO
  writing dependency_links to C:\Users\work\AppData\Local\Temp\pip-modern-metadata-r0woe_nh\tree_sitter.egg-info\dependency_links.txt
  writing top-level names to C:\Users\work\AppData\Local\Temp\pip-modern-metadata-r0woe_nh\tree_sitter.egg-info\top_level.txt
  writing manifest file 'C:\Users\work\AppData\Local\Temp\pip-modern-metadata-r0woe_nh\tree_sitter.egg-info\SOURCES.txt'
  reading manifest file 'C:\Users\work\AppData\Local\Temp\pip-modern-metadata-r0woe_nh\tree_sitter.egg-info\SOURCES.txt'
  reading manifest template 'MANIFEST.in'
  no previously-included directories found matching 'tree_sitter\core'
  no previously-included directories found matching 'tree_sitter\core\lib\src\wasm'
  adding license file 'LICENSE'
  writing manifest file 'C:\Users\work\AppData\Local\Temp\pip-modern-metadata-r0woe_nh\tree_sitter.egg-info\SOURCES.txt'
  creating 'C:\Users\work\AppData\Local\Temp\pip-modern-metadata-r0woe_nh\tree_sitter-0.21.0.dist-info'
  Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: tree-sitter
  Running command Building wheel for tree-sitter (pyproject.toml)
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build\lib.win32-cpython-311
  creating build\lib.win32-cpython-311\tree_sitter
  copying tree_sitter\__init__.py -> build\lib.win32-cpython-311\tree_sitter
  copying tree_sitter\_binding.pyi -> build\lib.win32-cpython-311\tree_sitter
  copying tree_sitter\py.typed -> build\lib.win32-cpython-311\tree_sitter
  running build_ext
  building 'tree_sitter._binding' extension
  creating build\temp.win32-cpython-311
  creating build\temp.win32-cpython-311\Release
  creating build\temp.win32-cpython-311\Release\tree_sitter
  creating build\temp.win32-cpython-311\Release\tree_sitter\core
  creating build\temp.win32-cpython-311\Release\tree_sitter\core\lib
  creating build\temp.win32-cpython-311\Release\tree_sitter\core\lib\src
  "C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.41.34120\bin\HostX86\x86\cl.exe" /c /nologo /O2 /W3 /GL /DNDEBUG /MD -DPY_SSIZE_T_CLEAN -UTREE_SITTER_FEATURE_WASM -Itree_sitter/core/lib/include -Itree_sitter/core/lib/src -IC:\Users\work\AppData\Local\Programs\Python\Python311-arm64\include -IC:\Users\work\AppData\Local\Programs\Python\Python311-arm64\Include "-IC:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.41.34120\include" "-IC:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Auxiliary\VS\include" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\um" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\shared" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\winrt" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\cppwinrt" "-IC:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.41.34120\include" "-IC:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Auxiliary\VS\include" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\um" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\shared" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\winrt" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\cppwinrt" /Tctree_sitter/binding.c /Fobuild\temp.win32-cpython-311\Release\tree_sitter/binding.obj
  binding.c
  tree_sitter/binding.c(1562): warning C4244: 'function': conversion from 'uint64_t' to 'unsigned long', possible loss of data
  "C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.41.34120\bin\HostX86\x86\cl.exe" /c /nologo /O2 /W3 /GL /DNDEBUG /MD -DPY_SSIZE_T_CLEAN -UTREE_SITTER_FEATURE_WASM -Itree_sitter/core/lib/include -Itree_sitter/core/lib/src -IC:\Users\work\AppData\Local\Programs\Python\Python311-arm64\include -IC:\Users\work\AppData\Local\Programs\Python\Python311-arm64\Include "-IC:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.41.34120\include" "-IC:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Auxiliary\VS\include" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\um" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\shared" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\winrt" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\cppwinrt" "-IC:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.41.34120\include" "-IC:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Auxiliary\VS\include" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\um" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\shared" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\winrt" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\cppwinrt" /Tctree_sitter/core/lib/src/lib.c /Fobuild\temp.win32-cpython-311\Release\tree_sitter/core/lib/src/lib.obj
  lib.c
  C:\Users\work\AppData\Local\Temp\pip-install-ravjci_k\tree-sitter_b862f86a0af143eba4fb24144907e5eb\tree_sitter\core\lib\src\./query.c(3392): warning C4018: '>': signed/unsigned mismatch
  C:\Users\work\AppData\Local\Temp\pip-install-ravjci_k\tree-sitter_b862f86a0af143eba4fb24144907e5eb\tree_sitter\core\lib\src\./subtree.c(183): warning C4244: 'initializing': conversion from 'TSSymbol' to 'uint8_t', possible loss of data
  C:\Users\work\AppData\Local\Temp\pip-install-ravjci_k\tree-sitter_b862f86a0af143eba4fb24144907e5eb\tree_sitter\core\lib\src\./subtree.c(233): warning C4244: '=': conversion from 'TSSymbol' to 'uint8_t', possible loss of data
  "C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.41.34120\bin\HostX86\x86\link.exe" /nologo /INCREMENTAL:NO /LTCG /DLL /MANIFEST:EMBED,ID=2 /MANIFESTUAC:NO /LIBPATH:C:\Users\work\AppData\Local\Programs\Python\Python311-arm64\libs /LIBPATH:C:\Users\work\AppData\Local\Programs\Python\Python311-arm64 /LIBPATH:C:\Users\work\AppData\Local\Programs\Python\Python311-arm64\PCbuild\win32 "/LIBPATH:C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.41.34120\lib\x86" "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\lib\10.0.22621.0\ucrt\x86" "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\\lib\10.0.22621.0\\um\x86" "/LIBPATH:C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.41.34120\lib\x86" "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\lib\10.0.22621.0\ucrt\x86" "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\\lib\10.0.22621.0\\um\x86" /EXPORT:PyInit__binding build\temp.win32-cpython-311\Release\tree_sitter/binding.obj build\temp.win32-cpython-311\Release\tree_sitter/core/lib/src/lib.obj /OUT:build\lib.win32-cpython-311\tree_sitter\_binding.cp311-win_arm64.pyd /IMPLIB:build\temp.win32-cpython-311\Release\tree_sitter\_binding.cp311-win_arm64.lib
     Creating library build\temp.win32-cpython-311\Release\tree_sitter\_binding.cp311-win_arm64.lib and object build\temp.win32-cpython-311\Release\tree_sitter\_binding.cp311-win_arm64.exp
  binding.obj : error LNK2001: unresolved external symbol __imp___PyArg_Parse_SizeT
  binding.obj : error LNK2001: unresolved external symbol __imp__PyDict_SetDefault
  binding.obj : error LNK2001: unresolved external symbol __imp__PyBool_FromLong
  binding.obj : error LNK2001: unresolved external symbol __imp___PyObject_CallFunction_SizeT
  binding.obj : error LNK2001: unresolved external symbol __imp__PyImport_ImportModule
  binding.obj : error LNK2001: unresolved external symbol __imp__PyBytes_AsString
  binding.obj : error LNK2001: unresolved external symbol __imp__PyBytes_FromObject
  binding.obj : error LNK2001: unresolved external symbol __imp__PyExc_RuntimeError
  binding.obj : error LNK2001: unresolved external symbol __imp__PyLong_FromLong
  binding.obj : error LNK2001: unresolved external symbol __imp__PyErr_SetNone
  binding.obj : error LNK2001: unresolved external symbol __imp__PyExc_MemoryError
  binding.obj : error LNK2001: unresolved external symbol __imp__PyBytes_FromString
  binding.obj : error LNK2001: unresolved external symbol __imp___Py_NoneStruct
  binding.obj : error LNK2001: unresolved external symbol __imp__PyMemoryView_FromObject
  binding.obj : error LNK2001: unresolved external symbol __imp__PyList_Size
  binding.obj : error LNK2001: unresolved external symbol __imp__PyErr_SetString
  binding.obj : error LNK2001: unresolved external symbol __imp__PyExc_ValueError
  binding.obj : error LNK2001: unresolved external symbol __imp__PyLong_FromUnsignedLong
  binding.obj : error LNK2001: unresolved external symbol __imp___PyArg_ParseTupleAndKeywords_SizeT
  binding.obj : error LNK2001: unresolved external symbol __imp__PyErr_Format
  binding.obj : error LNK2001: unresolved external symbol __imp__PyTuple_Type
  binding.obj : error LNK2001: unresolved external symbol __imp___Py_FalseStruct
  binding.obj : error LNK2001: unresolved external symbol __imp___Py_Dealloc
  binding.obj : error LNK2001: unresolved external symbol __imp__PyTuple_GetItem
  binding.obj : error LNK2001: unresolved external symbol __imp__PyModule_AddObjectRef
  binding.obj : error LNK2001: unresolved external symbol __imp___PyArg_ParseTuple_SizeT
  binding.obj : error LNK2001: unresolved external symbol __imp__PyUnicode_FromFormat
  binding.obj : error LNK2001: unresolved external symbol __imp__PyObject_GetBuffer
  binding.obj : error LNK2001: unresolved external symbol __imp__PyList_New
  binding.obj : error LNK2001: unresolved external symbol __imp__PyModule_Create2
  binding.obj : error LNK2001: unresolved external symbol __imp__PySlice_New
  binding.obj : error LNK2001: unresolved external symbol __imp__PyExc_SyntaxError
  binding.obj : error LNK2001: unresolved external symbol __imp__PyObject_GetAttrString
  binding.obj : error LNK2001: unresolved external symbol __imp___PyObject_CallMethod_SizeT
  binding.obj : error LNK2001: unresolved external symbol __imp__PyErr_Clear
  binding.obj : error LNK2001: unresolved external symbol __imp__PyList_Append
  binding.obj : error LNK2001: unresolved external symbol __imp__PyObject_RichCompareBool
  binding.obj : error LNK2001: unresolved external symbol __imp__PyTuple_Size
  binding.obj : error LNK2001: unresolved external symbol __imp__PyType_GetModuleState
  binding.obj : error LNK2001: unresolved external symbol __imp___Py_HashPointer
  binding.obj : error LNK2001: unresolved external symbol __imp__PyBytes_Size
  binding.obj : error LNK2001: unresolved external symbol __imp__PyDict_SetItem
  binding.obj : error LNK2001: unresolved external symbol __imp__PyDict_New
  binding.obj : error LNK2001: unresolved external symbol __imp__PyList_GetItem
  binding.obj : error LNK2001: unresolved external symbol __imp__PyObject_IsInstance
  binding.obj : error LNK2001: unresolved external symbol __imp__PyLong_FromVoidPtr
  binding.obj : error LNK2001: unresolved external symbol __imp__PyExc_StopIteration
  binding.obj : error LNK2001: unresolved external symbol __imp__PyLong_AsVoidPtr
  binding.obj : error LNK2001: unresolved external symbol __imp__PyObject_GetItem
  binding.obj : error LNK2001: unresolved external symbol __imp__PyExc_TypeError
  binding.obj : error LNK2001: unresolved external symbol __imp__PyType_FromModuleAndSpec
  binding.obj : error LNK2001: unresolved external symbol __imp__PyExc_NameError
  binding.obj : error LNK2001: unresolved external symbol __imp__PyTuple_Pack
  binding.obj : error LNK2001: unresolved external symbol __imp__PyCallable_Check
  binding.obj : error LNK2001: unresolved external symbol __imp__PyModule_GetState
  binding.obj : error LNK2001: unresolved external symbol __imp__PyList_SetItem
  binding.obj : error LNK2001: unresolved external symbol __imp__PyUnicode_FromString
  binding.obj : error LNK2001: unresolved external symbol __imp__PyLong_FromSize_t
  binding.obj : error LNK2001: unresolved external symbol __imp__PyBuffer_Release
  binding.obj : error LNK2001: unresolved external symbol __imp__PyObject_Call
  binding.obj : error LNK2001: unresolved external symbol __imp__PyUnicode_FromStringAndSize
  build\lib.win32-cpython-311\tree_sitter\_binding.cp311-win_arm64.pyd : fatal error LNK1120: 61 unresolved externals
  error: command 'C:\\Program Files (x86)\\Microsoft Visual Studio\\2022\\BuildTools\\VC\\Tools\\MSVC\\14.41.34120\\bin\\HostX86\\x86\\link.exe' failed with exit code 1120
  error: subprocess-exited-with-error

  × Building wheel for tree-sitter (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> See above for output.

  note: This error originates from a subprocess, and is likely not a problem with pip.
  full command: 'C:\Users\work\AppData\Local\Programs\Python\Python311-arm64\python.exe' 'C:\Users\work\AppData\Local\Programs\Python\Python311-arm64\Lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py' build_wheel 'C:\Users\work\AppData\Local\Temp\tmphv9hh3pn'
  cwd: C:\Users\work\AppData\Local\Temp\pip-install-ravjci_k\tree-sitter_b862f86a0af143eba4fb24144907e5eb
  Building wheel for tree-sitter (pyproject.toml) ... error
  ERROR: Failed building wheel for tree-sitter
Failed to build tree-sitter
ERROR: ERROR: Failed to build installable wheels for some pyproject.toml based projects (tree-sitter)
ObserverOfTime commented 3 weeks ago

It won't be backported though.

vintagedave commented 3 weeks ago

I understand -- I am planning to try to use a newer version of tree-sitter.

It was through installing at all that I realised there was no ARM64 wheel, and it seemed worth filing an issue :)

vintagedave commented 2 weeks ago

Thankyou!