search

chroma-core / chroma

the AI-native open-source embedding database
https://www.trychroma.com/
Apache License 2.0
15.43k stars 1.3k forks source link

[Install issue]: Cannot build hnswlib extension #1485

Open TheRealBeef opened 11 months ago

TheRealBeef commented 11 months ago

What happened?

I'm installing a separate plugin for LM Studio that depends on ChromaDB, but the installation of ChromaDB continues to fail despite my best efforts.

I've tried Python 3.9, 3.10, and 3.11 and have the same issue.

First, I ran into the "requires MS Build tools" error that seems common here, but after installing the build tools with its C++ development things am met with this error related to building the wheel for hnswlib, which I see is a common error when trying to install ChromaDB. However, none of the solutions that I've seen here in Issues or elsewhere on forums seem to resolve it.

I've gone back and manually added the directories for the Build Tools / SDK to the PATH but that didn't seem to solve anything either.

Building/running in WSL (one other solution I've seen) isn't an option as I run a number of VirtualBox VMs and VirtualBox gets wrecked by the virtualization service that WSL needs.

Versions

ChromaDB 0.3.29-py3, Python 3.9/3.10/3.11, Windows 11

Relevant log output

Building wheels for collected packages: hnswlib
  Building wheel for hnswlib (pyproject.toml) ... error
  error: subprocess-exited-with-error

  × Building wheel for hnswlib (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [12 lines of output]
      running bdist_wheel
      running build
      running build_ext
      building 'hnswlib' extension
      creating build
      creating build\temp.win-amd64-cpython-311
      creating build\temp.win-amd64-cpython-311\Release
      creating build\temp.win-amd64-cpython-311\Release\python_bindings
      "C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.38.33130\bin\HostX86\x64\cl.exe" /c /nologo /O2 /W3 /GL /DNDEBUG /MD -IC:\Users\luser\AppData\Local\Temp\pip-build-env-qny0404k\overlay\Lib\site-packages\pybind11\include -IC:\Users\luser\AppData\Local\Temp\pip-build-env-qny0404k\overlay\Lib\site-packages\numpy\core\include -I./hnswlib/ -IC:\ChromaDB-Plugin-for-LM-Studio-2.7\src\include "-IC:\Program Files\Python311\include" "-IC:\Program Files\Python311\Include" "-IC:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.38.33130\include" "-IC:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.38.33130\ATLMFC\include" "-IC:\Program Files\Microsoft Visual Studio\2022\Community\VC\Auxiliary\VS\include" /EHsc /Tp./python_bindings/bindings.cpp /Fobuild\temp.win-amd64-cpython-311\Release\./python_bindings/bindings.obj /EHsc /openmp /O2 /DVERSION_INFO=\\\"0.8.0\\\"
      bindings.cpp
      C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.38.33130\include\yvals.h(20): fatal error C1083: Cannot open include file: 'crtdbg.h': No such file or directory
      error: command 'C:\\Program Files\\Microsoft Visual Studio\\2022\\Community\\VC\\Tools\\MSVC\\14.38.33130\\bin\\HostX86\\x64\\cl.exe' failed with exit code 2
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for hnswlib
Failed to build hnswlib
ERROR: Could not build wheels for hnswlib, which is required to install pyproject.toml-based projects
TheRealBeef commented 11 months ago

Running the same steps, except in Developer Powershell for VS 2022 instead of the command prompt or powershell, provides much more verbose output

Building wheels for collected packages: hnswlib
  Building wheel for hnswlib (pyproject.toml) ... error
  error: subprocess-exited-with-error

  × Building wheel for hnswlib (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [223 lines of output]
      running bdist_wheel
      running build
      running build_ext
      building 'hnswlib' extension
      creating build
      creating build\temp.win32-cpython-311
      creating build\temp.win32-cpython-311\Release
      creating build\temp.win32-cpython-311\Release\python_bindings
      "C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.38.33130\bin\HostX86\x86\cl.exe" /c /nologo /O2 /W3 /GL /DNDEBUG /MD -IC:\Users\luser\AppData\Local\Temp\pip-build-env-fb1kaijv\overlay\Lib\site-packages\pybind11\include -IC:\Users\luser\AppData\Local\Temp\pip-build-env-fb1kaijv\overlay\Lib\site-packages\numpy\core\include -I./hnswlib/ -IC:\ChromaDB-Plugin-for-LM-Studio-2.7\src\include "-IC:\Program Files\Python311\include" "-IC:\Program Files\Python311\Include" "-IC:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.38.33130\include" "-IC:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.38.33130\ATLMFC\include" "-IC:\Program Files\Microsoft Visual Studio\2022\Community\VC\Auxiliary\VS\include" -Iinclude\10.0.22000.0\ucrt "-IC:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.38.33130\include" "-IC:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.38.33130\ATLMFC\include" "-IC:\Program Files\Microsoft Visual Studio\2022\Community\VC\Auxiliary\VS\include" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.22000.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22000.0\\um" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22000.0\\shared" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22000.0\\winrt" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22000.0\\cppwinrt" "-IC:\Program Files (x86)\Windows Kits\NETFXSDK\4.8\include\um" /EHsc /Tp./python_bindings/bindings.cpp /Fobuild\temp.win32-cpython-311\Release\./python_bindings/bindings.obj /EHsc /openmp /O2 /DVERSION_INFO=\\\"0.8.0\\\"
      bindings.cpp
      ./python_bindings/bindings.cpp(910): warning C4996: 'pybind11_init': PYBIND11_PLUGIN is deprecated, use PYBIND11_MODULE
      ./python_bindings/bindings.cpp(911): warning C4996: 'pybind11::module_::module_': Use PYBIND11_MODULE or module_::create_extension_module instead
      ./python_bindings/bindings.cpp(631): warning C4018: '<=': signed/unsigned mismatch
      ./python_bindings/bindings.cpp(631): note: the template instantiation context (the oldest one first) is
      ./python_bindings/bindings.cpp(913): note: see reference to class template instantiation 'pybind11::class_<Index<float,float>>' being compiled
      C:\Users\luser\AppData\Local\Temp\pip-build-env-fb1kaijv\overlay\Lib\site-packages\pybind11\include\pybind11\pybind11.h(1516): note: see reference to class template instantiation 'std::is_polymorphic<Index<float,float>>' being compiled
      C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.38.33130\include\type_traits(658): note: see reference to class template instantiation 'Index<float,float>' being compiled
      ./python_bindings/bindings.cpp(612): note: while compiling class template member function 'pybind11::object Index<float,float>::knnQuery_return_numpy(pybind11::object,size_t,int,const std::function<bool (hnswlib::labeltype)> &)'
      ./python_bindings/bindings.cpp(988): note: see the first reference to 'Index<float,float>::knnQuery_return_numpy' in 'pybind11_init'
      ./python_bindings/bindings.cpp(264): warning C4018: '<=': signed/unsigned mismatch
      ./python_bindings/bindings.cpp(264): note: the template instantiation context (the oldest one first) is
      ./python_bindings/bindings.cpp(251): note: while compiling class template member function 'void Index<float,float>::addItems(pybind11::object,pybind11::object,int,bool)'
      ./python_bindings/bindings.cpp(988): note: see the first reference to 'Index<float,float>::addItems' in 'pybind11_init'
      C:\Users\luser\AppData\Local\Temp\pip-install-qmosbx4k\hnswlib_a35c5e0ef44041749b33fcb2a6c6dc9d\hnswlib\bruteforce.h(111): warning C4018: '<': signed/unsigned mismatch
      C:\Users\luser\AppData\Local\Temp\pip-install-qmosbx4k\hnswlib_a35c5e0ef44041749b33fcb2a6c6dc9d\hnswlib\bruteforce.h(111): note: the template instantiation context (the oldest one first) is
      ./python_bindings/bindings.cpp(992): note: see reference to class template instantiation 'pybind11::class_<BFIndex<float,float>>' being compiled
      C:\Users\luser\AppData\Local\Temp\pip-build-env-fb1kaijv\overlay\Lib\site-packages\pybind11\include\pybind11\pybind11.h(1516): note: see reference to class template instantiation 'std::is_polymorphic<BFIndex<float,float>>' being compiled
      C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.38.33130\include\type_traits(658): note: see reference to class template instantiation 'BFIndex<float,float>' being compiled
      ./python_bindings/bindings.cpp(850): note: while compiling class template member function 'pybind11::object BFIndex<float,float>::knnQuery_return_numpy(pybind11::object,size_t,int,const std::function<bool (hnswlib::labeltype)> &)'
      ./python_bindings/bindings.cpp(1011): note: see the first reference to 'BFIndex<float,float>::knnQuery_return_numpy' in 'pybind11_init'
      ./python_bindings/bindings.cpp(875): note: see reference to class template instantiation 'hnswlib::BruteforceSearch<dist_t>' being compiled
              with
              [
                  dist_t=float
              ]
      C:\Users\luser\AppData\Local\Temp\pip-install-qmosbx4k\hnswlib_a35c5e0ef44041749b33fcb2a6c6dc9d\hnswlib\bruteforce.h(107): note: while compiling class template member function 'std::priority_queue<std::pair<dist_t,hnswlib::labeltype>,std::vector<std::pair<dist_t,hnswlib::labeltype>,std::allocator<std::pair<dist_t,hnswlib::labeltype>>>,std::less<_Ty>> hnswlib::BruteforceSearch<dist_t>::searchKnn(const void *,size_t,hnswlib::BaseFilterFunctor *) const'
              with
              [
                  dist_t=float,
                  _Ty=std::pair<float,hnswlib::labeltype>
              ]
      ./python_bindings/bindings.cpp(875): note: see the first reference to 'hnswlib::BruteforceSearch<dist_t>::searchKnn' in 'BFIndex<float,float>::knnQuery_return_numpy::<lambda_f3fd84ed64eeede0115d45a3299fccbf>::operator ()'
              with
              [
                  dist_t=float
              ]
      C:\Users\luser\AppData\Local\Temp\pip-install-qmosbx4k\hnswlib_a35c5e0ef44041749b33fcb2a6c6dc9d\hnswlib\bruteforce.h(119): warning C4018: '<': signed/unsigned mismatch
      C:\Users\luser\AppData\Local\Temp\pip-install-qmosbx4k\hnswlib_a35c5e0ef44041749b33fcb2a6c6dc9d\hnswlib\hnswalg.h(1060): warning C4267: 'argument': conversion from 'size_t' to 'unsigned short', possible loss of data
      C:\Users\luser\AppData\Local\Temp\pip-install-qmosbx4k\hnswlib_a35c5e0ef44041749b33fcb2a6c6dc9d\hnswlib\hnswalg.h(1060): note: the template instantiation context (the oldest one first) is
      ./python_bindings/bindings.cpp(958): note: see reference to class template instantiation 'hnswlib::HierarchicalNSW<dist_t>' being compiled
              with
              [
                  dist_t=float
              ]
      C:\Users\luser\AppData\Local\Temp\pip-install-qmosbx4k\hnswlib_a35c5e0ef44041749b33fcb2a6c6dc9d\hnswlib\hnswalg.h(995): note: while compiling class template member function 'void hnswlib::HierarchicalNSW<dist_t>::updatePoint(const void *,hnswlib::tableint,float)'
              with
              [
                  dist_t=float
              ]
      C:\Users\luser\AppData\Local\Temp\pip-install-qmosbx4k\hnswlib_a35c5e0ef44041749b33fcb2a6c6dc9d\hnswlib\hnswalg.h(990): note: see the first reference to 'hnswlib::HierarchicalNSW<dist_t>::updatePoint' in 'hnswlib::HierarchicalNSW<dist_t>::addPoint'
              with
              [
                  dist_t=float
              ]
      C:\Users\luser\AppData\Local\Temp\pip-install-qmosbx4k\hnswlib_a35c5e0ef44041749b33fcb2a6c6dc9d\hnswlib\hnswalg.h(542): warning C4267: 'argument': conversion from 'size_t' to 'unsigned short', possible loss of data
      C:\Users\luser\AppData\Local\Temp\pip-install-qmosbx4k\hnswlib_a35c5e0ef44041749b33fcb2a6c6dc9d\hnswlib\hnswalg.h(542): note: the template instantiation context (the oldest one first) is
      C:\Users\luser\AppData\Local\Temp\pip-install-qmosbx4k\hnswlib_a35c5e0ef44041749b33fcb2a6c6dc9d\hnswlib\hnswalg.h(506): note: while compiling class template member function 'hnswlib::tableint hnswlib::HierarchicalNSW<dist_t>::mutuallyConnectNewElement(const void *,hnswlib::tableint,std::priority_queue<std::pair<dist_t,hnswlib::labeltype>,std::vector<std::pair<dist_t,hnswlib::labeltype>,std::allocator<std::pair<dist_t,hnswlib::labeltype>>>,hnswlib::HierarchicalNSW<dist_t>::CompareByFirst> &,int,bool)'
              with
              [
                  dist_t=float
              ]
      C:\Users\luser\AppData\Local\Temp\pip-install-qmosbx4k\hnswlib_a35c5e0ef44041749b33fcb2a6c6dc9d\hnswlib\hnswalg.h(1253): note: see the first reference to 'hnswlib::HierarchicalNSW<dist_t>::mutuallyConnectNewElement' in 'hnswlib::HierarchicalNSW<dist_t>::addPoint'
              with
              [
                  dist_t=float
              ]
      C:\Users\luser\AppData\Local\Temp\pip-install-qmosbx4k\hnswlib_a35c5e0ef44041749b33fcb2a6c6dc9d\hnswlib\hnswalg.h(962): note: see the first reference to 'hnswlib::HierarchicalNSW<dist_t>::addPoint' in 'hnswlib::HierarchicalNSW<dist_t>::addPoint'
              with
              [
                  dist_t=float
              ]
      C:\Users\luser\AppData\Local\Temp\pip-install-qmosbx4k\hnswlib_a35c5e0ef44041749b33fcb2a6c6dc9d\hnswlib\hnswalg.h(588): warning C4267: 'argument': conversion from 'size_t' to 'unsigned short', possible loss of data
      creating C:\Users\luser\AppData\Local\Temp\pip-install-qmosbx4k\hnswlib_a35c5e0ef44041749b33fcb2a6c6dc9d\build\lib.win32-cpython-311
      "C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.38.33130\bin\HostX86\x86\link.exe" /nologo /INCREMENTAL:NO /LTCG /DLL /MANIFEST:EMBED,ID=2 /MANIFESTUAC:NO /LIBPATH:C:\ChromaDB-Plugin-for-LM-Studio-2.7\src\libs "/LIBPATH:C:\Program Files\Python311\libs" "/LIBPATH:C:\Program Files\Python311" /LIBPATH:C:\ChromaDB-Plugin-for-LM-Studio-2.7\src\PCbuild\win32 "/LIBPATH:C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.38.33130\ATLMFC\lib\x86" "/LIBPATH:C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.38.33130\lib\x86" /LIBPATH:lib\10.0.22000.0\ucrt\x86 "/LIBPATH:C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.38.33130\ATLMFC\lib\x86" "/LIBPATH:C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.38.33130\lib\x86" "/LIBPATH:C:\Program Files (x86)\Windows Kits\NETFXSDK\4.8\lib\um\x86" "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\lib\10.0.22000.0\ucrt\x86" "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\\lib\10.0.22000.0\\um\x86" /EXPORT:PyInit_hnswlib build\temp.win32-cpython-311\Release\./python_bindings/bindings.obj /OUT:build\lib.win32-cpython-311\hnswlib.cp311-win_amd64.pyd /IMPLIB:build\temp.win32-cpython-311\Release\./python_bindings\hnswlib.cp311-win_amd64.lib
         Creating library build\temp.win32-cpython-311\Release\./python_bindings\hnswlib.cp311-win_amd64.lib and object build\temp.win32-cpython-311\Release\./python_bindings\hnswlib.cp311-win_amd64.exp
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyThread_tss_create
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyUnicode_AsEncodedString
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyBaseObject_Type
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyMem_Calloc
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyExc_BufferError
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyObject_SetAttr
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyCFunction_Type
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyWeakref_NewRef
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyThreadState_Get
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyLong_AsUnsignedLong
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyObject_Malloc
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyImport_ImportModule
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyBytes_AsString
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyErr_NormalizeException
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyErr_Occurred
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyLong_FromSsize_t
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyUnicode_DecodeUTF8
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyUnicode_AsUTF8AndSize
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyEval_SaveThread
      bindings.obj : error LNK2001: unresolved external symbol __imp___PyThreadState_UncheckedGet
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyExc_RuntimeError
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyObject_SetAttrString
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyLong_FromLong
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyObject_GC_UnTrack
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyInstanceMethod_Type
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyExc_MemoryError
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyDict_GetItemWithError
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyLong_FromUnsignedLongLong
      bindings.obj : error LNK2001: unresolved external symbol __imp__Py_GetVersion
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyCapsule_GetContext
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyDict_Contains
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyFrame_GetLineNumber
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyFloat_AsDouble
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyLong_AsUnsignedLongLong
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyGILState_Release
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyBytes_AsStringAndSize
      bindings.obj : error LNK2001: unresolved external symbol __imp___Py_NoneStruct
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyLong_AsLongLong
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyTuple_New
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyObject_GenericSetDict
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyDict_Size
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyLong_FromLongLong
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyFloat_FromDouble
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyList_Size
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyByteArray_AsString
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyErr_SetString
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyErr_WriteUnraisable
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyExc_ValueError
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyLong_FromUnsignedLong
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyDict_Next
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyErr_Format
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyDict_Type
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyThreadState_New
      bindings.obj : error LNK2001: unresolved external symbol __imp___Py_FalseStruct
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyFloat_Type
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyMethod_Type
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyType_IsSubtype
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyErr_Restore
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyExc_OverflowError
      bindings.obj : error LNK2001: unresolved external symbol __imp___Py_Dealloc
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyTuple_GetItem
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyCapsule_GetPointer
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyErr_Fetch
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyObject_CallFunctionObjArgs
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyLong_AsLong
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyObject_GenericGetDict
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyObject_ClearWeakRefs
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyUnicode_FromFormat
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyObject_GetBuffer
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyList_New
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyUnicode_DecodeUTF16
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyModule_Create2
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyType_Ready
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyObject_GetAttrString
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyErr_Clear
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyTuple_Size
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyThread_tss_get
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyException_SetTraceback
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyCapsule_New
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyCapsule_GetName
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyBytes_Size
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyDict_New
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyList_GetItem
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyException_SetContext
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyInstanceMethod_New
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyObject_IsInstance
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyUnicode_DecodeUTF32
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyMem_Free
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyTuple_SetItem
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyCMethod_New
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyThread_tss_set
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyObject_GetItem
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyObject_CallObject
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyObject_HasAttrString
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyProperty_Type
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyIndex_Check
      bindings.obj : error LNK2001: unresolved external symbol __imp___Py_NotImplementedStruct
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyByteArray_Size
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyGILState_Ensure
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyCapsule_SetContext
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyNumber_Check
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyNumber_Long
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyThread_tss_alloc
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyThreadState_DeleteCurrent
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyExc_TypeError
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyThreadState_Clear
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyCapsule_Type
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyEval_GetBuiltins
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyDict_Copy
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyObject_Str
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyUnicode_AsUTF8String
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyModule_Type
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyCallable_Check
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyFrame_GetBack
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyFrame_GetCode
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyExc_IndexError
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyExc_ImportError
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyCapsule_SetPointer
      bindings.obj : error LNK2001: unresolved external symbol __imp___Py_TrueStruct
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyExc_SystemError
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyObject_SetItem
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyException_SetCause
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyDict_DelItemString
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyUnicode_FromString
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyLong_FromSize_t
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyEval_AcquireThread
      bindings.obj : error LNK2001: unresolved external symbol __imp___PyType_Lookup
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyGILState_GetThisThreadState
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyBuffer_Release
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyObject_Repr
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyByteArray_Type
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyNumber_Float
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyType_Type
      bindings.obj : error LNK2001: unresolved external symbol __imp__PySequence_Tuple
      bindings.obj : error LNK2001: unresolved external symbol __imp__PyEval_RestoreThread
      bindings.obj : error LNK2001: unresolved external symbol __imp___PyObject_GetDictPtr
      build\lib.win32-cpython-311\hnswlib.cp311-win_amd64.pyd : fatal error LNK1120: 136 unresolved externals
      error: command 'C:\\Program Files\\Microsoft Visual Studio\\2022\\Community\\VC\\Tools\\MSVC\\14.38.33130\\bin\\HostX86\\x86\\link.exe' failed with exit code 1120
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for hnswlib
Failed to build hnswlib
ERROR: Could not build wheels for hnswlib, which is required to install pyproject.toml-based projects
TheRealBeef commented 11 months ago

A workaround to getting hnswlib to function is to do it with conda instead (found on another forum post somewhere)

e.g. (you can change the name of the virtual environment from chromadb to whatever you like of course)

conda create --name chromadb python=3.11
conda activate chromadb
conda install -c conda-forge hnswlib

and then the rest of the ChromaDB stuff can work normally

HammadB commented 11 months ago

Can you use the latest version of chroma (0.4.18)? It ships precompiled hnswlib in our own fork chroma-hnswlib.

On Sun, Dec 10, 2023 at 9:29 AM Beef @.***> wrote:

A workaround to getting hnswlib to function is to do it with conda instead (found on another forum post somewhere)

e.g. (you can change the name of the virtual environment from chromadb to whatever you like of course)

conda create --name chromadb python=3.11 conda activate chromadb conda install -c conda-forge hnswlib

and then the rest of the ChromaDB stuff can work normally

— Reply to this email directly, view it on GitHub https://github.com/chroma-core/chroma/issues/1485#issuecomment-1849028050, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABKW32OVLPAPDTHWAKDS6SLYIXWPLAVCNFSM6AAAAABAOCKFOCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNBZGAZDQMBVGA . You are receiving this because you are subscribed to this thread.Message ID: @.***>