nmslib / hnswlib

Header-only C++/python library for fast approximate nearest neighbors
https://github.com/nmslib/hnswlib
Apache License 2.0
4.11k stars 607 forks source link

segmentation fault (core dumped) #525

Closed duylncanawan closed 5 months ago

duylncanawan commented 7 months ago

I tried using this to create a file index for my project. here is my code: import hnswlib import numpy as np

Example vectors and paths

index_data_1 = np.random.rand(1280).astype('float32') index_path_1 = '/path_1'

Add items to the index

indexer = hnswlib.Index(space='cosine', dim=1280) indexer.add_items([index_data_1, index_path_1]) indexer.save_index('example_index.idx')

but the error occurs when indexer.add_items executes.

error: segmentation fault (core dumped)

i try python gdb to run and this is my error:

Thread 1 "python" received signal SIGSEGV, Segmentation fault. 0x00007ffff6e3ed67 in Index<float, float>::addItems(pybind11::object, pybind11::object, int, bool) () from /home/anaconda3/envs/flask/lib/python3.9/site-packages/hnswlib.cpython-39-x86_64-linux-gnu.so (gdb) bt

0 0x00007ffff6e3ed67 in Index<float, float>::addItems(pybind11::object, pybind11::object, int, bool) ()

from /home/anaconda3/envs/flask/lib/python3.9/site-packages/hnswlib.cpython-39-x86_64-linux-gnu.so

1 0x00007ffff6e2aa6a in pybind11::cpp_function::initialize<pybind11::cpp_function::initialize<void, Index<float, float>, pybind11::object, pybind11::object, int, bool, pybind11::name, pybind11::is_method, pybind11::sibling, pybind11::arg, pybind11::arg_v, pybind11::arg_v, pybind11::arg_v>(void (Index<float, float>::)(pybind11::object, pybind11::object, int, bool), pybind11::name const&, pybind11::is_method const&, pybind11::sibling const&, pybind11::arg const&, pybind11::arg_v const&, pybind11::arg_v const&, pybind11::arg_v const&)::{lambda(Index<float, float>, pybind11::object, pybind11::object, int, bool)#1}, void, Index<float, float>, pybind11::object, pybind11::object, int, bool, pybind11::name, pybind11::is_method, pybind11::sibling, pybind11::arg, pybind11::arg_v, pybind11::arg_v, pybind11::arg_v>(pybind11::cpp_function::initialize<void, Index<float, float>, pybind11::object, pybind11::object, int, bool, pybind11::name, pybind11::is_method, pybind11::sibling, pybind11::arg, pybind11::arg_v, pybind11::arg_v, pybind11::arg_v>(void (Index<float, float>::)(pybind11::object, pybind11::object, int, bool), pybind11::name const&, pybind11::is_method const&, pybind11::sibling const&, pybind11::arg const&, pybind11::arg_v const&, pybind11::arg_v const&, pybind11::arg_v const&)::{lambda(Index<float, float>, pybind11::object, pybind11::object, int, bool)#1}&&, void ()(Index<float, float>*, pybind11::object, pybind11::object, int, bool), pybind11::name const&, pybind11::is_method const&, pybind11::sibling const&, pybind11::arg const&, pybind11::arg_v const&, pybind11::arg_v const&, pybind11::arg_v const&)::{lambda(pybind11::detail::function_call&)#3}::_FUN(pybind11::detail::function_call) ()

from /home/anaconda3/envs/flask/lib/python3.9/site-packages/hnswlib.cpython-39-x86_64-linux-gnu.so

2 0x00007ffff6e25342 in pybind11::cpp_function::dispatcher(_object, _object, _object*) ()

from /home/anaconda3/envs/flask/lib/python3.9/site-packages/hnswlib.cpython-39-x86_64-linux-gnu.so

3 0x00005555556d68f4 in cfunction_call (func=0x7ffff6e6ce50, args=, kwargs=) at /opt/conda/conda-bld/python-split_1654083059479/work/Objects/methodobject.c:543

4 0x000055555569047f in _PyObject_MakeTpCall (tstate=0x55555591ec60, callable=0x7ffff6e6ce50, args=, nargs=, keywords=)

at /opt/conda/conda-bld/python-split_1654083059479/work/Objects/call.c:191

5 0x000055555560a20f in _PyObject_VectorcallTstate (kwnames=, nargsf=, args=0x55555597b240, callable=0x7ffff6e6ce50, tstate=0x55555591ec60)

at /opt/conda/conda-bld/python-split_1654083059479/work/Include/cpython/abstract.h:116

6 method_vectorcall (method=, args=0x55555597b248, nargsf=, kwnames=) at /opt/conda/conda-bld/python-split_1654083059479/work/Objects/classobject.c:53

7 0x0000555555653300 in _PyObject_VectorcallTstate (kwnames=0x0, nargsf=, args=0x55555597b248, callable=0x7ffff6e78180, tstate=)

so what is my problem? i use python 3.9

kerighan commented 7 months ago

same problem here

duylncanawan commented 5 months ago

same problem here

I found the reason, The reason is due to the lack of init_index function, you must init_index before adding_items to avoid this error. this way works for me. indexer = hnswlib.Index(space='cosine', dim=1280) indexer.init_index(max_elements = num_elements, ef_construction = 200, M = 16)