search

xyzhang626 / embeddings.cpp

ggml implementation of embedding models including SentenceTransformer and BGE
MIT License
52 stars 3 forks source link

Illegal instruction (core dumped) #4

Open Mihaiii opened 8 months ago

Mihaiii commented 8 months ago

I'm following the examples from the readme. When I run:

python download-repo.py BAAI/bge-base-en-v1.5 # or any other model
sh run_conversions.sh bge-base-en-v1.5

It fails at the quantization step:

[...]
Done. Output file: bge-base-en-v1.5/ggml-model-f16.bin

Illegal instruction (core dumped)
Illegal instruction (core dumped)

Is there something I can do to make it work?

xyzhang626 commented 8 months ago

sorry for the late reply. What's your device? Or could you debug with it to find out what the specific step to make the core dumped?

colintoal commented 6 months ago

I am getting a crash at f16cintrin.h:41 - I suspect it might be that my old core i5 doesn't support f16c -- but I disabled it in the make, and I think it still crashed.

colintoal commented 6 months ago

fresh build, disabled everything, F16C support everywhere.. still crashing on: _v4sf A = builtin__ia32_vcvtph2ps ( __H);

Which is line 41 of ../x86_64-redhat-linux/12/include/f16cintrin.h

So is F16C support still turned on even though it is off when I configured the build ?

Seems like this version of GGML always enables F16C when its not built with MSVC - regardless of whether F16C support is enabled in embedding.cpp CMake.. I edited the CMakeLists config for the included GGML submodule and disabled F16C support, and everything appears to work. Apologies for having an ancient CPU that I am testing on.

xyzhang626 commented 6 months ago

thanks @colintoal for debugging it. Honestly the GGML version in this repo is pretty old. Part of effort in this repo has been integrated in the llama.cpp. Might be helpful to check here.