search

milvus-io / knowhere

Knowhere is an open-source vector search engine, integrating FAISS, HNSW, etc.
Apache License 2.0
201 stars 81 forks source link

Discussion: do we need to keep support SSE42 as minimal requirements for running knowhere #800

Open matrixji opened 1 year ago

matrixji commented 1 year ago

I'm integrating embd-milvus with 2.3 branch for the macos, it failed while run test with pymilvus, after a lit bit of investigation, the root cause is Knowhere compiled with avx2+ support: see here and macOS machine on GitHub action is a vm only support sse42. So it crashed with SIGILL, more logs could be fetched here.

SIGILL: illegal instruction
PC=0x10758d966 m=10 sigcode=1
signal arrived during cgo execution
instruction bytes: 0xc4 0xe2 0xf1 0xb9 0x5 0x29 0x83 0x4 0x0 0x48 0x83 0xc4 0x10 0x5d 0xc3 0x66

goroutine 993 [syscall]:
_ZN5faiss12getmillisecsEv
    /Users/runner/.milvus.io/milvus-server/2.3.0-beta.1/bin/libknowhere.dylib:0 pc=0x10758d967
_ZN5faiss10Clustering13train_encodedExPKhPKNS_5IndexERS3_PKf

If we still need to support Milvus run with sse42 as minimal instruction sets, this is should be fixed. Otherwise, If we mark avx2 as minimal supported instructions, I'll move those related verify works offline.

xiaofan-luan commented 1 year ago

yes I think so, we should still go with SSE support. @Presburger could you pls help on fixing it