[Bug]: Incompatibility between Milvus 2.3.6 and 2.4

[akcopy]

Is there an existing issue for this?

• [X] I have searched the existing issues

Environment

- Milvus version: milvusdb/milvus:2.4-latest-gpu
- Deployment mode(standalone or cluster): standalone
- MQ type(rocksmq, pulsar or kafka): n/a
- SDK version(e.g. pymilvus v2.0.0rc2): pymilvus 2.4.1
- OS(Ubuntu or CentOS): Ubuntu
- CPU/Memory: CPU - AMD EPYC 32 core, memory = 256GB RAM
- GPU: Nvidia RTX 3090 
- Others:

Current Behavior

I created a SCANN database in Milvus 2.3.6 which always worked. Now when I upgraded to Milvus 2.4.1, my queries with LangChain no longer work. I tested using the MilvusClient method (bypassing Langchain) and it still gave the exact same error. Langchain method: vector_db_custom = Milvus(embedding_function=embeddings,connection_args={"host":milvushost,"port":milvusport,"user":milvus_username,"password":milvus_password},collection_name=collectionName,search_params={"metric":"IP"}) params = {"nprobe":16384,"reorder_k":512} topk = 100 relevant_docs = vector_db_custom.similarity_search_with_score(query, k=topk,param=params)

MilvusClient method: searchresults = client.search(collection_name=collectionName,data=[query_vector],search_params={'metric_type':'IP','params':{"nprobe":16384,"reorder_k":500}},limit=100)

both comes up with an error like this: pymilvus.exceptions.MilvusException: <MilvusException: (code=65535, message=fail to search on QueryNode 105: worker(105) query failed: => failed to search: faiss inner error: Error in virtual void faiss::IndexRefineFlat::search(faiss::idx_t, const float, faiss::idx_t, float, faiss::idx_t, const faiss::SearchParameters) const at /go/src/github.com/milvus-io/milvus/cmake_build/thirdparty/knowhere/knowhere-src/thirdparty/faiss/faiss/IndexRefine.cpp:268: Error: '!(params)' failed: IndexRefineFlat params have incorrect type)>

What exactly changed between 2.3.6 and 2.4.1 in the syntax?

Expected Behavior

i expected this to work as per the syntax in 2.3.6.

Steps To Reproduce

No response

Milvus Log

No response

Anything else?

No response

[xiaofan-luan]

/assign @liliu-z

[chasingegg]

@alexanderguzhva could you take a look

[liliu-z]

Cannot reproduce and didn't see any path calling IndexRefineFlat::search. One of the possible explanation is the binary get polluted so faiss load it as IndexRefineFlat instead of IndexSCANN.

Did you try rollback the Milvus version? @akcopy