facebookresearch / faiss

A library for efficient similarity search and clustering of dense vectors.
https://faiss.ai
MIT License
31.34k stars 3.64k forks source link

help : use python predict score is different use c++ api #2355

Closed xjj210130 closed 2 years ago

xjj210130 commented 2 years ago

Summary

hi all: m_ptrIndexFlatL2 = std::make_unique(m_dim); m_ncentroids = 100;//int(4 sqrt(m_total)); faiss::IndexIVFPQ m_ptrModel = new faiss::IndexIVFPQ(m_ptrIndexFlatL2.get(), 64, 100, 8, 8); m_ptrModel->train(m_total, m_ptrData.get()); m_ptrModel->add(m_total, m_ptrData.get());i build index as following:

search as following: std::unique_ptr pxI(new idx_t[nk]); idx_t I= pxI.get();

std::unique_ptr<float > pxb(new float[n*k]);
float*D = pxb.get();
if (nullptr != m_ptrModel)
{
    m_ptrModel->nprobe = 256;
    m_ptrModel->search(k, input.data(), n, D, I);
}

The python build index para is the same: quantizer = faiss.IndexFlatL2(dim) faiss.IndexIVFPQ(quantizer, 64, 100, 8, 8)

however, use the same data to build index.the result is different: python: id : 1453739546047295488 score: 0.22 c++ : id:"1453739546047295488" score:4.4158867e-05

Platform

OS: linux Faiss version:

Installed from:

Faiss compilation options:

Running on:

Interface:

The result is different.

xjj210130 commented 2 years ago

i use m_ptrModel->nprobe =1, or not use m_ptrModel->nprobe the score is also different

mdouze commented 2 years ago

The python code calls the C++ implementation, so I don't see how they could be different.

xjj210130 commented 2 years ago

Thanks.,maybe the different is The accuracy loss at that point is due to just the PQ compression. Analyzing accuracy issues with IndexIVFPQ