Distance vs. similarity confusion in faiss

[salmanmashayekh]

search method of a faiss index returns L2 distances, rather than similarities. But the following line in pack_single_result assumes similarities:

results = [(self.index["sentences"][i], s) for i, s in zip(idx, dist) if s >= threshold]

I think it should change to the following:

results = [(self.index["sentences"][i], d) for i, d in zip(idx, dist) if d <= threshold]

Note the <= vs >= in the list comprehension.

Alternatively, we can change the metric_type to use cosine similarity instead of L2 distance:

index.metric_type = faiss.METRIC_INNER_PRODUCT

[gaotianyu1350]

Hi thanks for reporting this! It only occurs when using fast faiss and now this problem has been fixed!