Open leothomas opened 1 year ago
m = list(set(self.pool.map(lambda idx: idx.metric_type, self.sub_indexes)))
shouldn't it call idx.get_metric_type() ?
I'm not sure there is a get_metric_type()
function defined on the index class:
>>> idx.get_metric_type()
ServerException: File "/Users/leo/development-seed/similarity-search/similarity-search-deploy/env-similarity-search/lib/python3.8/site-packages/faiss/contrib/rpc.py", line 134, in one_function
ret = f(*args)
local variable 'f' referenced before assignment
Summary
Hey y'all, thanks for putting this library together, and taking a look at this question.
I've got an index split up across multiple servers and a client server that connects to the sub-indexes using the
socket
library, in order to execute searches on the distributed index (loosely following this demo).I want to be able to update/re-train the indexes with different parameters, especially switching between Cosine (inner product + normalization) and Euclidean (L2) distance metrics. While the client index, which executes the search, works the same way regardless of which distance metric is used, if the index I'm searching against uses the inner product search metric, I have to first normalize the search vectors.
I would like to access the sub indexes'
metric_type
parameter from the ClientIndex object, in order to know wether or not to normalize the search vectors before searching.Something like this (I've added a minimum reproducible example at the bottom):
In order to add a
metric_type
parameter to theClientIndex
class, I've updated theClientIndex
class with a method (get_metric_type()
) that queries themetric_type
parameter of each sub-index, validates that they're all the same and return the numerical value corresponding to the metric type.But the value returned by either:
idx.metric_type
oridx.metric_type()
is not an integer as expected:Platform
macOS Ventura 13.0.1 (22A400)
Faiss version:
faiss-cpu v1.7.2
Installed from: pip installed
Faiss compilation options: N/A
Running on:
Interface:
Reproduction instructions
Minimum reproducible example:
server.py
client.py
Note: for the sake of the MRE, I didn't train and add vectors to the index, but I am getting the same results with a trained and populated index
Thanks y'all!