search

facebookresearch / faiss

A library for efficient similarity search and clustering of dense vectors.
https://faiss.ai
MIT License
30.58k stars 3.57k forks source link

a plugin for faiss #3874

Open alexanderguzhva opened 1 day ago

alexanderguzhva commented 1 day ago

I'm finalizing a separate library for speeding up training of ProductQuantizer and ProductResidualQuantizer, which is written in C++ and can be easily integrated in C++ code via faiss::Index interfaces.

Does Faiss contain any examples of providing a plugin-style code for it's python wrapper, so it would be possible to inject the library into a python faiss as well? Or, maybe, to write a swig-wrapper for the library.

@mdouze

mdouze commented 22 hours ago

Not sure what you need, but basically there are two approaches to this:

  1. python callbacks that allow python to be called from C++ here https://github.com/facebookresearch/faiss/blob/main/faiss/python/python_callbacks.h
  2. independent SWIG modules that provide sub-classes of existing C++ objects wrapped in python, see https://github.com/facebookresearch/faiss/wiki/Python-C---code-snippets#wrapping-small-c-objects-for-use-from-python unfortunately this second option does not always work smoothly, so there is an open diff / PR to add tests to catch regressions, see https://github.com/facebookresearch/faiss/pull/3699