Installation of _swigfaiss_avx2.so seems to depend on installation of C++ headers and libraries

[Janosch]

Summary

I am compiling FAISS while building an alpine docker image. If I follow the installation steps from INSTALL.md, swigfaiss_avx2.py and _swigfaiss_avx2.so are missing in pythons site-packages/faiss-1.8.0-py3.12.egg/faiss.

If I install the C++ headers and libraries before installing the python package, the AVX2-enabled python module and shared library are copied to site-packages/faiss-1.8.0-py3.12.egg/faiss as expected.

If there is a dependency between step 3 and step 4, it should be reflected in the INSTALL.md.

Platform

OS: alpine:3.20 linux/amd64

Faiss version: 1f42e81

Installed from: Compiled

Faiss compilation options: FAISS_ENABLE_GPU=OFF BUILD_TESTING=OFF CMAKE_BUILD_TYPE=Release FAISS_OPT_LEVEL=avx2

Running on:

• [x] CPU
• [ ] GPU

Interface:

• [x] C++
• [x] Python

Reproduction instructions

1. Create docker image using this Dockerfile
2. docker build --platform linux/amd64 -t missing-avx2 .
3. docker run -it missing-avx2 sh
4. ls -lah /usr/lib/python3.12/site-packages/faiss-1.8.0-py3.12.egg/faiss
5. Verify that swigfaiss_avx2.py is missing

For comparison, see this Dockerfile which reverses steps 3 and 4 from the installation instructions and therefore works.

[alexanderguzhva]

@Janosch this was the default behavior all the time, as far as I remember. You compile faiss and then swigfaiss

[Janosch]

@alexanderguzhva The example of the failing Dockerfile was badly constructed, my bad. Of course, you need to build faiss before building swigfaiss bindings. My point is, that if you follow the steps in the INSTALL.md you end up without AVX2 support. I created a new Dockerfile as example.

What does not work:

1. Build faiss
2. Build swigfaiss
3. Install faiss

This, however, is working:

1. Build faiss
2. Install faiss
3. Build swigfaiss

Unless something is wrong with my build environment, it looks like AVX2 is only present in the python package if faiss has been build and installed on the system before building swigfaiss. If faiss is only built but not installed, the python package is available but without AVX2 support.

[alexanderguzhva]

@Janosch I always build make faiss, then make faiss_avx2 and then make faiss_avx512, same for swig. As far as I understand, make install is what builds all three plain+avx2+avx512 faiss versions.

I agree that there is a deficiency here.

[Janosch]

@alexanderguzhva Ok. What confused me is that setting FAISS_OPT_LEVEL=avx2 alone is not enough. One also needs to build the correct target, in my case faiss_avx2. I think a brief remark in INSTALL.md in Step 3 would be helpful.