faiss fails to import on python==3.10.15

[yangcong356]

Summary

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/data1/users/yangcong/miniconda3/envs/ccvit/lib/python3.10/site-packages/faiss/__init__.py", line 16, in <module>
    from .loader import *
  File "/data1/users/yangcong/miniconda3/envs/ccvit/lib/python3.10/site-packages/faiss/loader.py", line 134, in <module>
    from .swigfaiss import *
  File "/data1/users/yangcong/miniconda3/envs/ccvit/lib/python3.10/site-packages/faiss/swigfaiss.py", line 13, in <module>
    from . import _swigfaiss
ImportError: /data1/users/yangcong/miniconda3/envs/ccvit/lib/python3.10/site-packages/faiss/../../../libfaiss.so: undefined symbol: __libc_single_threaded

Platform

OS: Ubuntu 20.04.6 LTS

Faiss version: 1.9.0

Installed from: anaconda

Faiss compilation options:

Running on:

• [ ] CPU
• [x] GPU

Interface:

• [ ] C++
• [x] Python

Reproduction instructions

import faiss

[KirinZzzz]

me too

[KirinZzzz]

I install faiss using: conda install pytorch::faiss-gpu and encounter the same error

[mnorris11]

Hi, for me to sanity check, can you paste the steps you used to install?

Following https://github.com/facebookresearch/faiss/wiki/Installing-Faiss, I didn't see the issue, but my conda env is already set up.

I tried also:

conda install conda-forge::faiss
python
>>> import faiss 

without errors

[yangcong356]

@mnorris11 Thank you so much for answering my question!

I followed your installation method and was able to import the "faiss" package successfully. However, when I run the program, I get an error that there is no "StandardGpuResources" module.

Subsequently, I was using the "conda install pytorch::faiss-gpu" command to install the faiss-gpu version, and when I imported the faiss package, I had the following problem:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/data1/users/yangcong/miniconda3/envs/ccvit/lib/python3.10/site-packages/faiss/__init__.py", line 16, in <module>
    from .loader import *
  File "/data1/users/yangcong/miniconda3/envs/ccvit/lib/python3.10/site-packages/faiss/loader.py", line 134, in <module>
    from .swigfaiss import *
  File "/data1/users/yangcong/miniconda3/envs/ccvit/lib/python3.10/site-packages/faiss/swigfaiss.py", line 13, in <module>
    from . import _swigfaiss
ImportError: /data1/users/yangcong/miniconda3/envs/ccvit/lib/python3.10/site-packages/faiss/../../../libfaiss.so: undefined symbol: __libc_single_threaded

[mnorris11]

I tried these steps and could not reproduce with a fresh conda env:

$ conda create -n faiss_3943

$ conda update -y -n base -c defaults conda

$ conda activate faiss_3943

$ conda install -y -q python=3.11 cmake make swig mkl=2023 mkl-devel=2023 numpy scipy pytest gxx_linux-64=11.2 sysroot_linux-64 gflags

$ conda install pytorch::faiss-gpu

$ python
Python 3.11.10 (main, Oct  3 2024, 07:29:13) [GCC 11.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import faiss
>>> 

This is on CentOS. This does not work for you?

[lichenmao]

@mnorris11 Thanks, I followed your steps, but I still meet the following problems. Driver Version: 525.147.05 CUDA Version: 12.0

>>> import faiss
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/chimao/anaconda3/envs/fgpu/lib/python3.11/site-packages/faiss/__init__.py", line 16, in <module>
    from .loader import *
  File "/home/chimao/anaconda3/envs/fgpu/lib/python3.11/site-packages/faiss/loader.py", line 134, in <module>
    from .swigfaiss import *
  File "/home/chimao/anaconda3/envs/fgpu/lib/python3.11/site-packages/faiss/swigfaiss.py", line 13, in <module>
    from . import _swigfaiss
ImportError: /home/chimao/anaconda3/envs/fgpu/lib/python3.11/site-packages/faiss/../../../libfaiss.so: undefined symbol: __libc_single_threaded

[mnorris11]

Can you paste your conda list -n fgpu and ldd --version? We can check the some versions.

[mnorris11]

FYI there is also this ongoing issue: https://github.com/facebookresearch/faiss/pull/3925#issuecomment-2423088307

[lichenmao]

Can you paste your conda list -n fgpu and ldd --version? We can check the some versions.

the ldd --version is 2.17.

[yangcong356]

@mnorris11 Thank you so much for your reply! I deleted the previous environment and then successfully installed version 1.8.0 of faiss-gpu by following the steps below:

conda create --name faiss_1.8.0
conda activate faiss_1.8.0
conda install -c pytorch -c nvidia faiss-gpu=1.8.0 pytorch=*=*cuda* pytorch-cuda=11 numpy

[mnorris11]

Cool, glad to hear it's installed. I will close the issue then.

[mnorris11]

We saw from other issues online that if you see this error about __libc_single_threaded, glibc may need to be upgraded on your system.

[EvenGu]

Hi , I am running into the same issue when I tried to import faiss

>>> import faiss
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/projectnb/dnn-motion/yiweng/conda/envs/mjepa/lib/python3.10/site-packages/faiss/__init__.py", line 16, in <module>
    from .loader import *
  File "/projectnb/dnn-motion/yiweng/conda/envs/mjepa/lib/python3.10/site-packages/faiss/loader.py", line 134, in <module>
    from .swigfaiss import *
  File "/projectnb/dnn-motion/yiweng/conda/envs/mjepa/lib/python3.10/site-packages/faiss/swigfaiss.py", line 13, in <module>
    from . import _swigfaiss
ImportError: /projectnb/dnn-motion/yiweng/conda/envs/mjepa/lib/python3.10/site-packages/faiss/../../../libfaiss.so: undefined symbol: __libc_single_threaded

I installed faiss-gpu via conda as instructed and my version is 1.9.0 My ldd --version is 2.28

I don't know if this is relevant but my pre-existing conda environment has torch installed via pip. ---- hope that I don't need to reinstall everything?

>>> python -c 'import torch; print(torch.__version__)'
2.5.1+cu124

My OS is AlmaLinux 8

Any advice will be appreciated! Thank you!!

[mnorris11]

Hi @EvenGu , if you run conda list, what is the gxx version? On my system right now, it looks like libgcc 14.2.0