[Bug]: Segmentation fault on MacOS when using HuggingFaceEmbedding with FAISS

[gordonhart]

Bug Description

Seeing segmentation fault when running a minimal script with HuggingFaceEmbedding and FAISS. A simple import faiss is enough to trigger the failure in a new environment with nothing but llama-index-embeddings-huggingface and faiss-cpu installed.

$ poetry run python3 example.py
[1]    63240 segmentation fault  poetry run python3 example.py

Contents of example.py:

import faiss
from llama_index.embeddings.huggingface import HuggingFaceEmbedding
embed = HuggingFaceEmbedding(model_name="BAAI/bge-small-en-v1.5")
print("success")

As far as I can tell, the specific model passed to HuggingFaceEmbedding does not have an impact.

Potentially related to #9949, though that issue does not appear to relate to FAISS.

Version

0.10.51

Steps to Reproduce

From a fresh directory:

1. poetry init with Python 3.12 and click through init menu
2. poetry add llama-index-embeddings-huggingface faiss-cpu
3. poetry run python3 example.py (see contents of this file above)

Output:

$ poetry run python3 example.py
[1]    63995 segmentation fault  poetry run python3 example.py

If you remove the import faiss line from example.py, the output is as expected:

$ poetry run python3 example.py
success

Relevant Logs/Tracebacks

With DEBUG logging enabled:

poetry run python3 example.py
DEBUG:faiss.loader:Environment variable FAISS_OPT_LEVEL is not set, so let's pick the instruction set according to the current CPU
Environment variable FAISS_OPT_LEVEL is not set, so let's pick the instruction set according to the current CPU
INFO:faiss.loader:Loading faiss.
Loading faiss.
INFO:faiss.loader:Successfully loaded faiss.
Successfully loaded faiss.
INFO:sentence_transformers.SentenceTransformer:Load pretrained SentenceTransformer: BAAI/bge-small-en-v1.5
Load pretrained SentenceTransformer: BAAI/bge-small-en-v1.5
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): huggingface.co:443
Starting new HTTPS connection (1): huggingface.co:443
DEBUG:urllib3.connectionpool:https://huggingface.co:443 "HEAD /BAAI/bge-small-en-v1.5/resolve/main/modules.json HTTP/11" 200 0
https://huggingface.co:443 "HEAD /BAAI/bge-small-en-v1.5/resolve/main/modules.json HTTP/11" 200 0
DEBUG:urllib3.connectionpool:https://huggingface.co:443 "HEAD /BAAI/bge-small-en-v1.5/resolve/main/config_sentence_transformers.json HTTP/11" 200 0
https://huggingface.co:443 "HEAD /BAAI/bge-small-en-v1.5/resolve/main/config_sentence_transformers.json HTTP/11" 200 0
DEBUG:urllib3.connectionpool:https://huggingface.co:443 "HEAD /BAAI/bge-small-en-v1.5/resolve/main/README.md HTTP/11" 200 0
https://huggingface.co:443 "HEAD /BAAI/bge-small-en-v1.5/resolve/main/README.md HTTP/11" 200 0
DEBUG:urllib3.connectionpool:https://huggingface.co:443 "HEAD /BAAI/bge-small-en-v1.5/resolve/main/modules.json HTTP/11" 200 0
https://huggingface.co:443 "HEAD /BAAI/bge-small-en-v1.5/resolve/main/modules.json HTTP/11" 200 0
DEBUG:urllib3.connectionpool:https://huggingface.co:443 "HEAD /BAAI/bge-small-en-v1.5/resolve/main/sentence_bert_config.json HTTP/11" 200 0
https://huggingface.co:443 "HEAD /BAAI/bge-small-en-v1.5/resolve/main/sentence_bert_config.json HTTP/11" 200 0
DEBUG:urllib3.connectionpool:https://huggingface.co:443 "HEAD /BAAI/bge-small-en-v1.5/resolve/main/config.json HTTP/11" 200 0
https://huggingface.co:443 "HEAD /BAAI/bge-small-en-v1.5/resolve/main/config.json HTTP/11" 200 0
[1]    64618 segmentation fault  poetry run python3 example.py

[dosubot[bot]]

Hey @gordonhart! 👋 I'm here to help you with any bugs, questions, or contributions you have in mind. If you need assistance, feel free to reach out to me.

To resolve the segmentation fault on MacOS when using HuggingFaceEmbedding with FAISS, you can try setting the device to cpu explicitly. This can help avoid issues related to GPU usage which might be causing the segmentation fault. Here is a minimal script to demonstrate this:

import faiss
from llama_index.legacy.embeddings.huggingface import HuggingFaceEmbedding

# Initialize the HuggingFaceEmbedding with device set to 'cpu'
embedding_model = HuggingFaceEmbedding(
    model_name="BAAI/bge-small-en-v1.5",
    device="cpu"
)

print("success")

In this script, the device parameter is explicitly set to "cpu" when initializing the HuggingFaceEmbedding class. This should help in avoiding segmentation faults related to GPU issues on MacOS ^[1].

To continue talking to Dosu, mention @dosu.

[gordonhart]

@dosu adding device="cpu" to the HuggingFaceEmbedding initialization did not fix the issue.

[dosubot[bot]]

If setting the device to "cpu" did not resolve the issue, you can try the following steps to address the segmentation fault:

1. Check Library Versions: Ensure you are using compatible versions of the faiss-cpu, torch, and transformers libraries. The recommended versions are:

   • faiss-cpu: 1.7.3
   • torch: 1.13.1
   • transformers: 4.25.1

   You can install these specific versions using the following commands:

   pip install faiss-cpu==1.7.3
   pip install torch==1.13.1
   pip install transformers==4.25.1

2. Update Dependencies: Ensure all your dependencies are up to date and compatible with each other. You can update your dependencies using:

   pip install --upgrade faiss-cpu torch transformers

3. Environment Testing: Try running your script in a different environment, such as a Linux-based system, to rule out any specific incompatibilities with your current MacOS setup.

4. Enable Debugging: Enable DEBUG logging level in your application to get more detailed error messages and tracebacks. This can provide more clues as to what might be causing the segmentation fault.

If the issue persists, you might want to check for any known compatibility issues between FAISS and the transformers library on MacOS. There is a documented issue related to FAISS on MacOS where a user encountered a TypeError when running the FAISS example notebook ^[1].

By following these steps, you should be able to identify and resolve the segmentation fault issue.

To continue talking to Dosu, mention @dosu.