[Question]: ReadTimeoutError for HuggingFaceEmbedding

Question Validation

[X] I have searched both the documentation and discord for an answer.

Question

Hi there - I'm a bit new to all of this, but I was wondering if there was a simple answer for the error I am seeing.

This code:

from llama_index.embeddings.huggingface import HuggingFaceEmbedding

# loads BAAI/bge-small-en
# embed_model = HuggingFaceEmbedding()

# loads BAAI/bge-small-en-v1.5
embed_model = HuggingFaceEmbedding(model_name="BAAI/bge-small-en-v1.5")

Generates this error:

Traceback (most recent call last):
  File "/home/user/.local/share/virtualenvs/OpenDevin-SV2fu80l/lib/python3.11/site-packages/urllib3/connectionpool.py", line 537, in _make_request
    response = conn.getresponse()
               ^^^^^^^^^^^^^^^^^^
  File "/home/user/.local/share/virtualenvs/OpenDevin-SV2fu80l/lib/python3.11/site-packages/urllib3/connection.py", line 466, in getresponse
    httplib_response = super().getresponse()
                       ^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/.conda/envs/devin/lib/python3.11/http/client.py", line 1390, in getresponse
    response.begin()
  File "/home/user/.conda/envs/devin/lib/python3.11/http/client.py", line 325, in begin
    version, status, reason = self._read_status()
                              ^^^^^^^^^^^^^^^^^^^
  File "/home/user/.conda/envs/devin/lib/python3.11/http/client.py", line 286, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/.conda/envs/devin/lib/python3.11/socket.py", line 706, in readinto
    return self._sock.recv_into(b)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/.conda/envs/devin/lib/python3.11/ssl.py", line 1314, in recv_into
    return self.read(nbytes, buffer)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/.conda/envs/devin/lib/python3.11/ssl.py", line 1166, in read
    return self._sslobj.read(len, buffer)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TimeoutError: The read operation timed out

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/user/.local/share/virtualenvs/OpenDevin-SV2fu80l/lib/python3.11/site-packages/requests/adapters.py", line 486, in send
    resp = conn.urlopen(
           ^^^^^^^^^^^^^
  File "/home/user/.local/share/virtualenvs/OpenDevin-SV2fu80l/lib/python3.11/site-packages/urllib3/connectionpool.py", line 847, in urlopen
    retries = retries.increment(
              ^^^^^^^^^^^^^^^^^^
  File "/home/user/.local/share/virtualenvs/OpenDevin-SV2fu80l/lib/python3.11/site-packages/urllib3/util/retry.py", line 470, in increment
    raise reraise(type(error), error, _stacktrace)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/.local/share/virtualenvs/OpenDevin-SV2fu80l/lib/python3.11/site-packages/urllib3/util/util.py", line 39, in reraise
    raise value
  File "/home/user/.local/share/virtualenvs/OpenDevin-SV2fu80l/lib/python3.11/site-packages/urllib3/connectionpool.py", line 793, in urlopen
    response = self._make_request(
               ^^^^^^^^^^^^^^^^^^^
  File "/home/user/.local/share/virtualenvs/OpenDevin-SV2fu80l/lib/python3.11/site-packages/urllib3/connectionpool.py", line 539, in _make_request
    self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
  File "/home/user/.local/share/virtualenvs/OpenDevin-SV2fu80l/lib/python3.11/site-packages/urllib3/connectionpool.py", line 370, in _raise_timeout
    raise ReadTimeoutError(
urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='cdn-lfs.huggingface.co', port=443): Read timed out. (read timeout=10)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/user/1-test-huggingface-network.py", line 7, in <module>
    embed_model = HuggingFaceEmbedding(model_name="BAAI/bge-small-en-v1.5")
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/.local/share/virtualenvs/OpenDevin-SV2fu80l/lib/python3.11/site-packages/llama_index/embeddings/huggingface/base.py", line 86, in __init__
    self._model = SentenceTransformer(
                  ^^^^^^^^^^^^^^^^^^^^
  File "/home/user/.local/share/virtualenvs/OpenDevin-SV2fu80l/lib/python3.11/site-packages/sentence_transformers/SentenceTransformer.py", line 191, in __init__
    modules = self._load_sbert_model(
              ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/.local/share/virtualenvs/OpenDevin-SV2fu80l/lib/python3.11/site-packages/sentence_transformers/SentenceTransformer.py", line 1233, in _load_sbert_model
    module = Transformer(model_name_or_path, cache_dir=cache_folder, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/.local/share/virtualenvs/OpenDevin-SV2fu80l/lib/python3.11/site-packages/sentence_transformers/models/Transformer.py", line 36, in __init__
    self._load_model(model_name_or_path, config, cache_dir, **model_args)
  File "/home/user/.local/share/virtualenvs/OpenDevin-SV2fu80l/lib/python3.11/site-packages/sentence_transformers/models/Transformer.py", line 65, in _load_model
    self.auto_model = AutoModel.from_pretrained(
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/.local/share/virtualenvs/OpenDevin-SV2fu80l/lib/python3.11/site-packages/transformers/models/auto/auto_factory.py", line 563, in from_pretrained
    return model_class.from_pretrained(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/.local/share/virtualenvs/OpenDevin-SV2fu80l/lib/python3.11/site-packages/transformers/modeling_utils.py", line 3190, in from_pretrained
    resolved_archive_file = cached_file(pretrained_model_name_or_path, filename, **cached_file_kwargs)
                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/.local/share/virtualenvs/OpenDevin-SV2fu80l/lib/python3.11/site-packages/transformers/utils/hub.py", line 398, in cached_file
    resolved_file = hf_hub_download(
                    ^^^^^^^^^^^^^^^^
  File "/home/user/.local/share/virtualenvs/OpenDevin-SV2fu80l/lib/python3.11/site-packages/huggingface_hub/utils/_validators.py", line 119, in _inner_fn
    return fn(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^
  File "/home/user/.local/share/virtualenvs/OpenDevin-SV2fu80l/lib/python3.11/site-packages/huggingface_hub/file_download.py", line 1492, in hf_hub_download
    http_get(
  File "/home/user/.local/share/virtualenvs/OpenDevin-SV2fu80l/lib/python3.11/site-packages/huggingface_hub/file_download.py", line 456, in http_get
    r = _request_wrapper(
        ^^^^^^^^^^^^^^^^^
  File "/home/user/.local/share/virtualenvs/OpenDevin-SV2fu80l/lib/python3.11/site-packages/huggingface_hub/file_download.py", line 392, in _request_wrapper
    response = get_session().request(method=method, url=url, **params)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/.local/share/virtualenvs/OpenDevin-SV2fu80l/lib/python3.11/site-packages/requests/sessions.py", line 589, in request
    resp = self.send(prep, **send_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/.local/share/virtualenvs/OpenDevin-SV2fu80l/lib/python3.11/site-packages/requests/sessions.py", line 703, in send
    r = adapter.send(request, **kwargs)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/.local/share/virtualenvs/OpenDevin-SV2fu80l/lib/python3.11/site-packages/huggingface_hub/utils/_http.py", line 68, in send
    return super().send(request, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/.local/share/virtualenvs/OpenDevin-SV2fu80l/lib/python3.11/site-packages/requests/adapters.py", line 532, in send
    raise ReadTimeout(e, request=request)
requests.exceptions.ReadTimeout: (ReadTimeoutError("HTTPSConnectionPool(host='cdn-lfs.huggingface.co', port=443): Read timed out. (read timeout=10)"), '(Request ID: 43a49b3d-dbe0-4e6d-ac4d-38db18893de5)')

I ran the same code on a Collab instance and seemed to run fine. I could not see any obvious ways to increase the ReadTimeout. Any tips?

Thanks!

Hey there, @oculushut! Great to see you diving into the depths of LlamaIndex again. Running into a bit of a snag, are we? Let's see what we can do about that pesky ReadTimeoutError.

Based on the information you've provided, it seems like the ReadTimeoutError you're encountering is due to a network issue on your local setup. The HuggingFaceEmbedding class from the llama_index.embeddings.huggingface module, which you're using to load the model, does not directly expose a timeout parameter that you can adjust. This class uses the sentence_transformers library for embedding operations, which may internally use network operations to download models but does not expose direct control over network parameters like timeout.

However, there's a different class in the same module, HuggingFaceInferenceAPIEmbedding, which does allow adjusting the timeout parameter. This class uses the Hugging Face Inference API for network operations. Here's the relevant part of the code for reference:

timeout: Optional[float] = Field(
    default=None,
    description=(
        "The maximum number of seconds to wait for a response from the server."
        " Loading a new model in Inference API can take up to several minutes."
        " Defaults to None, meaning it will loop until the server is available."
    ),
)

This timeout parameter can be set during the initialization of a HuggingFaceInferenceAPIEmbedding instance. However, please note that this class is not the one you're currently using (HuggingFaceEmbedding), and switching to it might require changes to your code.

If you're only experiencing this issue on your local setup and not on Google Colab, it might be worth investigating if there are any network restrictions or issues that could be causing the timeout. This could include firewall settings, network congestion, or ISP-related issues.

If you continue to experience this issue, please provide more details about your local setup (e.g., operating system, network configuration, Python version, etc.) and any error messages or logs you have. This will help us better understand the issue and provide more targeted assistance.

You can find more information about the HuggingFaceEmbedding and HuggingFaceInferenceAPIEmbedding classes in the LlamaIndex repository.

Sources

#### About Dosu

This response is meant to be useful and save you time. It is **not meant to be a precise solution**, but rather a starting point for your own research. Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant. If you want to continue the conversation, start your reply with @dosu-bot.

run-llama / llama_index