Bug in langchain_community/embeddings/huggingface_hub.py HuggingFaceHubEmbeddings.embed_query

jrabyIBM commented 10 months ago

Checked other resources

[X] I added a very descriptive title to this issue.
[X] I searched the LangChain documentation with the integrated search.
[X] I used the GitHub search to find a similar question and didn't find it.

Example Code

from langchain_community.embeddings import HuggingFaceHubEmbeddings url = "https://svvwc5yh51gt1pp3.us-east-1.aws.endpoints.huggingface.cloud" text = "You do not need a weatherman to know which way the wind blows" token = ""

embeddings = HuggingFaceHubEmbeddings( model = url, huggingfacehub_api_token=token ) qembed = embeddings.embed_query(text)

Description

The code above should call my HuggingFace inference endpoint and generate an embedding. Instead, it throws:

Traceback (most recent call last): File "/Users/jraby/python/hfhub/callhub2.py", line 29, in qembed = embeddings.embed_query(text) File "/opt/homebrew/lib/python3.10/site-packages/langchain_community/embeddings/huggingface_hub.py", line 109, in embed_query response = self.embed_documents([text])[0] KeyError: 0

The "[0]" at the end of line 109 is incorrect. self.embed_documents returns JSON, -not a list. The first node in the JSON is 'embeddings', and it contains the list of embedding arrays.

I believe that this line: response = self.embed_documents([text])[0] Should be replaced with this: response = self.embed_documents([text])['embeddings'][0]

This resolves the problem for me.

System Info

Platform = MacOS python version = 3.10 pip3 huggingface_hub (0.20.2)

Related Components

[ ] LLMs/Chat Models
[X] Embedding Models
[ ] Prompts / Prompt Templates / Prompt Selectors
[ ] Output Parsers
[ ] Document Loaders
[ ] Vector Stores / Retrievers
[ ] Memory
[ ] Agents / Agent Executors
[ ] Tools / Toolkits
[ ] Chains
[ ] Callbacks/Tracing
[ ] Async

dosubot[bot] commented 10 months ago

🤖

Thank you for bringing this issue to our attention. It's great to see your detailed analysis of the problem and the proposed fix. Your contribution to LangChain would be highly valuable.

I encourage you to go ahead and create a pull request with your proposed fix. The maintainers will review it and decide if the change should be merged. Your proactive approach to problem-solving is appreciated, and your contribution will help improve LangChain for all users. Keep up the good work!

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

tempdeltavalue commented 10 months ago

@jrabyIBM

Did you run it on you machine without amazon endpoint locally ?

Sorry, a bit off-topic, but I'm also facing some difficulties when running HF with Langchain on pretty similar task. It seems they are more focused on collaborating with OpenAI than on local inference.

fyi, my issue

jrabyIBM commented 10 months ago

@tempdeltavalue no, I have not run it against a local model. I've just update the comment above. The fix should be to replace:

response = self.embed_documents([text])[0] with this: response = self.embed_documents([text])['embeddings'][0]

in langchain_community/embeddings/huggingface_hub.py. Making this change resolves the problem for me.

keenborder786 commented 10 months ago

@jrabyIBM Did you try SelfHostedHuggingFaceEmbeddings in langchain_community/embeddings/self_hosted_hugging_face.py. That might fix your issue.

jrabyIBM commented 10 months ago

@keenborder786 Thanks for the suggestion. However this is clearly a bug, which prevents use of the HuggingFaceHubEmbeddings class. It may be that some other langchain classes are not affected by this bug, but that doesn't change the fact that HuggingFaceHubEmbeddings is broken.

The embed_query method calls the embed_documents method, which returns JSON like this:


{
     'embeddings': 
         [
             [0.08688841015100479, 
              ...
             ]
         ]
 }

The current code then calls: response = self.embed_documents([text])[0]

The line of code throws a KeyError. It is incorrect code for processing the JSON that is returned. The correct code should be: response = self.embed_documents([text])['embeddings'][0]

I need to use HuggingFaceHubEmbeddings to use a HuggingFace inference endpoint.

keenborder786 commented 10 months ago

@javi-aranda the following code is working fine for me, if their was a bug then the following should also have not worked.


from langchain_community.embeddings import HuggingFaceHubEmbeddings
text = "You do not need a weatherman to know which way the wind blows"
embeddings = HuggingFaceHubEmbeddings(
model = 'TinyLlama/TinyLlama-1.1B-Chat-v1.0',
huggingfacehub_api_token=''
)
qembed = embeddings.embed_query(text)

jrabyIBM commented 10 months ago

@keenborder786 that is very similar to the code that throws a KeyError exception for me. I think that the difference is that you are specifying an actual model (which I think means that you are accessing a self-hosted local model?) whereas I am specifying a URL in the model field, which tells langchain that this is a HuggingFace hosted inference model.

I think your experiment is helpful. It clarifies that the bug only occurs when accessing a remote, HuggingFace inference endpoint via HuggingFaceHubEmbeddings.

langchain-ai / langchain