使用HuggingFaceInferenceAPIEmbedding无法连接到api接口

zhongwentao commented 8 months ago

Question Validation

[X] I have searched both the documentation and discord for an answer.

Question

我开启代理后，用postman请求api接口是通的，但是用HuggingFaceInferenceAPIEmbedding去调就提示无法连接。代码如下：

import unittest
import os
import logging
import sys

from llama_index.core import VectorStoreIndex, SimpleDirectoryReader, Settings, StorageContext, load_index_from_storage
from llama_index.legacy.embeddings import HuggingFaceInferenceAPIEmbedding
from llama_index.llms.huggingface import HuggingFaceInferenceAPI

token = "*****"

# 日志设置
logging.basicConfig(stream=sys.stdout, level=logging.DEBUG)
logging.getLogger().addHandler(logging.StreamHandler(stream=sys.stdout))

Settings.llm = HuggingFaceInferenceAPI(model_name="Qwen/Qwen1.5-14B-Chat", token=token)
Settings.embed_model = HuggingFaceInferenceAPIEmbedding(model_name="sentence-transformers/all-MiniLM-L6-v2", token=token)
print(Settings.embed_model)

PERSIST_DIR = "../storage"
if not os.path.exists(PERSIST_DIR):
    documents = SimpleDirectoryReader('../docs/rag').load_data()
    index = VectorStoreIndex.from_documents(documents)
    index.storage_context.persist(persist_dir=PERSIST_DIR)
else:
    storage_context = StorageContext.from_defaults(persist_dir=PERSIST_DIR)
    index = load_index_from_storage(storage_context)

query_engine = index.as_query_engine()
resp = query_engine.query('请介绍下文档大致内容')

print(resp)

并且我直接用request去请求接口也是能通的，谁能帮我看看，是我用法有问题吗？

dosubot[bot] commented 8 months ago

🤖

Based on the code you provided and the context from the LlamaIndex repository, it seems like the issue might be related to the proxy settings for the API connection. When you use Postman with the proxy turned on, Postman is configured to route its requests through the proxy, which is not automatically the case for your Python code.

In the provided code snippet, there is no mention or handling of proxy configurations for the HuggingFaceInferenceAPIEmbedding class or its parent classes. This means that if your network environment requires a proxy to access external APIs, your code will not be able to connect to the Hugging Face API unless it is explicitly configured to use the proxy.

To resolve this issue, you would need to modify the code to ensure that HTTP requests made by the HuggingFaceInferenceAPIEmbedding class (or its underlying HTTP client) are configured to use the necessary proxy. This typically involves setting environment variables (HTTP_PROXY and HTTPS_PROXY) or configuring the HTTP client library you're using (e.g., requests, httpx) to use a proxy.

If you're using the requests library, for example, you can configure it to use a proxy by setting the proxies parameter in your request:

import requests

proxies = {
    'http': 'http://10.10.1.10:3128',
    'https': 'http://10.10.1.10:1080',
}

response = requests.get('http://example.com', proxies=proxies)

However, since your code is likely using an asynchronous HTTP client (given the asynchronous nature of the embedding methods), you'll need to check the documentation for the specific HTTP client library you're using to see how to configure it to use a proxy.

If the Hugging Face Inference API client is based on httpx, configuring a proxy might look like this:

import httpx

proxies = {
    'http://': 'http://10.10.1.10:3128',
    'https':': 'http://10.10.1.10:1080',
}

async with httpx.AsyncClient(proxies=proxies) as client:
    response = await client.get('http://example.com')

You would need to integrate proxy configuration into the part of your code that makes the HTTP requests to the Hugging Face API. This might involve modifying the HuggingFaceInferenceAPI class or wherever the HTTP client is instantiated and used.

For more details, you can refer to the source code and this issue in the LlamaIndex repository.

I hope this helps! If you have any other questions or need further clarification, feel free to ask.

Sources

#### About Dosu

This response is meant to be useful and save you time. It is **not meant to be a precise solution**, but rather a starting point for your own research. Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant. If you want to continue the conversation, start your reply with @dosu-bot.

logan-markewich commented 8 months ago

Try setting the task type

HuggingFaceInferenceAPIEmbedding(..., task="feature-extraction")

zhongwentao commented 8 months ago

试过了没有用。我跟了下代码用request请求hugging api接口时，它会去获取全局的代理，而HuggingFaceInferenceAPIEmbedding我不知道要怎么把代理配置传给它

logan-markewich commented 8 months ago

Hmm, I don't think the embedding class supports a proxy right now? 🤔

run-llama / llama_index