search

HKUDS / LightRAG

"LightRAG: Simple and Fast Retrieval-Augmented Generation"
https://arxiv.org/abs/2410.05779
MIT License
6.05k stars 621 forks source link

OpenAI API Connection error when embedding #120

Open EvelynBai opened 2 days ago

EvelynBai commented 2 days ago

Hi, I'm encountering an API connection error when making an asynchronous call to the OpenAI API when embedding.

I have tested my API key and connection using a synchronous API call, and everything worked fine. The screenshot of the successful test is attached below:

截屏2024-10-24 下午5 51 47

Below is my code snippet for the asynchronous embedding call. (Note: I've removed the actual API key, but I have verified that it is correct):

` import os import asyncio from lightrag import LightRAG, QueryParam from lightrag.llm import openai_complete_if_cache, openai_embedding from lightrag.utils import EmbeddingFunc import numpy as np

WORKING_DIR = "./fin_demo"

if not os.path.exists(WORKING_DIR): os.mkdir(WORKING_DIR)

async def llm_model_func( prompt, system_prompt=None, history_messages=[], kwargs ) -> str: return await openai_complete_if_cache( "gpt-4o-mini-2024-07-18", prompt, system_prompt=system_prompt, history_messages=history_messages, api_key = "", base_url="https://api.openai.com/v1", kwargs, )

async def embedding_func(texts: list[str]) -> np.ndarray: return await openai_embedding( texts, model="text-embedding-3-small", api_key = "", base_url="https://api.openai.com/v1", )

async def test_funcs(): result = await llm_model_func("How are you?") print("llm_model_func: ", result)

result = await embedding_func(["How are you?"])
print("embedding_func: ", result)

asyncio.run(test_funcs())

rag = LightRAG( working_dir=WORKING_DIR, llm_model_func=llm_model_func, embedding_func=EmbeddingFunc( embedding_dim=4096, max_token_size=8192, func=embedding_func ), )

with open("./Fin_demo.txt") as f: rag.insert(f.read())

print( rag.query("What are the top themes?", param=QueryParam(mode="naive")) )

print( rag.query("What are the top themes?", param=QueryParam(mode="local")) )

print( rag.query("What are the top themes?", param=QueryParam(mode="global")) )

print( rag.query("What are the top themes?", param=QueryParam(mode="hybrid")) ) `

My log is as follows:

`(lightrag) ws_0802@node1:~/BYF/LightRAG/examples$ python lightrag_openai_compatible_demo.py INFO:httpx:HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 200 OK" llm_model_func: I'm just a computer program, so I don't have feelings, but I'm here and ready to help you! How can I assist you today? INFO:httpx:HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK" embedding_func: [[ 0.02756707 -0.0265443 -0.02709136 ... -0.01808866 -0.04026837 0.00252123]] INFO:lightrag:Logger initialized for working directory: ./fin_demo DEBUG:lightrag:LightRAG init with param: working_dir = ./fin_demo, chunk_token_size = 1200, chunk_overlap_token_size = 100, tiktoken_model_name = gpt-4o-mini, entity_extract_max_gleaning = 1, entity_summary_to_max_tokens = 500, node_embedding_algorithm = node2vec, node2vec_params = {'dimensions': 1536, 'num_walks': 10, 'walk_length': 40, 'window_size': 2, 'iterations': 3, 'random_seed': 3}, embedding_func = {'embedding_dim': 4096, 'max_token_size': 8192, 'func': <function embedding_func at 0x7ff354eda950>}, embedding_batch_num = 32, embedding_func_max_async = 16, llm_model_func = <function llm_model_func at 0x7ff354fb3d90>, llm_model_name = meta-llama/Llama-3.2-1B-Instruct, llm_model_max_token_size = 32768, llm_model_max_async = 16, key_string_value_json_storage_cls = <class 'lightrag.storage.JsonKVStorage'>, vector_db_storage_cls = <class 'lightrag.storage.NanoVectorDBStorage'>, vector_db_storage_cls_kwargs = {}, graph_storage_cls = <class 'lightrag.storage.NetworkXStorage'>, enable_llm_cache = True, addon_params = {}, convert_response_to_json_func = <function convert_response_to_json at 0x7ff298d14af0>

INFO:lightrag:Load KV full_docs with 0 data INFO:lightrag:Load KV text_chunks with 0 data INFO:lightrag:Load KV llm_response_cache with 0 data INFO:nano-vectordb:Init {'embedding_dim': 4096, 'metric': 'cosine', 'storage_file': './fin_demo/vdb_entities.json'} 0 data INFO:nano-vectordb:Init {'embedding_dim': 4096, 'metric': 'cosine', 'storage_file': './fin_demo/vdb_relationships.json'} 0 data INFO:nano-vectordb:Init {'embedding_dim': 4096, 'metric': 'cosine', 'storage_file': './fin_demo/vdb_chunks.json'} 0 data INFO:lightrag:Creating a new event loop in a sub-thread. INFO:lightrag:[New Docs] inserting 1 docs INFO:lightrag:[New Chunks] inserting 338 chunks INFO:lightrag:Inserting 338 vectors to chunks ERROR:asyncio:Task exception was never retrieved future: <Task finished name='Task-19' coro=<AsyncClient.aclose() done, defined at /home/ws_0802/anaconda3/envs/lightrag/lib/python3.10/site-packages/httpx/_client.py:2024> exception=RuntimeError('Event loop is closed')> Traceback (most recent call last): File "/home/ws_0802/anaconda3/envs/lightrag/lib/python3.10/site-packages/httpx/_client.py", line 2031, in aclose await self._transport.aclose() File "/home/ws_0802/anaconda3/envs/lightrag/lib/python3.10/site-packages/httpx/_transports/default.py", line 389, in aclose await self._pool.aclose() File "/home/ws_0802/anaconda3/envs/lightrag/lib/python3.10/site-packages/httpcore/_async/connection_pool.py", line 313, in aclose await self._close_connections(closing_connections) File "/home/ws_0802/anaconda3/envs/lightrag/lib/python3.10/site-packages/httpcore/_async/connection_pool.py", line 305, in _close_connections await connection.aclose() File "/home/ws_0802/anaconda3/envs/lightrag/lib/python3.10/site-packages/httpcore/_async/connection.py", line 171, in aclose await self._connection.aclose() File "/home/ws_0802/anaconda3/envs/lightrag/lib/python3.10/site-packages/httpcore/_async/http11.py", line 265, in aclose await self._network_stream.aclose() File "/home/ws_0802/anaconda3/envs/lightrag/lib/python3.10/site-packages/httpcore/_backends/anyio.py", line 55, in aclose await self._stream.aclose() File "/home/ws_0802/anaconda3/envs/lightrag/lib/python3.10/site-packages/anyio/streams/tls.py", line 201, in aclose await self.transport_stream.aclose() File "/home/ws_0802/anaconda3/envs/lightrag/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 1287, in aclose self._transport.close() File "/home/ws_0802/anaconda3/envs/lightrag/lib/python3.10/asyncio/selector_events.py", line 706, in close self._loop.call_soon(self._call_connection_lost, None) File "/home/ws_0802/anaconda3/envs/lightrag/lib/python3.10/asyncio/base_events.py", line 753, in call_soon self._check_closed() File "/home/ws_0802/anaconda3/envs/lightrag/lib/python3.10/asyncio/base_events.py", line 515, in _check_closed raise RuntimeError('Event loop is closed') RuntimeError: Event loop is closed ERROR:asyncio:Task exception was never retrieved future: <Task finished name='Task-20' coro=<AsyncClient.aclose() done, defined at /home/ws_0802/anaconda3/envs/lightrag/lib/python3.10/site-packages/httpx/_client.py:2024> exception=RuntimeError('Event loop is closed')> Traceback (most recent call last): File "/home/ws_0802/anaconda3/envs/lightrag/lib/python3.10/site-packages/httpx/_client.py", line 2031, in aclose await self._transport.aclose() File "/home/ws_0802/anaconda3/envs/lightrag/lib/python3.10/site-packages/httpx/_transports/default.py", line 389, in aclose await self._pool.aclose() File "/home/ws_0802/anaconda3/envs/lightrag/lib/python3.10/site-packages/httpcore/_async/connection_pool.py", line 313, in aclose await self._close_connections(closing_connections) File "/home/ws_0802/anaconda3/envs/lightrag/lib/python3.10/site-packages/httpcore/_async/connection_pool.py", line 305, in _close_connections await connection.aclose() File "/home/ws_0802/anaconda3/envs/lightrag/lib/python3.10/site-packages/httpcore/_async/connection.py", line 171, in aclose await self._connection.aclose() File "/home/ws_0802/anaconda3/envs/lightrag/lib/python3.10/site-packages/httpcore/_async/http11.py", line 265, in aclose await self._network_stream.aclose() File "/home/ws_0802/anaconda3/envs/lightrag/lib/python3.10/site-packages/httpcore/_backends/anyio.py", line 55, in aclose await self._stream.aclose() File "/home/ws_0802/anaconda3/envs/lightrag/lib/python3.10/site-packages/anyio/streams/tls.py", line 201, in aclose await self.transport_stream.aclose() File "/home/ws_0802/anaconda3/envs/lightrag/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 1287, in aclose self._transport.close() File "/home/ws_0802/anaconda3/envs/lightrag/lib/python3.10/asyncio/selector_events.py", line 706, in close self._loop.call_soon(self._call_connection_lost, None) File "/home/ws_0802/anaconda3/envs/lightrag/lib/python3.10/asyncio/base_events.py", line 753, in call_soon self._check_closed() File "/home/ws_0802/anaconda3/envs/lightrag/lib/python3.10/asyncio/base_events.py", line 515, in _check_closed raise RuntimeError('Event loop is closed') RuntimeError: Event loop is closed INFO:httpx:HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK" INFO:lightrag:Writing graph with 0 nodes, 0 edges Traceback (most recent call last): File "/home/ws_0802/anaconda3/envs/lightrag/lib/python3.10/site-packages/httpx/_transports/default.py", line 72, in map_httpcore_exceptions yield File "/home/ws_0802/anaconda3/envs/lightrag/lib/python3.10/site-packages/httpx/_transports/default.py", line 377, in handle_async_request resp = await self._pool.handle_async_request(req) File "/home/ws_0802/anaconda3/envs/lightrag/lib/python3.10/site-packages/httpcore/_async/connection_pool.py", line 216, in handle_async_request raise exc from None File "/home/ws_0802/anaconda3/envs/lightrag/lib/python3.10/site-packages/httpcore/_async/connection_pool.py", line 196, in handle_async_request response = await connection.handle_async_request( File "/home/ws_0802/anaconda3/envs/lightrag/lib/python3.10/site-packages/httpcore/_async/connection.py", line 99, in handle_async_request raise exc File "/home/ws_0802/anaconda3/envs/lightrag/lib/python3.10/site-packages/httpcore/_async/connection.py", line 76, in handle_async_request stream = await self._connect(request) File "/home/ws_0802/anaconda3/envs/lightrag/lib/python3.10/site-packages/httpcore/_async/connection.py", line 154, in _connect stream = await stream.start_tls(**kwargs) File "/home/ws_0802/anaconda3/envs/lightrag/lib/python3.10/site-packages/httpcore/_backends/anyio.py", line 69, in start_tls with map_exceptions(exc_map): File "/home/ws_0802/anaconda3/envs/lightrag/lib/python3.10/contextlib.py", line 153, in exit self.gen.throw(typ, value, traceback) File "/home/ws_0802/anaconda3/envs/lightrag/lib/python3.10/site-packages/httpcore/_exceptions.py", line 14, in map_exceptions raise to_exc(exc) from exc httpcore.ConnectError

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/home/ws_0802/anaconda3/envs/lightrag/lib/python3.10/site-packages/openai/_base_client.py", line 1571, in _request response = await self._client.send( File "/home/ws_0802/anaconda3/envs/lightrag/lib/python3.10/site-packages/httpx/_client.py", line 1674, in send response = await self._send_handling_auth( File "/home/ws_0802/anaconda3/envs/lightrag/lib/python3.10/site-packages/httpx/_client.py", line 1702, in _send_handling_auth response = await self._send_handling_redirects( File "/home/ws_0802/anaconda3/envs/lightrag/lib/python3.10/site-packages/httpx/_client.py", line 1739, in _send_handling_redirects response = await self._send_single_request(request) File "/home/ws_0802/anaconda3/envs/lightrag/lib/python3.10/site-packages/httpx/_client.py", line 1776, in _send_single_request response = await transport.handle_async_request(request) File "/home/ws_0802/anaconda3/envs/lightrag/lib/python3.10/site-packages/httpx/_transports/default.py", line 376, in handle_async_request with map_httpcore_exceptions(): File "/home/ws_0802/anaconda3/envs/lightrag/lib/python3.10/contextlib.py", line 153, in exit self.gen.throw(typ, value, traceback) File "/home/ws_0802/anaconda3/envs/lightrag/lib/python3.10/site-packages/httpx/_transports/default.py", line 89, in map_httpcore_exceptions raise mapped_exc(message) from exc httpx.ConnectError

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/home/ws_0802/anaconda3/envs/lightrag/lib/python3.10/site-packages/tenacity/asyncio/init.py", line 114, in call result = await fn(*args, **kwargs) File "/home/ws_0802/BYF/LightRAG/lightrag/llm.py", line 330, in openai_embedding response = await openai_async_client.embeddings.create( File "/home/ws_0802/anaconda3/envs/lightrag/lib/python3.10/site-packages/openai/resources/embeddings.py", line 236, in create return await self._post( File "/home/ws_0802/anaconda3/envs/lightrag/lib/python3.10/site-packages/openai/_base_client.py", line 1838, in post return await self.request(cast_to, opts, stream=stream, stream_cls=stream_cls) File "/home/ws_0802/anaconda3/envs/lightrag/lib/python3.10/site-packages/openai/_base_client.py", line 1532, in request return await self._request( File "/home/ws_0802/anaconda3/envs/lightrag/lib/python3.10/site-packages/openai/_base_client.py", line 1605, in _request raise APIConnectionError(request=request) from err openai.APIConnectionError: Connection error.

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/home/ws_0802/BYF/LightRAG/examples/lightrag_openai_compatible_demo.py", line 61, in rag.insert(f.read()) File "/home/ws_0802/BYF/LightRAG/lightrag/lightrag.py", line 162, in insert return loop.run_until_complete(self.ainsert(string_or_strings)) File "/home/ws_0802/anaconda3/envs/lightrag/lib/python3.10/asyncio/base_events.py", line 649, in run_until_complete return future.result() File "/home/ws_0802/BYF/LightRAG/lightrag/lightrag.py", line 206, in ainsert await self.chunks_vdb.upsert(inserting_chunks) File "/home/ws_0802/BYF/LightRAG/lightrag/storage.py", line 92, in upsert embeddings_list = await asyncio.gather( File "/home/ws_0802/BYF/LightRAG/lightrag/utils.py", line 87, in wait_func result = await func(*args, kwargs) File "/home/ws_0802/BYF/LightRAG/lightrag/utils.py", line 43, in call return await self.func(*args, *kwargs) File "/home/ws_0802/BYF/LightRAG/examples/lightrag_openai_compatible_demo.py", line 30, in embedding_func return await openai_embedding( File "/home/ws_0802/BYF/LightRAG/lightrag/utils.py", line 43, in call return await self.func(args, kwargs) File "/home/ws_0802/anaconda3/envs/lightrag/lib/python3.10/site-packages/tenacity/asyncio/init.py", line 189, in async_wrapped return await copy(fn, *args, *kwargs) File "/home/ws_0802/anaconda3/envs/lightrag/lib/python3.10/site-packages/tenacity/asyncio/init.py", line 111, in call do = await self.iter(retry_state=retry_state) File "/home/ws_0802/anaconda3/envs/lightrag/lib/python3.10/site-packages/tenacity/asyncio/init.py", line 153, in iter result = await action(retry_state) File "/home/ws_0802/anaconda3/envs/lightrag/lib/python3.10/site-packages/tenacity/_utils.py", line 99, in inner return call(args, **kwargs) File "/home/ws_0802/anaconda3/envs/lightrag/lib/python3.10/site-packages/tenacity/init.py", line 419, in exc_check raise retry_exc from fut.exception() tenacity.RetryError: RetryError[<Future at 0x7ff2875f4070 state=finished raised APIConnectionError>]`

LarFii commented 1 day ago

Based on the logs, it appears that embedding works initially, but after several calls, an error occurs. This might be related to an issue with the account.