dosubot[bot]
commented
8 months ago Hey there, @whxleemdddd! Great to see you back 😊. How's everything going besides this pesky issue?
Based on the information you provided, the openai.APIConnectionError you're encountering could be due to several reasons. Here are the most likely causes:
-
Network Issues: Ensure your machine has a stable internet connection. The error might indicate that your script cannot reach the OpenAI API servers, possibly due to network connectivity issues on your end.
-
Incorrect API Keys or Configuration: The script includes an argument for
--api-keys, which suggests it requires API keys to authenticate requests to an API server. If these keys are missing, incorrect, or have not been properly passed to the script, it would result in connection errors. Double-check that you have correctly set the API keys and that they are valid. -
Server Configuration Issues: The script is designed to launch several services (controller, worker, openai_api_server) and relies on specific ports and hostnames (
localhostin this case). If there's an issue with the server configuration, such as ports being blocked or already in use, or if the services do not start up correctly, it could lead to connection errors when trying to communicate between the components. -
Local Environment Setup: Since you're using an M1 Mac, ensure that all dependencies and the Python environment are correctly set up for ARM64 architecture. Compatibility issues could lead to unexpected errors. Specifically, check that all Python packages and dependencies installed via pip or conda are compatible with ARM64.
-
Firewall or Security Software: Sometimes, firewall settings or security software on your Mac might block outgoing connections to certain ports or domains. Check your firewall settings and ensure that your security software allows the script to make outbound connections to the OpenAI API servers.
-
Incorrect Use of the
--deviceArgument: You mentioned using an M1 Mac, but the script has a--deviceargument set tocudaby default. M1 Macs do not support CUDA because it's NVIDIA-specific. You should change this tomps(Apple's Metal Performance Shaders) for GPU acceleration on M1 chips orcpuif you're not using GPU acceleration.
To diagnose the issue further, check the logs located in the ./logs/ directory as mentioned in your script. These logs might contain more specific error messages that can help pinpoint the exact cause of the APIConnectionError.
Regarding your question about the openai_proxy field in the model_config.py file, it is designed to specify a proxy server for routing requests to the OpenAI API. This can be useful in scenarios where direct access to the OpenAI API is restricted or when you want to route traffic through a specific network path for reasons such as monitoring, logging, or bypassing geo-restrictions. If you're using a proxy server, ensure that the openai_proxy field is correctly set.
I also found some similar issues in the Langchain-Chatchat repository that might be helpful. You can refer to these issues for more information and potential solutions:
If none of the above solutions work, please provide more information about your network environment and the exact steps you're taking when the error occurs to help diagnose the issue.
Sources
- 我在model_config.py里配置了openai的key，但是对话界面选择openai-api进行聊天的时候报错，但是我已经开了国外的TZ了啊，请问这是为何
- 基于OpenAI的GPT接口模型，启动api.py后，使用chat/chat接口时报连接错误
- [BUG] 运行之后，对话不反应，api.py报错，APIConnectionError、我看好多人遇到相同的问题，都没找到解决方法吗
- server/llm_api_stale.py
- configs/model_config.py.example
WoNeiDieA
zRzRzRzRzRzRzR
zixiaotan21
whxleemdddd
tangrm
github-actions[bot]
shymaggie
操作系统：macOS-14.1.1-arm64-arm-64bit. python版本：3.10.13 (main, Sep 11 2023, 08:16:02) [Clang 14.0.6 ] 项目版本：v0.2.10 langchain版本：0.1.13. fastchat版本：0.2.36
当前使用的分词器：ChineseRecursiveTextSplitter 当前启动的LLM模型：['chatglm3-6b'] @ mps {'device': 'mps', 'host': '0.0.0.0', 'infer_turbo': False, 'model_path': '/Users/wanghongxi/Desktop/work/llm/models/chatglm3-6b', 'model_path_exists': True, 'port': 20002} 当前Embbedings模型： bge-large-zh-v1.5 @ mps
服务端运行信息： OpenAI API Server: http://127.0.0.1:20000/v1 Chatchat API Server: http://127.0.0.1:7861 Chatchat WEBUI Server: http://0.0.0.0:8501 ==============================Langchain-Chatchat Configuration==============================
You can now view your Streamlit app in your browser.
URL: http://0.0.0.0:8501
A new version of Streamlit is available.
See what's new at https://discuss.streamlit.io/c/announcements
Enter the following command to upgrade: $ pip install streamlit --upgrade
/Users/wanghongxi/miniconda3/envs/langchain-chatchat/lib/python3.10/site-packages/langchain/chat_models/init.py:31: LangChainDeprecationWarning: Importing chat models from langchain is deprecated. Importing from langchain will no longer be supported as of langchain==0.2.0. Please import from langchain-community instead:
from langchain_community.chat_models import ChatOpenAI.To install langchain-community run
pip install -U langchain-community. warnings.warn( /Users/wanghongxi/miniconda3/envs/langchain-chatchat/lib/python3.10/site-packages/langchain/llms/init.py:548: LangChainDeprecationWarning: Importing LLMs from langchain is deprecated. Importing from langchain will no longer be supported as of langchain==0.2.0. Please import from langchain-community instead:from langchain_community.llms import OpenAI.To install langchain-community run
pip install -U langchain-community. warnings.warn( /Users/wanghongxi/miniconda3/envs/langchain-chatchat/lib/python3.10/site-packages/langchain/document_loaders/init.py:36: LangChainDeprecationWarning: Importing document loaders from langchain is deprecated. Importing from langchain will no longer be supported as of langchain==0.2.0. Please import from langchain-community instead:from langchain_community.document_loaders import JSONLoader.To install langchain-community run
pip install -U langchain-community. warnings.warn( 2024-03-27 16:31:56,516 - _client.py[line:1027] - INFO: HTTP Request: POST http://127.0.0.1:20001/list_models "HTTP/1.1 200 OK" INFO: 127.0.0.1:58186 - "POST /llm_model/list_running_models HTTP/1.1" 200 OK 2024-03-27 16:31:56,517 - _client.py[line:1027] - INFO: HTTP Request: POST http://127.0.0.1:7861/llm_model/list_running_models "HTTP/1.1 200 OK" 2024-03-27 16:31:56,643 - _client.py[line:1027] - INFO: HTTP Request: POST http://127.0.0.1:20001/list_models "HTTP/1.1 200 OK" INFO: 127.0.0.1:58186 - "POST /llm_model/list_running_models HTTP/1.1" 200 OK 2024-03-27 16:31:56,645 - _client.py[line:1027] - INFO: HTTP Request: POST http://127.0.0.1:7861/llm_model/list_running_models "HTTP/1.1 200 OK" INFO: 127.0.0.1:58186 - "POST /llm_model/list_config_models HTTP/1.1" 200 OK 2024-03-27 16:31:56,650 - _client.py[line:1027] - INFO: HTTP Request: POST http://127.0.0.1:7861/llm_model/list_config_models "HTTP/1.1 200 OK" 2024-03-27 16:32:09,328 - _client.py[line:1027] - INFO: HTTP Request: POST http://127.0.0.1:20001/list_models "HTTP/1.1 200 OK" INFO: 127.0.0.1:58203 - "POST /llm_model/list_running_models HTTP/1.1" 200 OK 2024-03-27 16:32:09,329 - _client.py[line:1027] - INFO: HTTP Request: POST http://127.0.0.1:7861/llm_model/list_running_models "HTTP/1.1 200 OK" 2024-03-27 16:32:09,348 - _client.py[line:1027] - INFO: HTTP Request: POST http://127.0.0.1:20001/list_models "HTTP/1.1 200 OK" INFO: 127.0.0.1:58203 - "POST /llm_model/list_running_models HTTP/1.1" 200 OK 2024-03-27 16:32:09,349 - _client.py[line:1027] - INFO: HTTP Request: POST http://127.0.0.1:7861/llm_model/list_running_models "HTTP/1.1 200 OK" INFO: 127.0.0.1:58203 - "POST /llm_model/list_config_models HTTP/1.1" 200 OK 2024-03-27 16:32:09,358 - _client.py[line:1027] - INFO: HTTP Request: POST http://127.0.0.1:7861/llm_model/list_config_models "HTTP/1.1 200 OK" INFO: 127.0.0.1:58203 - "POST /chat/chat HTTP/1.1" 200 OK 2024-03-27 16:32:09,362 - _client.py[line:1027] - INFO: HTTP Request: POST http://127.0.0.1:7861/chat/chat "HTTP/1.1 200 OK" /Users/wanghongxi/miniconda3/envs/langchain-chatchat/lib/python3.10/site-packages/langchain_core/_api/deprecation.py:117: LangChainDeprecationWarning: The classlangchain_community.chat_models.openai.ChatOpenAIwas deprecated in langchain-community 0.0.10 and will be removed in 0.2.0. An updated version of the class exists in the langchain-openai package and should be used instead. To use it runpip install -U langchain-openaiand import asfrom langchain_openai import ChatOpenAI. warn_deprecated( /Users/wanghongxi/miniconda3/envs/langchain-chatchat/lib/python3.10/site-packages/langchain_core/_api/deprecation.py:117: LangChainDeprecationWarning: The functionacallwas deprecated in LangChain 0.1.0 and will be removed in 0.2.0. Use ainvoke instead. warn_deprecated( 2024-03-27 16:32:09,464 - _base_client.py[line:1524] - INFO: Retrying request to /chat/completions in 0.913845 seconds 2024-03-27 16:32:10,382 - _base_client.py[line:1524] - INFO: Retrying request to /chat/completions in 1.707381 seconds 2024-03-27 16:32:12,118 - utils.py[line:38] - ERROR: Connection error. Traceback (most recent call last): File "/Users/wanghongxi/miniconda3/envs/langchain-chatchat/lib/python3.10/site-packages/httpx/_transports/default.py", line 67, in map_httpcore_exceptions yield File "/Users/wanghongxi/miniconda3/envs/langchain-chatchat/lib/python3.10/site-packages/httpx/_transports/default.py", line 371, in handle_async_request resp = await self._pool.handle_async_request(req) File "/Users/wanghongxi/miniconda3/envs/langchain-chatchat/lib/python3.10/site-packages/httpcore/_async/connection_pool.py", line 216, in handle_async_request raise exc from None File "/Users/wanghongxi/miniconda3/envs/langchain-chatchat/lib/python3.10/site-packages/httpcore/_async/connection_pool.py", line 196, in handle_async_request response = await connection.handle_async_request( File "/Users/wanghongxi/miniconda3/envs/langchain-chatchat/lib/python3.10/site-packages/httpcore/_async/connection.py", line 99, in handle_async_request raise exc File "/Users/wanghongxi/miniconda3/envs/langchain-chatchat/lib/python3.10/site-packages/httpcore/_async/connection.py", line 76, in handle_async_request stream = await self._connect(request) File "/Users/wanghongxi/miniconda3/envs/langchain-chatchat/lib/python3.10/site-packages/httpcore/_async/connection.py", line 122, in _connect stream = await self._network_backend.connect_tcp(**kwargs) File "/Users/wanghongxi/miniconda3/envs/langchain-chatchat/lib/python3.10/site-packages/httpcore/_backends/auto.py", line 30, in connect_tcp return await self._backend.connect_tcp( File "/Users/wanghongxi/miniconda3/envs/langchain-chatchat/lib/python3.10/site-packages/httpcore/_backends/anyio.py", line 112, in connect_tcp with map_exceptions(exc_map): File "/Users/wanghongxi/miniconda3/envs/langchain-chatchat/lib/python3.10/contextlib.py", line 153, in exit self.gen.throw(typ, value, traceback) File "/Users/wanghongxi/miniconda3/envs/langchain-chatchat/lib/python3.10/site-packages/httpcore/_exceptions.py", line 14, in map_exceptions raise to_exc(exc) from exc httpcore.ConnectError: All connection attempts failedThe above exception was the direct cause of the following exception:
Traceback (most recent call last): File "/Users/wanghongxi/miniconda3/envs/langchain-chatchat/lib/python3.10/site-packages/openai/_base_client.py", line 1435, in _request response = await self._client.send( File "/Users/wanghongxi/miniconda3/envs/langchain-chatchat/lib/python3.10/site-packages/httpx/_client.py", line 1646, in send response = await self._send_handling_auth( File "/Users/wanghongxi/miniconda3/envs/langchain-chatchat/lib/python3.10/site-packages/httpx/_client.py", line 1674, in _send_handling_auth response = await self._send_handling_redirects( File "/Users/wanghongxi/miniconda3/envs/langchain-chatchat/lib/python3.10/site-packages/httpx/_client.py", line 1711, in _send_handling_redirects response = await self._send_single_request(request) File "/Users/wanghongxi/miniconda3/envs/langchain-chatchat/lib/python3.10/site-packages/httpx/_client.py", line 1748, in _send_single_request response = await transport.handle_async_request(request) File "/Users/wanghongxi/miniconda3/envs/langchain-chatchat/lib/python3.10/site-packages/httpx/_transports/default.py", line 370, in handle_async_request with map_httpcore_exceptions(): File "/Users/wanghongxi/miniconda3/envs/langchain-chatchat/lib/python3.10/contextlib.py", line 153, in exit self.gen.throw(typ, value, traceback) File "/Users/wanghongxi/miniconda3/envs/langchain-chatchat/lib/python3.10/site-packages/httpx/_transports/default.py", line 84, in map_httpcore_exceptions raise mapped_exc(message) from exc httpx.ConnectError: All connection attempts failed
The above exception was the direct cause of the following exception:
Traceback (most recent call last): File "/Users/wanghongxi/Langchain-Chatchat/server/utils.py", line 36, in wrap_done await fn File "/Users/wanghongxi/miniconda3/envs/langchain-chatchat/lib/python3.10/site-packages/langchain_core/_api/deprecation.py", line 154, in awarning_emitting_wrapper return await wrapped(*args, kwargs) File "/Users/wanghongxi/miniconda3/envs/langchain-chatchat/lib/python3.10/site-packages/langchain/chains/base.py", line 428, in acall return await self.ainvoke( File "/Users/wanghongxi/miniconda3/envs/langchain-chatchat/lib/python3.10/site-packages/langchain/chains/base.py", line 212, in ainvoke raise e File "/Users/wanghongxi/miniconda3/envs/langchain-chatchat/lib/python3.10/site-packages/langchain/chains/base.py", line 203, in ainvoke await self._acall(inputs, run_manager=run_manager) File "/Users/wanghongxi/miniconda3/envs/langchain-chatchat/lib/python3.10/site-packages/langchain/chains/llm.py", line 275, in _acall response = await self.agenerate([inputs], run_manager=run_manager) File "/Users/wanghongxi/miniconda3/envs/langchain-chatchat/lib/python3.10/site-packages/langchain/chains/llm.py", line 142, in agenerate return await self.llm.agenerate_prompt( File "/Users/wanghongxi/miniconda3/envs/langchain-chatchat/lib/python3.10/site-packages/langchain_core/language_models/chat_models.py", line 556, in agenerate_prompt return await self.agenerate( File "/Users/wanghongxi/miniconda3/envs/langchain-chatchat/lib/python3.10/site-packages/langchain_core/language_models/chat_models.py", line 516, in agenerate raise exceptions[0] File "/Users/wanghongxi/miniconda3/envs/langchain-chatchat/lib/python3.10/site-packages/langchain_core/language_models/chat_models.py", line 638, in _agenerate_with_cache result = await self._agenerate( File "/Users/wanghongxi/miniconda3/envs/langchain-chatchat/lib/python3.10/site-packages/langchain_community/chat_models/openai.py", line 526, in _agenerate return await agenerate_from_stream(stream_iter) File "/Users/wanghongxi/miniconda3/envs/langchain-chatchat/lib/python3.10/site-packages/langchain_core/language_models/chat_models.py", line 84, in agenerate_from_stream async for chunk in stream: File "/Users/wanghongxi/miniconda3/envs/langchain-chatchat/lib/python3.10/site-packages/langchain_community/chat_models/openai.py", line 490, in _astream async for chunk in await acompletion_with_retry( File "/Users/wanghongxi/miniconda3/envs/langchain-chatchat/lib/python3.10/site-packages/langchain_community/chat_models/openai.py", line 105, in acompletion_with_retry return await llm.async_client.create(kwargs) File "/Users/wanghongxi/miniconda3/envs/langchain-chatchat/lib/python3.10/site-packages/openai/resources/chat/completions.py", line 1300, in create return await self._post( File "/Users/wanghongxi/miniconda3/envs/langchain-chatchat/lib/python3.10/site-packages/openai/_base_client.py", line 1703, in post return await self.request(cast_to, opts, stream=stream, stream_cls=stream_cls) File "/Users/wanghongxi/miniconda3/envs/langchain-chatchat/lib/python3.10/site-packages/openai/_base_client.py", line 1406, in request return await self._request( File "/Users/wanghongxi/miniconda3/envs/langchain-chatchat/lib/python3.10/site-packages/openai/_base_client.py", line 1459, in _request return await self._retry_request( File "/Users/wanghongxi/miniconda3/envs/langchain-chatchat/lib/python3.10/site-packages/openai/_base_client.py", line 1528, in _retry_request return await self._request( File "/Users/wanghongxi/miniconda3/envs/langchain-chatchat/lib/python3.10/site-packages/openai/_base_client.py", line 1459, in _request return await self._retry_request( File "/Users/wanghongxi/miniconda3/envs/langchain-chatchat/lib/python3.10/site-packages/openai/_base_client.py", line 1528, in _retry_request return await self._request( File "/Users/wanghongxi/miniconda3/envs/langchain-chatchat/lib/python3.10/site-packages/openai/_base_client.py", line 1469, in _request raise APIConnectionError(request=request) from err openai.APIConnectionError: Connection error. 2024-03-27 16:32:12,134 - utils.py[line:40] - ERROR: APIConnectionError: Caught exception: Connection error.