search

chatchat-space / Langchain-Chatchat

Langchain-Chatchat（原Langchain-ChatGLM）基于 Langchain 与 ChatGLM, Qwen 与 Llama 等语言模型的 RAG 与 Agent 应用 | Langchain-Chatchat (formerly langchain-ChatGLM), local knowledge based LLM (like ChatGLM, Qwen and Llama) RAG and Agent app with langchain
Apache License 2.0
31.3k stars 5.45k forks source link

本地知识库web方式正常，api方式出错 #4363

Closed lettergod closed 2 months ago

lettergod commented 2 months ago

安装6月30日最新版本 Xinference qwen1.5-chat+bge-large-zh-v1.5

Langchain-Chatchat chatchat-config model --default_llm_model qwen1.5-chat

Langchain-Chatchat web方式与LLM对话正常 web方式与本地知识库正常

[https://github.com/chatchat-space/Langchain-Chatchat/blob/dev/docs/contributing/api.md]() 用此api运行 纯 LLM 对话正常 知识库对话错误 运行 base_url = "http://127.0.0.1:7861/chat" data = { "messages": [ {"role": "user", "content": "如何提问以获得高质量答案"}, ], "model": "qwen1.5-chat", "tool_choice": "search_local_knowledgebase", "extra_body": {"tool_input": {"database": "zb", "query": "如何提问以获得高质量答案"}}, "stream": True, }

import requests response = requests.post(f"{base_url}/chat/completions", json=data, stream=True) for line in response.iter_content(None, decode_unicode=True): print(line) 返回信息：

data: {"id": "chataef2cc4b-bb97-48e6-b325-17062ea15880", "object": "chat.completion.chunk", "model": "glm4-chat", "created": 1719732663, "status": 1, "message_type": 1, "message_id": null, "is_ref": false, "choices": [{"delta": {"content": "", "tool_calls": []}, "role": "assistant"}]}

data: {"id": "chata865e33e-18e0-487e-8224-d8293028717a", "object": "chat.completion.chunk", "model": "glm4-chat", "created": 1719732663, "status": 8, "message_type": 1, "message_id": null, "is_ref": false, "choices": [{"delta": {"content": "peer closed connection without sending complete message body (incomplete chunked read)", "tool_calls": []}, "role": "assistant"}]}

Langchain-Chatchat提示信息： httpx.RemoteProtocolError: peer closed connection without sending complete message body (incomplete chunked read) 2024-06-30 07:42:51,744 root 2476 ERROR RemoteProtocolError: Caught exception: peer closed connection without sending complete message body (incomplete chunked read)

Xinference提示信息： 2024-06-30 07:42:51,734 xinference.api.restful_api 1498 ERROR [address=0.0.0.0:46897, pid=1536] Model not found in the model list, uid: glm4-chat Traceback (most recent call last): File "/root/anaconda3/envs/xf/lib/python3.11/site-packages/xinference/api/restful_api.py", line 1459, in create_chat_completion model = await (await self._get_supervisor_ref()).get_model(model_uid) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ...................... File "/root/anaconda3/envs/xf/lib/python3.11/site-packages/xinference/core/supervisor.py", line 968, in get_model raise ValueError(f"Model not found in the model list, uid: {model_uid}") ^^^^^^^^^^^^^^^^^ ValueError: [address=0.0.0.0:46897, pid=1536] Model not found in the model list, uid: glm4-chat

如果在Xinference中运行多个模型，在纯 LLM 对话api 更换"model": "qwen1.5-chat",比如"model": "glm4-chat"，没有问题。

是不是"知识库对话的api“传递过程中，传错"model"这个参数了啊

lettergod commented 2 months ago

api写的是"model": "qwen1.5-chat", 到Xinference提示的是 Model not found in the model list, uid: glm4-chat

BaoxingZhang commented 2 months ago

我也遇到这个报错了，可以使用[/chat/chat/completions]兼容 openai 的统一 chat 接口，这个是没有问题。

单纯的/chat/chat接口没有找到在哪里赋值的glm4-chat这个值。我有检查这四个地方： 1、--default_llm_model 有检查 2、--set_model_platforms 有检查 3、model_providers.yaml 有检查 4、workspace_config.json 有检查

均不是glm4-chat，但是调用的时候就是使用的glm4-chat

etveritas commented 2 months ago

是不是版本的问题啊

fengzengfly commented 2 months ago

same problem

RKwork85 commented 2 months ago

同样的问题啊

ce33386fe5ede01cdbdd59d9e2917c77

liunux4odoo commented 2 months ago

需要检查 LLM_MODEL_CONFIG 里面的 *_model 也是正确的模型。

plancktree commented 2 months ago

需要检查 LLM_MODEL_CONFIG 里面的 *_model 也是正确的模型。

怎么修改LLM_MODEL_CONFIG 里面的参数变量呢？通过命令行似乎无法修改

plancktree commented 2 months ago

我也遇到这个报错了，可以使用[/chat/chat/completions]兼容 openai 的统一 chat 接口，这个是没有问题。

单纯的/chat/chat接口没有找到在哪里赋值的glm4-chat这个值。我有检查这四个地方： 1、--default_llm_model 有检查 2、--set_model_platforms 有检查 3、model_providers.yaml 有检查 4、workspace_config.json 有检查

均不是glm4-chat，但是调用的时候就是使用的glm4-chat

我用/chat/chat/completions接口，这样如下报文调用知识库失败，正常模型对话正常，一直报错 requests.exceptions.ConnectTimeout: HTTPSConnectionPool(host='api.hub.langchain.com', port=443): Max retries exceeded with url: /commits/hwchase17/structured-chat-agent/?limit=100&offset=0 (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x7f3d75afd4e0>, 'Connection to api.hub.langchain.com timed out. (connect timeout=None)')) { "messages": [ {"role": "user", "content": "如何提问以获得高质量答案"} ], "model": "qwen", "tool_choice": "search_local_knowledgebase", "extra_body": {"tool_input": {"database": "samples", "query": "如何提问以获得高质量答案"}}, "stream": true, "top_k": 1 }

ELvis168 commented 2 months ago

看来都有这个问题啊，返回的是glm4-chat

ELvis168 commented 2 months ago

看来都有这个问题啊，返回的是glm4-chat 找到地方了，直接改源码：self.DEFAULT_LLM_MODEL = "glm4-chat" /home/xxx/miniconda3/envs/**/lib/python3.11/site-packages/chatchat/configs/_model_config.py