chatchat-space / Langchain-Chatchat

Langchain-Chatchat(原Langchain-ChatGLM)基于 Langchain 与 ChatGLM, Qwen 与 Llama 等语言模型的 RAG 与 Agent 应用 | Langchain-Chatchat (formerly langchain-ChatGLM), local knowledge based LLM (like ChatGLM, Qwen and Llama) RAG and Agent app with langchain
Apache License 2.0
31.91k stars 5.56k forks source link

[BUG] 部署本地模型后用request请求测试api接口时出错 #4706

Closed inrelif closed 2 months ago

inrelif commented 3 months ago

问题描述 / Problem Description 对于同一个问题用中文和英文分别作为prompt发送请求,会有不同的请求结果 英文返回502,中文可以正常返回结果,有没有大神知道这个是什么问题

英文情况

import requests import json

api_base_url = 'http://127.0.0.1:7861/v1' api_key = "EMPTY" prompt = 'Why is the temperature higher on the leeward slope than on the windward slope' headers = { "Authorization": f"Bearer {api_key}" }

data = { "model": "glm4-chat", "messages": [{"role": "user", "content": prompt}], "stream": False

} response = requests.post(f'{api_base_url}/chat/completions', headers=headers, data=json.dumps(data)) print(response) print(response.json())

<Response [502]>

JSONDecodeError Traceback (most recent call last) File ~/anaconda3/envs/chatchat/lib/python3.10/site-packages/requests/models.py:971, in Response.json(self, kwargs) 970 try: --> 971 return complexjson.loads(self.text, kwargs) 972 except JSONDecodeError as e: 973 # Catch JSON-related errors and raise as requests.JSONDecodeError 974 # This aliases json.JSONDecodeError and simplejson.JSONDecodeError

File ~/anaconda3/envs/chatchat/lib/python3.10/site-packages/simplejson/init.py:514, in loads(s, encoding, cls, object_hook, parse_float, parse_int, parse_constant, object_pairs_hook, use_decimal, allow_nan, **kw) 510 if (cls is None and encoding is None and object_hook is None and 511 parse_int is None and parse_float is None and 512 parse_constant is None and object_pairs_hook is None 513 and not use_decimal and not allow_nan and not kw): --> 514 return _default_decoder.decode(s) 515 if cls is None:

File ~/anaconda3/envs/chatchat/lib/python3.10/site-packages/simplejson/decoder.py:386, in JSONDecoder.decode(self, s, _w, _PY3) 385 s = str(s, self.encoding) --> 386 obj, end = self.raw_decode(s) 387 end = _w(s, end).end()

File ~/anaconda3/envs/chatchat/lib/python3.10/site-packages/simplejson/decoder.py:416, in JSONDecoder.raw_decode(self, s, idx, _w, _PY3) 415 idx += 3 --> 416 return self.scan_once(s, idx=_w(s, idx).end()) ... 973 # Catch JSON-related errors and raise as requests.JSONDecodeError 974 # This aliases json.JSONDecodeError and simplejson.JSONDecodeError --> 975 raise RequestsJSONDecodeError(e.msg, e.doc, e.pos)

JSONDecodeError: Expecting value: line 1 column 1 (char 0)

中文情况

import requests import json

api_base_url = 'http://127.0.0.1:7861/v1' api_key = "EMPTY" prompt = '为什么背风坡的温度比迎风坡高?' headers = { "Authorization": f"Bearer {api_key}" }

data = { "model": "glm4-chat", "messages": [{"role": "user", "content": prompt}], "stream": False

} response = requests.post(f'{api_base_url}/chat/completions', headers=headers, data=json.dumps(data)) print(response) print(response.json())

<Response [200]> {'id': 'chat44ef9ac6-561e-11ef-892e-00e2696655ff', 'choices': [{'finish_reason': 'stop', 'index': 0, 'logprobs': None, 'message': {'content': '背风坡的温度比迎风坡高的现象,主要与地形对气流的影响以及大气环流有关,具体原因如下:\n\n1. 地形阻挡:当湿润的空气遇到山脉等高地时,由于空气被迫上升,温度会下降(这是由于上升过程中气压下降,根据理想气体状态方程PV=nRT,在体积不变的情况下,温度T下降)。这种上升冷却的空气流过山脊,到达山脊另一侧时,空气被迫下沉。在下沉过程中,空气温度会升高(因为气压增加,根据理想气体状态方程,温度T升高)。\n\n2. 下沉增温:当空气从山顶下沉到山谷时,由于空气压力增加,其密度也会增加。根据理想气体状态方程,密度增加会导致温度升高,因此背风坡(即山脉的另一侧)的温度通常比迎风坡(即山脉的一侧)要高。\n\n3. 辐射冷却:迎风坡由于接受更多的太阳辐射,表面温度较高,导致夜间辐射冷却更加剧烈。而背风坡由于白天温度较高,夜间辐射冷却相对较慢,因此整体温度高于迎风坡。\n\n4. 水汽凝结:迎风坡上升的空气冷却后,水汽容易凝结成云和降水,导致迎风坡相对干燥,而背风坡由于空气下沉增温,水汽不易凝结,相对湿度较低。\n\n这种现象在气象学中被称为“地形增温”或“山影效应”,在山脉附近地区尤其明显。', 'role': 'assistant', 'function_call': None, 'tool_calls': None}}], 'created': 1723187347, 'model': 'glm4-chat', 'object': 'chat.completion', 'service_tier': None, 'system_fingerprint': None, 'usage': {'completion_tokens': -1, 'prompt_tokens': -1, 'total_tokens': -1}}

liunux4odoo commented 2 months ago

无法复现,按照提供的英文代码测试,正常返回:

<Response [200]>
{'id': 'chatf9c08635-cb5a-42bc-8b85-55243dfc8bac', 'choices': [{'finish_reason': 'stop', 'index': 0, 'logprobs': None, 'message': {'content': 'The difference in temperature between the windward (downwind) and leeward (upwind) slopes of a mountain or obstacle is influenced by several factors, including wind patterns, heat absorption, and how these two areas interact with air currents.\n\nHere\'s why you might observe higher temperatures on the leeward side:\n\n1. **Wind Direction and Speed**: On the windward slope, or迎风坡 ("Windswept Slope") as it\'s known in Chinese, the wind comes from the direction of the mountain. This means that as the wind blows over the mountains, its energy is partially used to lift the air up. Due to the friction, this can cause some of the wind\'s kinetic energy to be converted into heat, which warms the air. However, most of the warm air will continue moving forward, so if you were measuring temperature immediately after the wind hit a specific area, it would likely feel warmer due to increased air movement.\n\n2. **Emissivity of the Surface**: The surface of the ground can emit heat differently based on its material composition and thermal properties. If the land or soil is more efficient at absorbing solar radiation during the day (for example, blacktop or asphalt), it could lead to higher temperatures when compared to grassier or less absorptive surfaces.\n\n3. **Heat Retention**: On the leeward side, because there’s no wind (or very little wind) to carry away heat, this area tends to retain more heat throughout the night. Warmed by the sun during the day, the lower winds prevent this heat from escaping easily until later in the night. This effect persists even if other conditions suggest cooler conditions (such as being closer to a larger body of water like an ocean). \n\n4. **Inversion Layers**: Inversion layers, where cold air is trapped above warm air due to stable conditions, can form on the leeward side, further increasing temperatures by trapping warmth close to the ground.\n\n5. **Ascent and Descent**: As air ascends over a mountain barrier, it cools (the adiabatic lapse rate) unless there are other factors such as heating from the ground or warming through friction in the atmosphere. The air descending the leeward side of the hill can be warmer if it has lifted enough to reach the temperature inversion level, especially if condensation occurs high in the atmosphere, which releases latent heat back to the air near the ground.\n\nUnderstanding these dynamics helps explain natural phenomena seen in various geographical settings around the world. It also illustrates why certain weather patterns may exhibit unusual characteristics under specific conditions.', 'refusal': None, 'role': 'assistant', 'function_call': None, 'tool_calls': None}}], 'created': 1724854546, 'model': 'qwen2-int3', 'object': 'chat.completion', 'service_tier': None, 'system_fingerprint': None, 'usage': {'completion_tokens': 521, 'prompt_tokens': 35, 'total_tokens': 556}}

可以到模型加载框架下,仔细检查下传入的参数是否正确。