Got an Error while using Langchain Chatchat with internlm2.5.

frankSARU commented 4 days ago

System Info / 系統信息

CUDA 12.4 Transformer 4.44

Running Xinference with Docker? / 是否使用 Docker 运行 Xinfernece？

[ ] docker / docker
[X] pip install / 通过 pip install 安装
[ ] installation from source / 从源码安装

Version info / 版本信息

Version:Release: v0.16.0

The command used to start Xinference / 用以启动 xinference 的命令

HF_ENDPOINT=https://hf-mirror.com xinference-local --host 10.11.238.110 --port 9997

Reproduction / 复现过程

在使用chat功能时，xinf平台会出现如下报错（使用kb_chat功能，即知识库功能，并不会报错，其他知识库应用也未报错，排查了Langchain Chatchat相关的设置，chat.py脚本自上一次成功运行以来未做修改，在xinf页面报issue只是为了排查具体是哪个模块的问题）： TypeError: int() argument must be a string, a bytes-like object or a real number, not 'NoneType' 2024-11-18 17:24:13,778 xinference.api.restful_api 3537005 ERROR Chat completion stream got an error: [address=10.11.238.110:36475, pid=3545480] int() argument must be a string, a bytes-like object or a real number, not 'NoneType' Traceback (most recent call last): File "/home/pei/.conda/envs/xinf/lib/python3.11/site-packages/xinference/api/restful_api.py", line 1926, in stream_results async for item in iterator: File "/home/pei/.conda/envs/xinf/lib/python3.11/site-packages/xoscar/api.py", line 340, in anext return await self._actor_ref.xoscar_next(self._uid) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/pei/.conda/envs/xinf/lib/python3.11/site-packages/xoscar/backends/context.py", line 231, in send return self._process_result_message(result) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/pei/.conda/envs/xinf/lib/python3.11/site-packages/xoscar/backends/context.py", line 102, in _process_result_message raise message.as_instanceof_cause() File "/home/pei/.conda/envs/xinf/lib/python3.11/site-packages/xoscar/backends/pool.py", line 656, in send result = await self._run_coro(message.message_id, coro) ^^^^^^^^^^^^^^^^^ File "/home/pei/.conda/envs/xinf/lib/python3.11/site-packages/xoscar/backends/pool.py", line 367, in _run_coro return await coro File "/home/pei/.conda/envs/xinf/lib/python3.11/site-packages/xoscar/api.py", line 384, in on_receive return await super().on_receive(message) # type: ignore ^^^^^^^^^^^^^^^^^ File "xoscar/core.pyx", line 558, in on_receive raise ex File "xoscar/core.pyx", line 520, in xoscar.core._BaseActor.on_receive async with self._lock: ^^^^^^^^^^^^^^^^^ File "xoscar/core.pyx", line 521, in xoscar.core._BaseActor.on_receive with debug_async_timeout('actor_lock_timeout', ^^^^^^^^^^^^^^^^^ File "xoscar/core.pyx", line 526, in xoscar.core._BaseActor.on_receive result = await result ^^^^^^^^^^^^^^^^^ File "/home/pei/.conda/envs/xinf/lib/python3.11/site-packages/xoscar/api.py", line 431, in xoscar_next raise e File "/home/pei/.conda/envs/xinf/lib/python3.11/site-packages/xoscar/api.py", line 419, in __xoscar_next r = await asyncio.create_task(_async_wrapper(gen)) ^^^^^^^^^^^^^^^^^ File "/home/pei/.conda/envs/xinf/lib/python3.11/site-packages/xoscar/api.py", line 409, in _async_wrapper return await _gen.anext__() # noqa: F821 ^^^^^^^^^^^^^^^^^ File "/home/pei/.conda/envs/xinf/lib/python3.11/site-packages/xinference/core/model.py", line 440, in _to_async_gen async for v in gen: ^^^^^^^^^^^^^^^^^ File "/home/pei/.conda/envs/xinf/lib/python3.11/site-packages/xinference/core/model.py", line 568, in _queue_consumer raise RuntimeError(res[len(XINFERENCE_STREAMING_ERROR_FLAG) :]) ^^^^^^^^^^^^^^^^^ RuntimeError: [address=10.11.238.110:36475, pid=3545480] int() argument must be a string, a bytes-like object or a real number, not 'NoneType'

Expected behavior / 期待表现

与模型可以正常聊天交互

feme3344 commented 18 hours ago

同问

frankSARU commented 18 hours ago

问题补充：其他类似的AI对话框架在调用Langchain进行正常的非知识库对话时，Xinf平台不会报错。

xorbitsai / inference