[BUG] 运用本地知识库询问几句之后报错 / Concise description of the issue

LLM模型和embedding模型正常加载，能跳转至web页面，使用本地构建的知识库可以进行正常询问回答，但是询问三句左右就无法正常进行答复，重启模型或者重新加载web界面可以正常询问，我想知道是哪方面的问题

==============================Langchain-Chatchat Configuration============================== 操作系统：Windows-10-10.0.14393-SP0. python版本：3.10.0 | packaged by conda-forge | (default, Nov 10 2021, 13:20:59) [MSC v.1916 64 bit (AMD64)] 项目版本：v0.2.10 langchain版本：0.0.354. fastchat版本：0.2.35 当前使用的分词器：ChineseRecursiveTextSplitter 当前启动的LLM模型：['chatglm3-6b'] @ cuda {'device': 'cuda', 'host': '127.0.0.1', 'infer_turbo': False, 'model_path': 'F:\ChatGLM\langchain-ChatGLM\models\chatglm3-6b', 'model_path_exists': True, 'port': 20002} 当前Embbedings模型： text2vec @ cuda ==============================Langchain-Chatchat Configuration============================== 2024-03-26 14:44:07,315 - startup.py[line:655] - INFO: 正在启动服务： 2024-03-26 14:44:07,315 - startup.py[line:656] - INFO: 如需查看 llm_api 日志，请前往 F:\ChatGLM\langchain-ChatGLM\logs F:\Anaconda\envs\ChatGLM\lib\site-packages\langchain_core_api\deprecation.py:117: LangChainDeprecationWarning: 模型启动功能将于 Langchain-Chatchat 0.3.x重写,支持更多模式和加速启动，0.2.x中相关功能将废弃 warn_deprecated( 2024-03-26 14:44:16 | ERROR | stderr | INFO: Started server process [3532] 2024-03-26 14:44:16 | ERROR | stderr | INFO: Waiting for application startup. 2024-03-26 14:44:16 | ERROR | stderr | INFO: Application startup complete. 2024-03-26 14:44:16 | ERROR | stderr | INFO: Uvicorn running on http://127.0.0.1:20000 (Press CTRL+C to quit) 2024-03-26 14:44:17 | INFO | model_worker | Loading the model ['chatglm3-6b'] on worker 0f6e17ed ... 2024-03-26 14:44:17 | WARNING | transformers_modules.chatglm3-6b.tokenization_chatglm | Setting eos_token is not supported, use the default one. 2024-03-26 14:44:17 | WARNING | transformers_modules.chatglm3-6b.tokenization_chatglm | Setting pad_token is not supported, use the default one. 2024-03-26 14:44:17 | WARNING | transformers_modules.chatglm3-6b.tokenization_chatglm | Setting unk_token is not supported, use the default one. Loading checkpoint shards: 0%| | 0/7 [00:00<?, ?it/s] Loading checkpoint shards: 14%|██████████ | 1/7 [00:00<00:04, 1.20it/s] Loading checkpoint shards: 29%|████████████████████ | 2/7 [00:01<00:04, 1.04it/s] Loading checkpoint shards: 43%|██████████████████████████████ | 3/7 [00:02<00:04, 1.00s/it] Loading checkpoint shards: 57%|████████████████████████████████████████ | 4/7 [00:03<00:03, 1.03s/it] Loading checkpoint shards: 71%|██████████████████████████████████████████████████ | 5/7 [00:05<00:02, 1.07s/it] Loading checkpoint shards: 86%|██████████████████████████████████████████████████ ██████████ | 6/7 [00:06<00:01, 1.11s/it] Loading checkpoint shards: 100%|██████████████████████████████████████████████████ ████████████████████| 7/7 [00:07<00:00, 1.01it/s] Loading checkpoint shards: 100%|██████████████████████████████████████████████████ ████████████████████| 7/7 [00:07<00:00, 1.01s/it] 2024-03-26 14:44:25 | ERROR | stderr | 2024-03-26 14:44:30 | INFO | model_worker | Register to controller INFO: Started server process [9240] INFO: Waiting for application startup. INFO: Application startup complete. INFO: Uvicorn running on http://127.0.0.1:7861 (Press CTRL+C to quit) =============================Langchain-Chatchat Configuration============================== 操作系统：Windows-10-10.0.14393-SP0. python版本：3.10.0 | packaged by conda-forge | (default, Nov 10 2021, 13:20:59) [MSC v.1916 64 bit (AMD64)] 项目版本：v0.2.10 langchain版本：0.0.354. fastchat版本：0.2.35 当前使用的分词器：ChineseRecursiveTextSplitter 当前启动的LLM模型：['chatglm3-6b'] @ cuda {'device': 'cuda', 'host': '127.0.0.1', 'infer_turbo': False, 'model_path': 'F:\ChatGLM\langchain-ChatGLM\models\chatglm3-6b', 'model_path_exists': True, 'port': 20002} 当前Embbedings模型： text2vec @ cuda 服务端运行信息： OpenAI API Server: http://127.0.0.1:20000/v1 Chatchat API Server: http://127.0.0.1:7861 Chatchat WEBUI Server: http://127.0.0.1:8501 ==============================Langchain-Chatchat Configuration============================== You can now view your Streamlit app in your browser. URL: http://127.0.0.1:8501

下面是报错部分的log：

2024-03-26 14:47:58 | INFO | stdout | INFO: 127.0.0.1:56672 - "POST /v1/chat/completions HTTP/1.1" 200 OK 2024-03-26 14:47:58,862 - _client.py[line:1773] - INFO: HTTP Request: POST http://127.0.0.1:20000/v1/chat/completions "HTTP/1.1 200 OK" 2024-03-26 14:47:58 | INFO | httpx | HTTP Request: POST http://127.0.0.1:20002/worker_generate_stream "HTTP/1.1 200 OK" ERROR: Exception in ASGI application Traceback (most recent call last): File "F:\Anaconda\envs\ChatGLM\lib\site-packages\starlette\responses.py", line 261, in call await wrap(partial(self.listen_for_disconnect, receive)) File "F:\Anaconda\envs\ChatGLM\lib\site-packages\starlette\responses.py", line 257, in wrap await func() File "F:\Anaconda\envs\ChatGLM\lib\site-packages\starlette\responses.py", line 234, in listen_for_disconnect message = await receive() File "F:\Anaconda\envs\ChatGLM\lib\site-packages\uvicorn\protocols\http\h11_impl.py", line 538, in receive await self.message_event.wait() File "F:\Anaconda\envs\ChatGLM\lib\asyncio\locks.py", line 213, in wait await fut asyncio.exceptions.CancelledError: Cancelled by cancel scope 188000f6860

During handling of the above exception, another exception occurred:

Exception Group Traceback (most recent call last):	File "F:\Anaconda\envs\ChatGLM\lib\site-packages\uvicorn\protocols\http\h11_impl.py", line 408, in run_asgi	result = await app( # type: ignore[func-returns-value]	File "F:\Anaconda\envs\ChatGLM\lib\site-packages\uvicorn\middleware\proxy_headers.py", line 69, in call	return await self.app(scope, receive, send)	File "F:\Anaconda\envs\ChatGLM\lib\site-packages\fastapi\applications.py", line 1054, in call	await super().call(scope, receive, send)	File "F:\Anaconda\envs\ChatGLM\lib\site-packages\starlette\applications.py", line 119, in call	await self.middleware_stack(scope, receive, send)	File "F:\Anaconda\envs\ChatGLM\lib\site-packages\starlette\middleware\errors.py", line 186, in call	raise exc	File "F:\Anaconda\envs\ChatGLM\lib\site-packages\starlette\middleware\errors.py", line 164, in call	await self.app(scope, receive, _send)	File "F:\Anaconda\envs\ChatGLM\lib\site-packages\starlette\middleware\exceptions.py", line 62, in call	await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)	File "F:\Anaconda\envs\ChatGLM\lib\site-packages\starlette_exception_handler.py", line 64, in wrapped_app	raise exc	File "F:\Anaconda\envs\ChatGLM\lib\site-packages\starlette_exception_handler.py", line 53, in wrapped_app	await app(scope, receive, sender)	File "F:\Anaconda\envs\ChatGLM\lib\site-packages\starlette\routing.py", line 762, in call	await self.middleware_stack(scope, receive, send)	File "F:\Anaconda\envs\ChatGLM\lib\site-packages\starlette\routing.py", line 782, in app	await route.handle(scope, receive, send)	File "F:\Anaconda\envs\ChatGLM\lib\site-packages\starlette\routing.py", line 297, in handle	await self.app(scope, receive, send)	File "F:\Anaconda\envs\ChatGLM\lib\site-packages\starlette\routing.py", line 77, in app	await wrap_app_handling_exceptions(app, request)(scope, receive, send)	File "F:\Anaconda\envs\ChatGLM\lib\site-packages\starlette_exception_handler.py", line 64, in wrapped_app	raise exc	File "F:\Anaconda\envs\ChatGLM\lib\site-packages\starlette_exception_handler.py", line 53, in wrapped_app	await app(scope, receive, sender)	File "F:\Anaconda\envs\ChatGLM\lib\site-packages\starlette\routing.py", line 75, in app	await response(scope, receive, send)	File "F:\Anaconda\envs\ChatGLM\lib\site-packages\starlette\responses.py", line 254, in call	async with anyio.create_task_group() as task_group:	File "F:\Anaconda\envs\ChatGLM\lib\site-packages\anyio_backends_asyncio.py", line 678, in aexit	raise BaseExceptionGroup(	exceptiongroup.ExceptionGroup: unhandled errors in a TaskGroup (1 sub-exception) +-+---------------- 1 ----------------	Traceback (most recent call last):	File "F:\Anaconda\envs\ChatGLM\lib\site-packages\fastchat\serve\model_worker.py", line 114, in generate_stream_gate	for output in self.generate_stream_func(	File "F:\Anaconda\envs\ChatGLM\lib\site-packages\torch\autograd\grad_mode.py", line 43, in generator_context	response = gen.send(None)	File "F:\Anaconda\envs\ChatGLM\lib\site-packages\fastchat\model\model_chatglm.py", line 106, in generate_stream_chatglm	for total_ids in model.stream_generate(inputs, gen_kwargs):	File "F:\Anaconda\envs\ChatGLM\lib\site-packages\torch\autograd\grad_mode.py", line 43, in generator_context	response = gen.send(None)	File "C:\Users\Administrator/.cache\huggingface\modules\transformers_modules\chatglm3-6b\modeling_chatglm.py", line 1163, in stream_generate	outputs = self(	File "F:\Anaconda\envs\ChatGLM\lib\site-packages\torch\nn\modules\module.py", line 1130, in _call_impl	return forward_call(input, *kwargs)	File "C:\Users\Administrator/.cache\huggingface\modules\transformers_modules\chatglm3-6b\modeling_chatglm.py", line 941, in forward	transformer_outputs = self.transformer(	File "F:\Anaconda\envs\ChatGLM\lib\site-packages\torch\nn\modules\module.py", line 1130, in _call_impl	return forward_call(input, *kwargs)	File "C:\Users\Administrator/.cache\huggingface\modules\transformers_modules\chatglm3-6b\modeling_chatglm.py", line 834, in forward	hidden_states, presents, all_hidden_states, all_self_attentions = self.encoder(	File "F:\Anaconda\envs\ChatGLM\lib\site-packages\torch\nn\modules\module.py", line 1130, in _call_impl	return forward_call(input, *kwargs)	File "C:\Users\Administrator/.cache\huggingface\modules\transformers_modules\chatglm3-6b\modeling_chatglm.py", line 641, in forward	layer_ret = layer(	File "F:\Anaconda\envs\ChatGLM\lib\site-packages\torch\nn\modules\module.py", line 1130, in _call_impl	return forward_call(input, *kwargs)	File "C:\Users\Administrator/.cache\huggingface\modules\transformers_modules\chatglm3-6b\modeling_chatglm.py", line 544, in forward	attention_output, kv_cache = self.self_attention(	File "F:\Anaconda\envs\ChatGLM\lib\site-packages\torch\nn\modules\module.py", line 1130, in _call_impl	return forward_call(input, *kwargs)	File "C:\Users\Administrator/.cache\huggingface\modules\transformers_modules\chatglm3-6b\modeling_chatglm.py", line 441, in forward	context_layer = self.core_attention(query_layer, key_layer, value_layer, attention_mask)	File "F:\Anaconda\envs\ChatGLM\lib\site-packages\torch\nn\modules\module.py", line 1130, in _call_impl	return forward_call(input, *kwargs)	File "C:\Users\Administrator/.cache\huggingface\modules\transformers_modules\chatglm3-6b\modeling_chatglm.py", line 273, in forward	attention_scores = attention_scores * self.coeff	RuntimeError: CUDA out of memory. Tried to allocate 610.00 MiB (GPU 0; 15.91 GiB total capacity; 12.95 GiB already allocated; 358.19 MiB free; 13.04 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "F:\Anaconda\envs\ChatGLM\lib\site-packages\starlette\responses.py", line 257, in wrap
await func()
File "F:\Anaconda\envs\ChatGLM\lib\site-packages\starlette\responses.py", line 246, in stream_response
async for chunk in self.body_iterator:
File "F:\Anaconda\envs\ChatGLM\lib\site-packages\starlette\concurrency.py", line 63, in iterate_in_threadpool
yield await anyio.to_thread.run_sync(_next, as_iterator)
File "F:\Anaconda\envs\ChatGLM\lib\site-packages\anyio\to_thread.py", line 56, in run_sync
return await get_async_backend().run_sync_in_worker_thread(
File "F:\Anaconda\envs\ChatGLM\lib\site-packages\anyio_backends_asyncio.py", line 2144, in run_sync_in_worker_thread
return await future
File "F:\Anaconda\envs\ChatGLM\lib\site-packages\anyio_backends_asyncio.py", line 851, in run
result = context.run(func, *args)
File "F:\Anaconda\envs\ChatGLM\lib\site-packages\starlette\concurrency.py", line 52, in _next
return next(iterator)
File "F:\Anaconda\envs\ChatGLM\lib\site-packages\fastchat\serve\model_worker.py", line 133, in generate_stream_gate
except torch.cuda.OutOfMemoryError as e:
AttributeError: module 'torch.cuda' has no attribute 'OutOfMemoryError'

+------------------------------------ 2024-03-26 14:48:00 | ERROR | stderr | ERROR: Exception in ASGI application 2024-03-26 14:48:00 | ERROR | stderr | Traceback (most recent call last): 2024-03-26 14:48:00 | ERROR | stderr | File "F:\Anaconda\envs\ChatGLM\lib\site-packages\starlette\responses.py", line 261, in call 2024-03-26 14:48:00 | ERROR | stderr | await wrap(partial(self.listen_for_disconnect, receive)) 2024-03-26 14:48:00 | ERROR | stderr | File "F:\Anaconda\envs\ChatGLM\lib\site-packages\starlette\responses.py", line 257, in wrap 2024-03-26 14:48:00 | ERROR | stderr | await func() 2024-03-26 14:48:00 | ERROR | stderr | File "F:\Anaconda\envs\ChatGLM\lib\site-packages\starlette\responses.py", line 234, in listen_for_disconnect 2024-03-26 14:48:00 | ERROR | stderr | message = await receive() 2024-03-26 14:48:00 | ERROR | stderr | File "F:\Anaconda\envs\ChatGLM\lib\site-packages\uvicorn\protocols\http\h11_impl.py", line 538, in receive 2024-03-26 14:48:00 | ERROR | stderr | await self.message_event.wait() 2024-03-26 14:48:00 | ERROR | stderr | File "F:\Anaconda\envs\ChatGLM\lib\asyncio\locks.py", line 213, in wait 2024-03-26 14:48:00 | ERROR | stderr | await fut 2024-03-26 14:48:00 | ERROR | stderr | asyncio.exceptions.CancelledError: Cancelled by cancel scope 1bf60e592a0 2024-03-26 14:48:00 | ERROR | stderr | 2024-03-26 14:48:00 | ERROR | stderr | During handling of the above exception, another exception occurred: 2024-03-26 14:48:00 | ERROR | stderr | 2024-03-26 14:48:00 | ERROR | stderr | + Exception Group Traceback (most recent call last): 2024-03-26 14:48:00 | ERROR | stderr | | File "F:\Anaconda\envs\ChatGLM\lib\site-packages\uvicorn\protocols\http\h11_impl.py", line 408, in run_asgi 2024-03-26 14:48:00 | ERROR | stderr | | result = await app( # type: ignore[func-returns-value] 2024-03-26 14:48:00 | ERROR | stderr | | File "F:\Anaconda\envs\ChatGLM\lib\site-packages\uvicorn\middleware\proxy_headers.py", line 69, in call 2024-03-26 14:48:00 | ERROR | stderr | | return await self.app(scope, receive, send) 2024-03-26 14:48:00 | ERROR | stderr | | File "F:\Anaconda\envs\ChatGLM\lib\site-packages\fastapi\applications.py", line 1054, in call 2024-03-26 14:48:00 | ERROR | stderr | | await super().call(scope, receive, send) 2024-03-26 14:48:00 | ERROR | stderr | | File "F:\Anaconda\envs\ChatGLM\lib\site-packages\starlette\applications.py", line 119, in call 2024-03-26 14:48:00 | ERROR | stderr | | await self.middleware_stack(scope, receive, send) 2024-03-26 14:48:00 | ERROR | stderr | | File "F:\Anaconda\envs\ChatGLM\lib\site-packages\starlette\middleware\errors.py", line 186, in call 2024-03-26 14:48:00 | ERROR | stderr | | raise exc 2024-03-26 14:48:00 | ERROR | stderr | | File "F:\Anaconda\envs\ChatGLM\lib\site-packages\starlette\middleware\errors.py", line 164, in call 2024-03-26 14:48:00 | ERROR | stderr | | await self.app(scope, receive, _send) 2024-03-26 14:48:00 | ERROR | stderr | | File "F:\Anaconda\envs\ChatGLM\lib\site-packages\starlette\middleware\cors.py", line 83, in call 2024-03-26 14:48:00 | ERROR | stderr | | await self.app(scope, receive, send) 2024-03-26 14:48:00 | ERROR | stderr | | File "F:\Anaconda\envs\ChatGLM\lib\site-packages\starlette\middleware\exceptions.py", line 62, in call 2024-03-26 14:48:00 | ERROR | stderr | | await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send) 2024-03-26 14:48:00 | ERROR | stderr | | File "F:\Anaconda\envs\ChatGLM\lib\site-packages\starlette_exception_handler.py", line 64, in wrapped_app 2024-03-26 14:48:00 | ERROR | stderr | | raise exc 2024-03-26 14:48:00 | ERROR | stderr | | File "F:\Anaconda\envs\ChatGLM\lib\site-packages\starlette_exception_handler.py", line 53, in wrapped_app 2024-03-26 14:48:00 | ERROR | stderr | | await app(scope, receive, sender) 2024-03-26 14:48:00 | ERROR | stderr | | File "F:\Anaconda\envs\ChatGLM\lib\site-packages\starlette\routing.py", line 762, in call 2024-03-26 14:48:00 | ERROR | stderr | | await self.middleware_stack(scope, receive, send) 2024-03-26 14:48:00 | ERROR | stderr | | File "F:\Anaconda\envs\ChatGLM\lib\site-packages\starlette\routing.py", line 782, in app 2024-03-26 14:48:00 | ERROR | stderr | | await route.handle(scope, receive, send) 2024-03-26 14:48:00 | ERROR | stderr | | File "F:\Anaconda\envs\ChatGLM\lib\site-packages\starlette\routing.py", line 297, in handle 2024-03-26 14:48:00 | ERROR | stderr | | await self.app(scope, receive, send) 2024-03-26 14:48:00 | ERROR | stderr | | File "F:\Anaconda\envs\ChatGLM\lib\site-packages\starlette\routing.py", line 77, in app 2024-03-26 14:48:00 | ERROR | stderr | | await wrap_app_handling_exceptions(app, request)(scope, receive, send) 2024-03-26 14:48:00 | ERROR | stderr | | File "F:\Anaconda\envs\ChatGLM\lib\site-packages\starlette_exception_handler.py", line 64, in wrapped_app 2024-03-26 14:48:00 | ERROR | stderr | | raise exc 2024-03-26 14:48:00 | ERROR | stderr | | File "F:\Anaconda\envs\ChatGLM\lib\site-packages\starlette_exception_handler.py", line 53, in wrapped_app 2024-03-26 14:48:00 | ERROR | stderr | | await app(scope, receive, sender) 2024-03-26 14:48:00 | ERROR | stderr | | File "F:\Anaconda\envs\ChatGLM\lib\site-packages\starlette\routing.py", line 75, in app 2024-03-26 14:48:00 | ERROR | stderr | | await response(scope, receive, send) 2024-03-26 14:48:00 | ERROR | stderr | | File "F:\Anaconda\envs\ChatGLM\lib\site-packages\starlette\responses.py", line 254, in call 2024-03-26 14:48:00 | ERROR | stderr | | async with anyio.create_task_group() as task_group: 2024-03-26 14:48:00 | ERROR | stderr | | File "F:\Anaconda\envs\ChatGLM\lib\site-packages\anyio_backends_asyncio.py", line 678, in aexit 2024-03-26 14:48:00 | ERROR | stderr | | raise BaseExceptionGroup( 2024-03-26 14:48:00 | ERROR | stderr | | exceptiongroup.ExceptionGroup: unhandled errors in a TaskGroup (1 sub-exception) 2024-03-26 14:48:00 | ERROR | stderr | +-+---------------- 1 ---------------- 2024-03-26 14:48:00 | ERROR | stderr | | Traceback (most recent call last): 2024-03-26 14:48:00 | ERROR | stderr | | File "F:\Anaconda\envs\ChatGLM\lib\site-packages\httpx_transports\default.py", line 69, in map_httpcore_exceptions 2024-03-26 14:48:00 | ERROR | stderr | | yield 2024-03-26 14:48:00 | ERROR | stderr | | File "F:\Anaconda\envs\ChatGLM\lib\site-packages\httpx_transports\default.py", line 254, in aiter 2024-03-26 14:48:00 | ERROR | stderr | | async for part in self._httpcore_stream: 2024-03-26 14:48:00 | ERROR | stderr | | File "F:\Anaconda\envs\ChatGLM\lib\site-packages\httpcore_async\connection_pool.py", line 367, in aiter 2024-03-26 14:48:00 | ERROR | stderr | | raise exc from None 2024-03-26 14:48:00 | ERROR | stderr | | File "F:\Anaconda\envs\ChatGLM\lib\site-packages\httpcore_async\connection_pool.py", line 363, in aiter 2024-03-26 14:48:00 | ERROR | stderr | | async for part in self._stream: 2024-03-26 14:48:00 | ERROR | stderr | | File "F:\Anaconda\envs\ChatGLM\lib\site-packages\httpcore_async\http11.py", line 349, in aiter 2024-03-26 14:48:00 | ERROR | stderr | | raise exc 2024-03-26 14:48:00 | ERROR | stderr | | File "F:\Anaconda\envs\ChatGLM\lib\site-packages\httpcore_async\http11.py", line 341, in aiter 2024-03-26 14:48:00 | ERROR | stderr | | async for chunk in self._connection._receive_response_body(kwargs): 2024-03-26 14:48:00 | ERROR | stderr | | File "F:\Anaconda\envs\ChatGLM\lib\site-packages\httpcore_async\http11.py", line 210, in _receive_response_body 2024-03-26 14:48:00 | ERROR | stderr | | event = await self._receive_event(timeout=timeout) 2024-03-26 14:48:00 | ERROR | stderr | | File "F:\Anaconda\envs\ChatGLM\lib\site-packages\httpcore_async\http11.py", line 220, in _receive_event 2024-03-26 14:48:00 | ERROR | stderr | | with map_exceptions({h11.RemoteProtocolError: RemoteProtocolError}): 2024-03-26 14:48:00 | ERROR | stderr | | File "F:\Anaconda\envs\ChatGLM\lib\contextlib.py", line 153, in exit 2024-03-26 14:48:00 | ERROR | stderr | | self.gen.throw(typ, value, traceback) 2024-03-26 14:48:00 | ERROR | stderr | | File "F:\Anaconda\envs\ChatGLM\lib\site-packages\httpcore_exceptions.py", line 14, in map_exceptions 2024-03-26 14:48:00 | ERROR | stderr | | raise to_exc(exc) from exc 2024-03-26 14:48:00 | ERROR | stderr | | httpcore.RemoteProtocolError: peer closed connection without sending complete message body (incomplete chunked read) 2024-03-26 14:48:00 | ERROR | stderr | | 2024-03-26 14:48:00 | ERROR | stderr | | The above exception was the direct cause of the following exception: 2024-03-26 14:48:00 | ERROR | stderr | | 2024-03-26 14:48:00 | ERROR | stderr | | Traceback (most recent call last): 2024-03-26 14:48:00 | ERROR | stderr | | File "F:\Anaconda\envs\ChatGLM\lib\site-packages\starlette\responses.py", line 257, in wrap 2024-03-26 14:48:00 | ERROR | stderr | | await func() 2024-03-26 14:48:00 | ERROR | stderr | | File "F:\Anaconda\envs\ChatGLM\lib\site-packages\starlette\responses.py", line 246, in stream_response 2024-03-26 14:48:00 | ERROR | stderr | | async for chunk in self.body_iterator: 2024-03-26 14:48:00 | ERROR | stderr | | File "F:\Anaconda\envs\ChatGLM\lib\site-packages\fastchat\serve\openai_api_server.py", line 473, in chat_completion_stream_generator 2024-03-26 14:48:00 | ERROR | stderr | | async for content in generate_completion_stream(gen_params, worker_addr): 2024-03-26 14:48:00 | ERROR | stderr | | File "F:\Anaconda\envs\ChatGLM\lib\site-packages\fastchat\serve\openai_api_server.py", line 658, in generate_completion_stream 2024-03-26 14:48:00 | ERROR | stderr | | async for raw_chunk in response.aiter_raw(): 2024-03-26 14:48:00 | ERROR | stderr | | File "F:\Anaconda\envs\ChatGLM\lib\site-packages\httpx_models.py", line 987, in aiter_raw 2024-03-26 14:48:00 | ERROR | stderr | | async for raw_stream_bytes in self.stream: 2024-03-26 14:48:00 | ERROR | stderr | | File "F:\Anaconda\envs\ChatGLM\lib\site-packages\httpx_client.py", line 149, in aiter 2024-03-26 14:48:00 | ERROR | stderr | | async for chunk in self._stream: 2024-03-26 14:48:00 | ERROR | stderr | | File "F:\Anaconda\envs\ChatGLM\lib\site-packages\httpx_transports\default.py", line 253, in aiter 2024-03-26 14:48:00 | ERROR | stderr | | with map_httpcore_exceptions(): 2024-03-26 14:48:00 | ERROR | stderr | | File "F:\Anaconda\envs\ChatGLM\lib\contextlib.py", line 153, in exit 2024-03-26 14:48:00 | ERROR | stderr | | self.gen.throw(typ, value, traceback) 2024-03-26 14:48:00 | ERROR | stderr | | File "F:\Anaconda\envs\ChatGLM\lib\site-packages\httpx_transports\default.py", line 86, in map_httpcore_exceptions 2024-03-26 14:48:00 | ERROR | stderr | | raise mapped_exc(message) from exc 2024-03-26 14:48:00 | ERROR | stderr | | httpx.RemoteProtocolError: peer closed connection without sending complete message body (incomplete chunked read) 2024-03-26 14:48:00 | ERROR | stderr | +------------------------------------ 2024-03-26 14:48:00,141 - utils.py[line:38] - ERROR: peer closed connection without sending complete message body (incomplete chunked read) Traceback (most recent call last): File "F:\Anaconda\envs\ChatGLM\lib\site-packages\httpx_transports\default.py", line 69, in map_httpcore_exceptions yield File "F:\Anaconda\envs\ChatGLM\lib\site-packages\httpx_transports\default.py", line 254, in aiter async for part in self._httpcore_stream: File "F:\Anaconda\envs\ChatGLM\lib\site-packages\httpcore_async\connection_pool.py", line 367, in aiter raise exc from None File "F:\Anaconda\envs\ChatGLM\lib\site-packages\httpcore_async\connection_pool.py", line 363, in aiter async for part in self._stream: File "F:\Anaconda\envs\ChatGLM\lib\site-packages\httpcore_async\http11.py", line 349, in aiter raise exc File "F:\Anaconda\envs\ChatGLM\lib\site-packages\httpcore_async\http11.py", line 341, in aiter async for chunk in self._connection._receive_response_body(kwargs): File "F:\Anaconda\envs\ChatGLM\lib\site-packages\httpcore_async\http11.py", line 210, in _receive_response_body event = await self._receive_event(timeout=timeout) File "F:\Anaconda\envs\ChatGLM\lib\site-packages\httpcore_async\http11.py", line 220, in _receive_event with map_exceptions({h11.RemoteProtocolError: RemoteProtocolError}): File "F:\Anaconda\envs\ChatGLM\lib\contextlib.py", line 153, in exit self.gen.throw(typ, value, traceback) File "F:\Anaconda\envs\ChatGLM\lib\site-packages\httpcore_exceptions.py", line 14, in map_exceptions raise to_exc(exc) from exc httpcore.RemoteProtocolError: peer closed connection without sending complete message body (incomplete chunked read)

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "F:\ChatGLM\langchain-ChatGLM\server\utils.py", line 36, in wrap_done await fn File "F:\Anaconda\envs\ChatGLM\lib\site-packages\langchain\chains\base.py", line 385, in acall raise e File "F:\Anaconda\envs\ChatGLM\lib\site-packages\langchain\chains\base.py", line 379, in acall await self._acall(inputs, run_manager=run_manager) File "F:\Anaconda\envs\ChatGLM\lib\site-packages\langchain\chains\llm.py", line 275, in _acall response = await self.agenerate([inputs], run_manager=run_manager) File "F:\Anaconda\envs\ChatGLM\lib\site-packages\langchain\chains\llm.py", line 142, in agenerate return await self.llm.agenerate_prompt( File "F:\Anaconda\envs\ChatGLM\lib\site-packages\langchain_core\language_models\chat_models.py", line 554, in agenerate_prompt return await self.agenerate( File "F:\Anaconda\envs\ChatGLM\lib\site-packages\langchain_core\language_models\chat_models.py", line 514, in agenerate raise exceptions[0] File "F:\Anaconda\envs\ChatGLM\lib\site-packages\langchain_core\language_models\chat_models.py", line 617, in _agenerate_with_cache return await self._agenerate( File "F:\Anaconda\envs\ChatGLM\lib\site-packages\langchain_community\chat_models\openai.py", line 522, in _agenerate return await agenerate_from_stream(stream_iter) File "F:\Anaconda\envs\ChatGLM\lib\site-packages\langchain_core\language_models\chat_models.py", line 87, in agenerate_from_stream async for chunk in stream: File "F:\Anaconda\envs\ChatGLM\lib\site-packages\langchain_community\chat_models\openai.py", line 488, in _astream async for chunk in await acompletion_with_retry( File "F:\Anaconda\envs\ChatGLM\lib\site-packages\openai_streaming.py", line 116, in aiter async for item in self._iterator: File "F:\Anaconda\envs\ChatGLM\lib\site-packages\openai_streaming.py", line 129, in stream async for sse in iterator: File "F:\Anaconda\envs\ChatGLM\lib\site-packages\openai_streaming.py", line 120, in _iter_events async for sse in self._decoder.aiter(self.response.aiter_lines()): File "F:\Anaconda\envs\ChatGLM\lib\site-packages\openai_streaming.py", line 231, in aiter async for line in iterator: File "F:\Anaconda\envs\ChatGLM\lib\site-packages\httpx_models.py", line 963, in aiter_lines async for text in self.aiter_text(): File "F:\Anaconda\envs\ChatGLM\lib\site-packages\httpx_models.py", line 950, in aiter_text async for byte_content in self.aiter_bytes(): File "F:\Anaconda\envs\ChatGLM\lib\site-packages\httpx_models.py", line 929, in aiter_bytes async for raw_bytes in self.aiter_raw(): File "F:\Anaconda\envs\ChatGLM\lib\site-packages\httpx_models.py", line 987, in aiter_raw async for raw_stream_bytes in self.stream: File "F:\Anaconda\envs\ChatGLM\lib\site-packages\httpx_client.py", line 149, in aiter async for chunk in self._stream: File "F:\Anaconda\envs\ChatGLM\lib\site-packages\httpx_transports\default.py", line 253, in aiter with map_httpcore_exceptions(): File "F:\Anaconda\envs\ChatGLM\lib\contextlib.py", line 153, in exit self.gen.throw(typ, value, traceback) File "F:\Anaconda\envs\ChatGLM\lib\site-packages\httpx_transports\default.py", line 86, in map_httpcore_exceptions raise mapped_exc(message) from exc httpx.RemoteProtocolError: peer closed connection without sending complete message body (incomplete chunked read) 2024-03-26 14:48:00,148 - utils.py[line:40] - ERROR: RemoteProtocolError: Caught exception: peer closed connection without sending complete message body (incomplete chunked read)

嗨，@488283943！很高兴再次看到你提出问题。希望这次我们也能一起找到解决办法。

您遇到的问题可能是由于以下原因：

httpx.RemoteProtocolError：这个错误通常发生在客户端和服务器之间的HTTP协议通信出现问题时。根据上下文，它可能与服务器的设置方式或请求处理方式有关。它也可能表示网络问题或与HTTP客户端库的兼容性问题。
AttributeError：当您的代码试图访问一个对象没有的属性时，就会发生这个错误。没有关于这个错误发生在哪里的具体细节，很难确定确切的原因。然而，它可能与对象的错误初始化或缺少导入有关。
asyncio.exceptions.CancelledError：当一个asyncio任务被取消时，就会引发这个错误。这可能是由于异步操作的不正确处理，或者任务由于超时或代码的其他部分显式取消而被取消。确保正确管理异步任务，并且没有意外的取消。
RuntimeError: CUDA out of memory：这个错误表明GPU内存不足以执行您试图执行的操作。考虑到您正在使用Langchain-Chatchat，这可能涉及到大型语言模型，这并不奇怪。错误表明您正在加载的模型或您正在执行的操作需要的GPU内存超过了可用的内存。命令行选项--max-gpu-memory 10GiB表明试图限制GPU内存使用，但可能仍然不足以满足您的需求，或者该设置可能没有像预期的那样有效地限制内存使用。

要解决这些问题：

对于httpx.RemoteProtocolError，确保您的服务器设置正确，没有网络问题。同时，验证与HTTP客户端库的兼容性。
对于AttributeError，查看堆栈跟踪以确定涉及的对象和属性，然后确保进行了所有必要的导入，并正确初始化了对象。
对于asyncio.exceptions.CancelledError，查看您的异步代码，确保任务被正确管理。查找任何不必要的任务取消，并确保正确处理错误。
对于RuntimeError: CUDA out of memory，考虑减小您正在加载的模型的大小或您正在使用的批量大小。如果可能，通过调整--num-gpus和--gpus选项使用更多的GPU。您可能还需要确保您的代码或您正在使用的库尊重--max-gpu-memory设置。

没有每个错误在代码中发生的地方的具体细节，这些都是一般性的建议。您可能需要在您的应用程序的特定使用和配置的上下文中调查每个错误。

另外，我在Langchain-Chatchat的存储库中找到了一些类似的问题，可能会对您有所帮助：

希望这些信息能帮助您解决问题。如果您需要更多的帮助，或者有更多的信息可以提供，欢迎您继续在这个问题下面留言。

Sources

#### About Dosu

This response is meant to be useful and save you time. It is **not meant to be a precise solution**, but rather a starting point for your own research. Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant. If you want to continue the conversation, start your reply with @dosu-bot.

这个问题已经被标记为 stale ，因为它已经超过 30 天没有任何活动。

LLM模型和embedding模型正常加载，能跳转至web页面，使用本地构建的知识库可以进行正常询问回答，但是询问三句左右就无法正常进行答复，重启模型或者重新加载web界面可以正常询问，我想知道是哪方面的问题

==============================Langchain-Chatchat Configuration============================== 操作系统：Windows-10-10.0.14393-SP0. python版本：3.10.0 | packaged by conda-forge | (default, Nov 10 2021, 13:20:59) [MSC v.1916 64 bit (AMD64)] 项目版本：v0.2.10 langchain版本：0.0.354. fastchat版本：0.2.35 当前使用的分词器：ChineseRecursiveTextSplitter 当前启动的LLM模型：['chatglm3-6b'] @ cuda {'device': 'cuda', 'host': '127.0.0.1', 'infer_turbo': False, 'model_path': 'F:\ChatGLM\langchain-ChatGLM\models\chatglm3-6b', 'model_path_exists': True, 'port': 20002} 当前Embbedings模型： text2vec @ cuda ==============================Langchain-Chatchat Configuration============================== 2024-03-26 14:44:07,315 - startup.py[line:655] - INFO: 正在启动服务： 2024-03-26 14:44:07,315 - startup.py[line:656] - INFO: 如需查看 llm_api 日志，请前往 F:\ChatGLM\langchain-ChatGLM\logs F:\Anaconda\envs\ChatGLM\lib\site-packages\langchain_core_api\deprecation.py:117: LangChainDeprecationWarning: 模型启动功能将于 Langchain-Chatchat 0.3.x重写,支持更多模式和加速启动，0.2.x中相关功能将废弃 warn_deprecated( 2024-03-26 14:44:16 | ERROR | stderr | INFO: Started server process [3532] 2024-03-26 14:44:16 | ERROR | stderr | INFO: Waiting for application startup. 2024-03-26 14:44:16 | ERROR | stderr | INFO: Application startup complete. 2024-03-26 14:44:16 | ERROR | stderr | INFO: Uvicorn running on http://127.0.0.1:20000 (Press CTRL+C to quit) 2024-03-26 14:44:17 | INFO | model_worker | Loading the model ['chatglm3-6b'] on worker 0f6e17ed ... 2024-03-26 14:44:17 | WARNING | transformers_modules.chatglm3-6b.tokenization_chatglm | Setting eos_token is not supported, use the default one. 2024-03-26 14:44:17 | WARNING | transformers_modules.chatglm3-6b.tokenization_chatglm | Setting pad_token is not supported, use the default one. 2024-03-26 14:44:17 | WARNING | transformers_modules.chatglm3-6b.tokenization_chatglm | Setting unk_token is not supported, use the default one. Loading checkpoint shards: 0%| | 0/7 [00:00<?, ?it/s] Loading checkpoint shards: 14%|██████████ | 1/7 [00:00<00:04, 1.20it/s] Loading checkpoint shards: 29%|████████████████████ | 2/7 [00:01<00:04, 1.04it/s] Loading checkpoint shards: 43%|██████████████████████████████ | 3/7 [00:02<00:04, 1.00s/it] Loading checkpoint shards: 57%|████████████████████████████████████████ | 4/7 [00:03<00:03, 1.03s/it] Loading checkpoint shards: 71%|██████████████████████████████████████████████████ | 5/7 [00:05<00:02, 1.07s/it] Loading checkpoint shards: 86%|██████████████████████████████████████████████████ ██████████ | 6/7 [00:06<00:01, 1.11s/it] Loading checkpoint shards: 100%|██████████████████████████████████████████████████ ████████████████████| 7/7 [00:07<00:00, 1.01it/s] Loading checkpoint shards: 100%|██████████████████████████████████████████████████ ████████████████████| 7/7 [00:07<00:00, 1.01s/it] 2024-03-26 14:44:25 | ERROR | stderr | 2024-03-26 14:44:30 | INFO | model_worker | Register to controller INFO: Started server process [9240] INFO: Waiting for application startup. INFO: Application startup complete. INFO: Uvicorn running on http://127.0.0.1:7861 (Press CTRL+C to quit) =============================Langchain-Chatchat Configuration============================== 操作系统：Windows-10-10.0.14393-SP0. python版本：3.10.0 | packaged by conda-forge | (default, Nov 10 2021, 13:20:59) [MSC v.1916 64 bit (AMD64)] 项目版本：v0.2.10 langchain版本：0.0.354. fastchat版本：0.2.35 当前使用的分词器：ChineseRecursiveTextSplitter 当前启动的LLM模型：['chatglm3-6b'] @ cuda {'device': 'cuda', 'host': '127.0.0.1', 'infer_turbo': False, 'model_path': 'F:\ChatGLM\langchain-ChatGLM\models\chatglm3-6b', 'model_path_exists': True, 'port': 20002} 当前Embbedings模型： text2vec @ cuda 服务端运行信息： OpenAI API Server: http://127.0.0.1:20000/v1 Chatchat API Server: http://127.0.0.1:7861 Chatchat WEBUI Server: http://127.0.0.1:8501 ==============================Langchain-Chatchat Configuration============================== You can now view your Streamlit app in your browser. URL: http://127.0.0.1:8501

下面是报错部分的log：

2024-03-26 14:47:58 | INFO | stdout | INFO: 127.0.0.1:56672 - "POST /v1/chat/completions HTTP/1.1" 200 OK 2024-03-26 14:47:58,862 - _client.py[line:1773] - INFO: HTTP Request: POST http://127.0.0.1:20000/v1/chat/completions "HTTP/1.1 200 OK" 2024-03-26 14:47:58 | INFO | httpx | HTTP Request: POST http://127.0.0.1:20002/worker_generate_stream "HTTP/1.1 200 OK" ERROR: Exception in ASGI application Traceback (most recent call last): File "F:\Anaconda\envs\ChatGLM\lib\site-packages\starlette\responses.py", line 261, in call await wrap(partial(self.listen_for_disconnect, receive)) File "F:\Anaconda\envs\ChatGLM\lib\site-packages\starlette\responses.py", line 257, in wrap await func() File "F:\Anaconda\envs\ChatGLM\lib\site-packages\starlette\responses.py", line 234, in listen_for_disconnect message = await receive() File "F:\Anaconda\envs\ChatGLM\lib\site-packages\uvicorn\protocols\http\h11_impl.py", line 538, in receive await self.message_event.wait() File "F:\Anaconda\envs\ChatGLM\lib\asyncio\locks.py", line 213, in wait await fut asyncio.exceptions.CancelledError: Cancelled by cancel scope 188000f6860

During handling of the above exception, another exception occurred:

Exception Group Traceback (most recent call last): File "F:\Anaconda\envs\ChatGLM\lib\site-packages\uvicorn\protocols\http\h11_impl.py", line 408, in run_asgi result = await app( # type: ignore[func-returns-value] File "F:\Anaconda\envs\ChatGLM\lib\site-packages\uvicorn\middleware\proxy_headers.py", line 69, in call return await self.app(scope, receive, send) File "F:\Anaconda\envs\ChatGLM\lib\site-packages\fastapi\applications.py", line 1054, in call await super().call(scope, receive, send) File "F:\Anaconda\envs\ChatGLM\lib\site-packages\starlette\applications.py", line 119, in call await self.middleware_stack(scope, receive, send) File "F:\Anaconda\envs\ChatGLM\lib\site-packages\starlette\middleware\errors.py", line 186, in call raise exc File "F:\Anaconda\envs\ChatGLM\lib\site-packages\starlette\middleware\errors.py", line 164, in call await self.app(scope, receive, _send) File "F:\Anaconda\envs\ChatGLM\lib\site-packages\starlette\middleware\exceptions.py", line 62, in call await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send) File "F:\Anaconda\envs\ChatGLM\lib\site-packages\starlette_exception_handler.py", line 64, in wrapped_app raise exc File "F:\Anaconda\envs\ChatGLM\lib\site-packages\starlette_exception_handler.py", line 53, in wrapped_app await app(scope, receive, sender) File "F:\Anaconda\envs\ChatGLM\lib\site-packages\starlette\routing.py", line 762, in call await self.middleware_stack(scope, receive, send) File "F:\Anaconda\envs\ChatGLM\lib\site-packages\starlette\routing.py", line 782, in app await route.handle(scope, receive, send) File "F:\Anaconda\envs\ChatGLM\lib\site-packages\starlette\routing.py", line 297, in handle await self.app(scope, receive, send) File "F:\Anaconda\envs\ChatGLM\lib\site-packages\starlette\routing.py", line 77, in app await wrap_app_handling_exceptions(app, request)(scope, receive, send) File "F:\Anaconda\envs\ChatGLM\lib\site-packages\starlette_exception_handler.py", line 64, in wrapped_app raise exc File "F:\Anaconda\envs\ChatGLM\lib\site-packages\starlette_exception_handler.py", line 53, in wrapped_app await app(scope, receive, sender) File "F:\Anaconda\envs\ChatGLM\lib\site-packages\starlette\routing.py", line 75, in app await response(scope, receive, send) File "F:\Anaconda\envs\ChatGLM\lib\site-packages\starlette\responses.py", line 254, in call async with anyio.create_task_group() as task_group: File "F:\Anaconda\envs\ChatGLM\lib\site-packages\anyio_backends_asyncio.py", line 678, in aexit raise BaseExceptionGroup( exceptiongroup.ExceptionGroup: unhandled errors in a TaskGroup (1 sub-exception) +-+---------------- 1 ---------------- Traceback (most recent call last): File "F:\Anaconda\envs\ChatGLM\lib\site-packages\fastchat\serve\model_worker.py", line 114, in generate_stream_gate for output in self.generate_stream_func( File "F:\Anaconda\envs\ChatGLM\lib\site-packages\torch\autograd\grad_mode.py", line 43, in generator_context response = gen.send(None) File "F:\Anaconda\envs\ChatGLM\lib\site-packages\fastchat\model\model_chatglm.py", line 106, in generate_stream_chatglm for total_ids in model.stream_generate(inputs, gen_kwargs): File "F:\Anaconda\envs\ChatGLM\lib\site-packages\torch\autograd\grad_mode.py", line 43, in generator_context response = gen.send(None) File "C:\Users\Administrator/.cache\huggingface\modules\transformers_modules\chatglm3-6b\modeling_chatglm.py", line 1163, in stream_generate outputs = self( File "F:\Anaconda\envs\ChatGLM\lib\site-packages\torch\nn\modules\module.py", line 1130, in _call_impl return forward_call(*input, **kwargs) File "C:\Users\Administrator/.cache\huggingface\modules\transformers_modules\chatglm3-6b\modeling_chatglm.py", line 941, in forward transformer_outputs = self.transformer( File "F:\Anaconda\envs\ChatGLM\lib\site-packages\torch\nn\modules\module.py", line 1130, in _call_impl return forward_call(*input, **kwargs) File "C:\Users\Administrator/.cache\huggingface\modules\transformers_modules\chatglm3-6b\modeling_chatglm.py", line 834, in forward hidden_states, presents, all_hidden_states, all_self_attentions = self.encoder( File "F:\Anaconda\envs\ChatGLM\lib\site-packages\torch\nn\modules\module.py", line 1130, in _call_impl return forward_call(*input, **kwargs) File "C:\Users\Administrator/.cache\huggingface\modules\transformers_modules\chatglm3-6b\modeling_chatglm.py", line 641, in forward layer_ret = layer( File "F:\Anaconda\envs\ChatGLM\lib\site-packages\torch\nn\modules\module.py", line 1130, in _call_impl return forward_call(*input, **kwargs) File "C:\Users\Administrator/.cache\huggingface\modules\transformers_modules\chatglm3-6b\modeling_chatglm.py", line 544, in forward attention_output, kv_cache = self.self_attention( File "F:\Anaconda\envs\ChatGLM\lib\site-packages\torch\nn\modules\module.py", line 1130, in _call_impl return forward_call(*input, **kwargs) File "C:\Users\Administrator/.cache\huggingface\modules\transformers_modules\chatglm3-6b\modeling_chatglm.py", line 441, in forward context_layer = self.core_attention(query_layer, key_layer, value_layer, attention_mask) File "F:\Anaconda\envs\ChatGLM\lib\site-packages\torch\nn\modules\module.py", line 1130, in _call_impl return forward_call(*input, **kwargs) File "C:\Users\Administrator/.cache\huggingface\modules\transformers_modules\chatglm3-6b\modeling_chatglm.py", line 273, in forward attention_scores = attention_scores * self.coeff RuntimeError: CUDA out of memory. Tried to allocate 610.00 MiB (GPU 0; 15.91 GiB total capacity; 12.95 GiB already allocated; 358.19 MiB free; 13.04 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

During handling of the above exception, another exception occurred:

Traceback (most recent call last):

File "F:\Anaconda\envs\ChatGLM\lib\site-packages\starlette\responses.py", line 257, in wrap

await func()

File "F:\Anaconda\envs\ChatGLM\lib\site-packages\starlette\responses.py", line 246, in stream_response

async for chunk in self.body_iterator:

File "F:\Anaconda\envs\ChatGLM\lib\site-packages\starlette\concurrency.py", line 63, in iterate_in_threadpool

yield await anyio.to_thread.run_sync(_next, as_iterator)

File "F:\Anaconda\envs\ChatGLM\lib\site-packages\anyio\to_thread.py", line 56, in run_sync

return await get_async_backend().run_sync_in_worker_thread(

File "F:\Anaconda\envs\ChatGLM\lib\site-packages\anyio_backends_asyncio.py", line 2144, in run_sync_in_worker_thread

return await future

File "F:\Anaconda\envs\ChatGLM\lib\site-packages\anyio_backends_asyncio.py", line 851, in run

result = context.run(func, *args)

File "F:\Anaconda\envs\ChatGLM\lib\site-packages\starlette\concurrency.py", line 52, in _next

return next(iterator)

File "F:\Anaconda\envs\ChatGLM\lib\site-packages\fastchat\serve\model_worker.py", line 133, in generate_stream_gate

except torch.cuda.OutOfMemoryError as e:

AttributeError: module 'torch.cuda' has no attribute 'OutOfMemoryError'

+------------------------------------ 2024-03-26 14:48:00 | ERROR | stderr | ERROR: Exception in ASGI application 2024-03-26 14:48:00 | ERROR | stderr | Traceback (most recent call last): 2024-03-26 14:48:00 | ERROR | stderr | File "F:\Anaconda\envs\ChatGLM\lib\site-packages\starlette\responses.py", line 261, in call 2024-03-26 14:48:00 | ERROR | stderr | await wrap(partial(self.listen_for_disconnect, receive)) 2024-03-26 14:48:00 | ERROR | stderr | File "F:\Anaconda\envs\ChatGLM\lib\site-packages\starlette\responses.py", line 257, in wrap 2024-03-26 14:48:00 | ERROR | stderr | await func() 2024-03-26 14:48:00 | ERROR | stderr | File "F:\Anaconda\envs\ChatGLM\lib\site-packages\starlette\responses.py", line 234, in listen_for_disconnect 2024-03-26 14:48:00 | ERROR | stderr | message = await receive() 2024-03-26 14:48:00 | ERROR | stderr | File "F:\Anaconda\envs\ChatGLM\lib\site-packages\uvicorn\protocols\http\h11_impl.py", line 538, in receive 2024-03-26 14:48:00 | ERROR | stderr | await self.message_event.wait() 2024-03-26 14:48:00 | ERROR | stderr | File "F:\Anaconda\envs\ChatGLM\lib\asyncio\locks.py", line 213, in wait 2024-03-26 14:48:00 | ERROR | stderr | await fut 2024-03-26 14:48:00 | ERROR | stderr | asyncio.exceptions.CancelledError: Cancelled by cancel scope 1bf60e592a0 2024-03-26 14:48:00 | ERROR | stderr | 2024-03-26 14:48:00 | ERROR | stderr | During handling of the above exception, another exception occurred: 2024-03-26 14:48:00 | ERROR | stderr | 2024-03-26 14:48:00 | ERROR | stderr | + Exception Group Traceback (most recent call last): 2024-03-26 14:48:00 | ERROR | stderr | | File "F:\Anaconda\envs\ChatGLM\lib\site-packages\uvicorn\protocols\http\h11_impl.py", line 408, in run_asgi 2024-03-26 14:48:00 | ERROR | stderr | | result = await app( # type: ignore[func-returns-value] 2024-03-26 14:48:00 | ERROR | stderr | | File "F:\Anaconda\envs\ChatGLM\lib\site-packages\uvicorn\middleware\proxy_headers.py", line 69, in call 2024-03-26 14:48:00 | ERROR | stderr | | return await self.app(scope, receive, send) 2024-03-26 14:48:00 | ERROR | stderr | | File "F:\Anaconda\envs\ChatGLM\lib\site-packages\fastapi\applications.py", line 1054, in call 2024-03-26 14:48:00 | ERROR | stderr | | await super().call(scope, receive, send) 2024-03-26 14:48:00 | ERROR | stderr | | File "F:\Anaconda\envs\ChatGLM\lib\site-packages\starlette\applications.py", line 119, in call 2024-03-26 14:48:00 | ERROR | stderr | | await self.middleware_stack(scope, receive, send) 2024-03-26 14:48:00 | ERROR | stderr | | File "F:\Anaconda\envs\ChatGLM\lib\site-packages\starlette\middleware\errors.py", line 186, in call 2024-03-26 14:48:00 | ERROR | stderr | | raise exc 2024-03-26 14:48:00 | ERROR | stderr | | File "F:\Anaconda\envs\ChatGLM\lib\site-packages\starlette\middleware\errors.py", line 164, in call 2024-03-26 14:48:00 | ERROR | stderr | | await self.app(scope, receive, _send) 2024-03-26 14:48:00 | ERROR | stderr | | File "F:\Anaconda\envs\ChatGLM\lib\site-packages\starlette\middleware\cors.py", line 83, in call 2024-03-26 14:48:00 | ERROR | stderr | | await self.app(scope, receive, send) 2024-03-26 14:48:00 | ERROR | stderr | | File "F:\Anaconda\envs\ChatGLM\lib\site-packages\starlette\middleware\exceptions.py", line 62, in call 2024-03-26 14:48:00 | ERROR | stderr | | await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send) 2024-03-26 14:48:00 | ERROR | stderr | | File "F:\Anaconda\envs\ChatGLM\lib\site-packages\starlette_exception_handler.py", line 64, in wrapped_app 2024-03-26 14:48:00 | ERROR | stderr | | raise exc 2024-03-26 14:48:00 | ERROR | stderr | | File "F:\Anaconda\envs\ChatGLM\lib\site-packages\starlette_exception_handler.py", line 53, in wrapped_app 2024-03-26 14:48:00 | ERROR | stderr | | await app(scope, receive, sender) 2024-03-26 14:48:00 | ERROR | stderr | | File "F:\Anaconda\envs\ChatGLM\lib\site-packages\starlette\routing.py", line 762, in call 2024-03-26 14:48:00 | ERROR | stderr | | await self.middleware_stack(scope, receive, send) 2024-03-26 14:48:00 | ERROR | stderr | | File "F:\Anaconda\envs\ChatGLM\lib\site-packages\starlette\routing.py", line 782, in app 2024-03-26 14:48:00 | ERROR | stderr | | await route.handle(scope, receive, send) 2024-03-26 14:48:00 | ERROR | stderr | | File "F:\Anaconda\envs\ChatGLM\lib\site-packages\starlette\routing.py", line 297, in handle 2024-03-26 14:48:00 | ERROR | stderr | | await self.app(scope, receive, send) 2024-03-26 14:48:00 | ERROR | stderr | | File "F:\Anaconda\envs\ChatGLM\lib\site-packages\starlette\routing.py", line 77, in app 2024-03-26 14:48:00 | ERROR | stderr | | await wrap_app_handling_exceptions(app, request)(scope, receive, send) 2024-03-26 14:48:00 | ERROR | stderr | | File "F:\Anaconda\envs\ChatGLM\lib\site-packages\starlette_exception_handler.py", line 64, in wrapped_app 2024-03-26 14:48:00 | ERROR | stderr | | raise exc 2024-03-26 14:48:00 | ERROR | stderr | | File "F:\Anaconda\envs\ChatGLM\lib\site-packages\starlette_exception_handler.py", line 53, in wrapped_app 2024-03-26 14:48:00 | ERROR | stderr | | await app(scope, receive, sender) 2024-03-26 14:48:00 | ERROR | stderr | | File "F:\Anaconda\envs\ChatGLM\lib\site-packages\starlette\routing.py", line 75, in app 2024-03-26 14:48:00 | ERROR | stderr | | await response(scope, receive, send) 2024-03-26 14:48:00 | ERROR | stderr | | File "F:\Anaconda\envs\ChatGLM\lib\site-packages\starlette\responses.py", line 254, in call 2024-03-26 14:48:00 | ERROR | stderr | | async with anyio.create_task_group() as task_group: 2024-03-26 14:48:00 | ERROR | stderr | | File "F:\Anaconda\envs\ChatGLM\lib\site-packages\anyio_backends_asyncio.py", line 678, in aexit 2024-03-26 14:48:00 | ERROR | stderr | | raise BaseExceptionGroup( 2024-03-26 14:48:00 | ERROR | stderr | | exceptiongroup.ExceptionGroup: unhandled errors in a TaskGroup (1 sub-exception) 2024-03-26 14:48:00 | ERROR | stderr | +-+---------------- 1 ---------------- 2024-03-26 14:48:00 | ERROR | stderr | | Traceback (most recent call last): 2024-03-26 14:48:00 | ERROR | stderr | | File "F:\Anaconda\envs\ChatGLM\lib\site-packages\httpx_transports\default.py", line 69, in map_httpcore_exceptions 2024-03-26 14:48:00 | ERROR | stderr | | yield 2024-03-26 14:48:00 | ERROR | stderr | | File "F:\Anaconda\envs\ChatGLM\lib\site-packages\httpx_transports\default.py", line 254, in aiter 2024-03-26 14:48:00 | ERROR | stderr | | async for part in self._httpcore_stream: 2024-03-26 14:48:00 | ERROR | stderr | | File "F:\Anaconda\envs\ChatGLM\lib\site-packages\httpcore_async\connection_pool.py", line 367, in aiter 2024-03-26 14:48:00 | ERROR | stderr | | raise exc from None 2024-03-26 14:48:00 | ERROR | stderr | | File "F:\Anaconda\envs\ChatGLM\lib\site-packages\httpcore_async\connection_pool.py", line 363, in aiter 2024-03-26 14:48:00 | ERROR | stderr | | async for part in self._stream: 2024-03-26 14:48:00 | ERROR | stderr | | File "F:\Anaconda\envs\ChatGLM\lib\site-packages\httpcore_async\http11.py", line 349, in aiter 2024-03-26 14:48:00 | ERROR | stderr | | raise exc 2024-03-26 14:48:00 | ERROR | stderr | | File "F:\Anaconda\envs\ChatGLM\lib\site-packages\httpcore_async\http11.py", line 341, in aiter 2024-03-26 14:48:00 | ERROR | stderr | | async for chunk in self._connection._receive_response_body(kwargs): 2024-03-26 14:48:00 | ERROR | stderr | | File "F:\Anaconda\envs\ChatGLM\lib\site-packages\httpcore_async\http11.py", line 210, in _receive_response_body 2024-03-26 14:48:00 | ERROR | stderr | | event = await self._receive_event(timeout=timeout) 2024-03-26 14:48:00 | ERROR | stderr | | File "F:\Anaconda\envs\ChatGLM\lib\site-packages\httpcore_async\http11.py", line 220, in _receive_event 2024-03-26 14:48:00 | ERROR | stderr | | with map_exceptions({h11.RemoteProtocolError: RemoteProtocolError}): 2024-03-26 14:48:00 | ERROR | stderr | | File "F:\Anaconda\envs\ChatGLM\lib\contextlib.py", line 153, in exit 2024-03-26 14:48:00 | ERROR | stderr | | self.gen.throw(typ, value, traceback) 2024-03-26 14:48:00 | ERROR | stderr | | File "F:\Anaconda\envs\ChatGLM\lib\site-packages\httpcore_exceptions.py", line 14, in map_exceptions 2024-03-26 14:48:00 | ERROR | stderr | | raise to_exc(exc) from exc 2024-03-26 14:48:00 | ERROR | stderr | | httpcore.RemoteProtocolError: peer closed connection without sending complete message body (incomplete chunked read) 2024-03-26 14:48:00 | ERROR | stderr | | 2024-03-26 14:48:00 | ERROR | stderr | | The above exception was the direct cause of the following exception: 2024-03-26 14:48:00 | ERROR | stderr | | 2024-03-26 14:48:00 | ERROR | stderr | | Traceback (most recent call last): 2024-03-26 14:48:00 | ERROR | stderr | | File "F:\Anaconda\envs\ChatGLM\lib\site-packages\starlette\responses.py", line 257, in wrap 2024-03-26 14:48:00 | ERROR | stderr | | await func() 2024-03-26 14:48:00 | ERROR | stderr | | File "F:\Anaconda\envs\ChatGLM\lib\site-packages\starlette\responses.py", line 246, in stream_response 2024-03-26 14:48:00 | ERROR | stderr | | async for chunk in self.body_iterator: 2024-03-26 14:48:00 | ERROR | stderr | | File "F:\Anaconda\envs\ChatGLM\lib\site-packages\fastchat\serve\openai_api_server.py", line 473, in chat_completion_stream_generator 2024-03-26 14:48:00 | ERROR | stderr | | async for content in generate_completion_stream(gen_params, worker_addr): 2024-03-26 14:48:00 | ERROR | stderr | | File "F:\Anaconda\envs\ChatGLM\lib\site-packages\fastchat\serve\openai_api_server.py", line 658, in generate_completion_stream 2024-03-26 14:48:00 | ERROR | stderr | | async for raw_chunk in response.aiter_raw(): 2024-03-26 14:48:00 | ERROR | stderr | | File "F:\Anaconda\envs\ChatGLM\lib\site-packages\httpx_models.py", line 987, in aiter_raw 2024-03-26 14:48:00 | ERROR | stderr | | async for raw_stream_bytes in self.stream: 2024-03-26 14:48:00 | ERROR | stderr | | File "F:\Anaconda\envs\ChatGLM\lib\site-packages\httpx_client.py", line 149, in aiter 2024-03-26 14:48:00 | ERROR | stderr | | async for chunk in self._stream: 2024-03-26 14:48:00 | ERROR | stderr | | File "F:\Anaconda\envs\ChatGLM\lib\site-packages\httpx_transports\default.py", line 253, in aiter 2024-03-26 14:48:00 | ERROR | stderr | | with map_httpcore_exceptions(): 2024-03-26 14:48:00 | ERROR | stderr | | File "F:\Anaconda\envs\ChatGLM\lib\contextlib.py", line 153, in exit 2024-03-26 14:48:00 | ERROR | stderr | | self.gen.throw(typ, value, traceback) 2024-03-26 14:48:00 | ERROR | stderr | | File "F:\Anaconda\envs\ChatGLM\lib\site-packages\httpx_transports\default.py", line 86, in map_httpcore_exceptions 2024-03-26 14:48:00 | ERROR | stderr | | raise mapped_exc(message) from exc 2024-03-26 14:48:00 | ERROR | stderr | | httpx.RemoteProtocolError: peer closed connection without sending complete message body (incomplete chunked read) 2024-03-26 14:48:00 | ERROR | stderr | +------------------------------------ 2024-03-26 14:48:00,141 - utils.py[line:38] - ERROR: peer closed connection without sending complete message body (incomplete chunked read) Traceback (most recent call last): File "F:\Anaconda\envs\ChatGLM\lib\site-packages\httpx_transports\default.py", line 69, in map_httpcore_exceptions yield File "F:\Anaconda\envs\ChatGLM\lib\site-packages\httpx_transports\default.py", line 254, in aiter async for part in self._httpcore_stream: File "F:\Anaconda\envs\ChatGLM\lib\site-packages\httpcore_async\connection_pool.py", line 367, in aiter raise exc from None File "F:\Anaconda\envs\ChatGLM\lib\site-packages\httpcore_async\connection_pool.py", line 363, in aiter async for part in self._stream: File "F:\Anaconda\envs\ChatGLM\lib\site-packages\httpcore_async\http11.py", line 349, in aiter raise exc File "F:\Anaconda\envs\ChatGLM\lib\site-packages\httpcore_async\http11.py", line 341, in aiter async for chunk in self._connection._receive_response_body(kwargs): File "F:\Anaconda\envs\ChatGLM\lib\site-packages\httpcore_async\http11.py", line 210, in _receive_response_body event = await self._receive_event(timeout=timeout) File "F:\Anaconda\envs\ChatGLM\lib\site-packages\httpcore_async\http11.py", line 220, in _receive_event with map_exceptions({h11.RemoteProtocolError: RemoteProtocolError}): File "F:\Anaconda\envs\ChatGLM\lib\contextlib.py", line 153, in exit self.gen.throw(typ, value, traceback) File "F:\Anaconda\envs\ChatGLM\lib\site-packages\httpcore_exceptions.py", line 14, in map_exceptions raise to_exc(exc) from exc httpcore.RemoteProtocolError: peer closed connection without sending complete message body (incomplete chunked read)

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "F:\ChatGLM\langchain-ChatGLM\server\utils.py", line 36, in wrap_done await fn File "F:\Anaconda\envs\ChatGLM\lib\site-packages\langchain\chains\base.py", line 385, in acall raise e File "F:\Anaconda\envs\ChatGLM\lib\site-packages\langchain\chains\base.py", line 379, in acall await self._acall(inputs, run_manager=run_manager) File "F:\Anaconda\envs\ChatGLM\lib\site-packages\langchain\chains\llm.py", line 275, in _acall response = await self.agenerate([inputs], run_manager=run_manager) File "F:\Anaconda\envs\ChatGLM\lib\site-packages\langchain\chains\llm.py", line 142, in agenerate return await self.llm.agenerate_prompt( File "F:\Anaconda\envs\ChatGLM\lib\site-packages\langchain_core\language_models\chat_models.py", line 554, in agenerate_prompt return await self.agenerate( File "F:\Anaconda\envs\ChatGLM\lib\site-packages\langchain_core\language_models\chat_models.py", line 514, in agenerate raise exceptions[0] File "F:\Anaconda\envs\ChatGLM\lib\site-packages\langchain_core\language_models\chat_models.py", line 617, in _agenerate_with_cache return await self._agenerate( File "F:\Anaconda\envs\ChatGLM\lib\site-packages\langchain_community\chat_models\openai.py", line 522, in _agenerate return await agenerate_from_stream(stream_iter) File "F:\Anaconda\envs\ChatGLM\lib\site-packages\langchain_core\language_models\chat_models.py", line 87, in agenerate_from_stream async for chunk in stream: File "F:\Anaconda\envs\ChatGLM\lib\site-packages\langchain_community\chat_models\openai.py", line 488, in _astream async for chunk in await acompletion_with_retry( File "F:\Anaconda\envs\ChatGLM\lib\site-packages\openai_streaming.py", line 116, in aiter async for item in self._iterator: File "F:\Anaconda\envs\ChatGLM\lib\site-packages\openai_streaming.py", line 129, in stream async for sse in iterator: File "F:\Anaconda\envs\ChatGLM\lib\site-packages\openai_streaming.py", line 120, in _iter_events async for sse in self._decoder.aiter(self.response.aiter_lines()): File "F:\Anaconda\envs\ChatGLM\lib\site-packages\openai_streaming.py", line 231, in aiter async for line in iterator: File "F:\Anaconda\envs\ChatGLM\lib\site-packages\httpx_models.py", line 963, in aiter_lines async for text in self.aiter_text(): File "F:\Anaconda\envs\ChatGLM\lib\site-packages\httpx_models.py", line 950, in aiter_text async for byte_content in self.aiter_bytes(): File "F:\Anaconda\envs\ChatGLM\lib\site-packages\httpx_models.py", line 929, in aiter_bytes async for raw_bytes in self.aiter_raw(): File "F:\Anaconda\envs\ChatGLM\lib\site-packages\httpx_models.py", line 987, in aiter_raw async for raw_stream_bytes in self.stream: File "F:\Anaconda\envs\ChatGLM\lib\site-packages\httpx_client.py", line 149, in aiter async for chunk in self._stream: File "F:\Anaconda\envs\ChatGLM\lib\site-packages\httpx_transports\default.py", line 253, in aiter with map_httpcore_exceptions(): File "F:\Anaconda\envs\ChatGLM\lib\contextlib.py", line 153, in exit self.gen.throw(typ, value, traceback) File "F:\Anaconda\envs\ChatGLM\lib\site-packages\httpx_transports\default.py", line 86, in map_httpcore_exceptions raise mapped_exc(message) from exc httpx.RemoteProtocolError: peer closed connection without sending complete message body (incomplete chunked read) 2024-03-26 14:48:00,148 - utils.py[line:40] - ERROR: RemoteProtocolError: Caught exception: peer closed connection without sending complete message body (incomplete chunked read)

请问您解决了吗怎么解决的呢？我也遇到了相同的问题

chatchat-space / Langchain-Chatchat

[BUG] 运用本地知识库询问几句之后报错 / Concise description of the issue #3524

Sources