开发部署下，本地运行后无法进行知识问题，显示网络问题

[Sumail2020]

问题描述 / Problem Description 本地成功运行后，打开webui无法进行问答。终端显示网络连接问题，但本地运行应该不走网络，不知道在哪里设置。

复现问题的步骤 / Steps to Reproduce

1. 运行python startup.py -a
2. 打开127.0.0.1:80
3. 点击LLM问答，输入问题
4. 一直没反应

*预期的结果 / Expected Result 应该出现回答/ Describe the expected result.

实际结果 / Actual Result 未出现回答，终端显示Http code 403 from API (但是模型都下载到本地了，而且也应该正确加载了，正常来说应该不走网络的)/ Describe the actual result.

[maamzhe]

你是不是模型地址写的不对？其他地方地址都是/左斜杠，你的地址是\右斜杠，右斜杠不会转译么？

[Sumail2020]

你是不是模型地址写的不对？其他地方地址都是/左斜杠，你的地址是\右斜杠，右斜杠不会转译么？

修改了，但还是不行，应该不是地址的问题。我在考虑用2.2.0版本试一下

[libo119111]

请问断网之后出现这个问题怎么解决

[libo119111]

还是不行我试过了，断网之后就会出现这个问题

[libo119111]

问题描述 / Problem Description 本地成功运行后，打开webui无法进行问答。终端显示网络连接问题，但本地运行应该不走网络，不知道在哪里设置。

复现问题的步骤 / Steps to Reproduce

1. 运行python startup.py -a
2. 打开127.0.0.1:80
3. 点击LLM问答，输入问题
4. 一直没反应

*预期的结果 / Expected Result 应该出现回答/ Describe the expected result.

实际结果 / Actual Result 未出现回答，终端显示Http code 403 from API (但是模型都下载到本地了，而且也应该正确加载了，正常来说应该不走网络的)/ Describe the actual result.

兄弟问题解决了么，我这也有这个问题了

[jinghong123]

问题复现 1.python.exe init_database.py 2.python.exe startup.py --all-api 3.python.exe server\llm_api.py 4.streamlit run webui.py --server.port 7860

结果：

==============================Langchain-Chatchat Configuration============================== 操作系统：Windows-10-10.0.22621-SP0. python版本：3.10.0 | packaged by conda-forge | (default, Nov 10 2021, 13:20:59) [MSC v.1916 64 bit (AMD64)] 项目版本：v0.2.4 langchain版本：0.0.287. fastchat版本：0.2.28

当前使用的分词器：ChineseRecursiveTextSplitter 当前启动的LLM模型：['chatglm2-6b-int4'] @ cuda {'api_base_url': 'http://192.168.1.43:8888/v1', 'api_key': 'EMPTY', 'local_model_path': 'C:\chatGLM1\Langchain-Chatchat\model\chatglm2-6b-int4'} 当前Embbedings模型： m3e-base @ cuda ==============================Langchain-Chatchat Configuration==============================

database talbes created filling kb infos to database 2023-10-06 21:54:33,088 - faiss_cache.py[line:74] - INFO: loading vector store in 'samples' from disk. {'kb_name': 'samples', 'vs_type': 'faiss', 'embed_model': 'm3e-base', 'file_count': 1, 'create_time': datetime.datetime(2023, 9, 26, 15, 19, 21)} 2023-10-06 21:54:33,288 - SentenceTransformer.py[line:66] - INFO: Load pretrained SentenceTransformer: C:\chatGLM1\Langchain-Chatchat\model\m3e-base 2023-10-06 21:54:34,486 - loader.py[line:54] - INFO: Loading faiss with AVX2 support. 2023-10-06 21:54:34,487 - loader.py[line:58] - INFO: Could not load library with AVX2 support due to: ModuleNotFoundError("No module named 'faiss.swigfaiss_avx2'") 2023-10-06 21:54:34,487 - loader.py[line:64] - INFO: Loading faiss. 2023-10-06 21:54:34,564 - loader.py[line:66] - INFO: Successfully loaded faiss. 已将 samples/test.txt 添加到数据库 总计用时： 0:00:01.527891 (ChatGLM1) PS C:\chatGLM1\Langchain-Chatchat> C:\Users\zhang\anaconda3\envs\ChatGLM1\python.exe startup.py --all-api

==============================Langchain-Chatchat Configuration============================== 操作系统：Windows-10-10.0.22621-SP0. python版本：3.10.0 | packaged by conda-forge | (default, Nov 10 2021, 13:20:59) [MSC v.1916 64 bit (AMD64)] 项目版本：v0.2.4 langchain版本：0.0.287. fastchat版本：0.2.28

当前使用的分词器：ChineseRecursiveTextSplitter 当前启动的LLM模型：['chatglm2-6b-int4'] @ cuda {'api_base_url': 'http://192.168.1.43:8888/v1', 'api_key': 'EMPTY', 'local_model_path': 'C:\chatGLM1\Langchain-Chatchat\model\chatglm2-6b-int4'} 当前Embbedings模型： m3e-base @ cuda ==============================Langchain-Chatchat Configuration==============================

2023-10-06 21:54:45,187 - startup.py[line:541] - INFO: 正在启动服务： 2023-10-06 21:54:45,187 - startup.py[line:542] - INFO: 如需查看 llm_api 日志，请前往 C:\chatGLM1\Langchain-Chatchat\logs 2023-10-06 21:54:47 | ERROR | stderr | INFO: Started server process [20580] 2023-10-06 21:54:47 | ERROR | stderr | INFO: Waiting for application startup. 2023-10-06 21:54:47 | ERROR | stderr | INFO: Application startup complete. 2023-10-06 21:54:47 | ERROR | stderr | INFO: Uvicorn running on http://192.168.1.43:8888 (Press CTRL+C to quit) 2023-10-06 21:54:52 | WARNING | transformers_modules.chatglm2-6b-int4.quantization | Failed to load cpm_kernels:No module named 'cpm_kernels' C:\Users\zhang.cache\huggingface\modules\transformers_modules\chatglm2-6b-int4\quantization_kernels_parallel.c:1:0: warning: -fPIC ignored for target (all code is position independent)

include

^ 2023-10-06 21:54:53 | WARNING | transformers_modules.chatglm2-6b-int4.quantization | Load parallel cpu kernel failed C:\Users\zhang.cache\huggingface\modules\transformers_modules\chatglm2-6b-int4\quantization_kernels_parallel.so: Traceback (most recent call last): File "C:\Users\zhang/.cache\huggingface\modules\transformers_modules\chatglm2-6b-int4\quantization.py", line 148, in init kernels = ctypes.cdll.LoadLibrary(kernel_file) File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\ctypes__init__.py", line 452, in LoadLibrary return self._dlltype(name) File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\ctypes__init.py", line 374, in init__ self._handle = _dlopen(self._name, mode) FileNotFoundError: Could not find module 'C:\Users\zhang.cache\huggingface\modules\transformers_modules\chatglm2-6b-int4\quantization_kernels_parallel.so' (or one of its dependencies). Try using the full path with constructor syntax.

C:\Users\zhang.cache\huggingface\modules\transformers_modules\chatglm2-6b-int4\quantization_kernels.c:1:0: warning: -fPIC ignored for target (all code is position independent) void compress_int4_weight(void weight, void out, int n, int m) ^ INFO: Started server process [12124] INFO: Waiting for application startup.

INFO: Application startup complete.

==============================Langchain-Chatchat Configuration============================== 操作系统：Windows-10-10.0.22621-SP0. python版本：3.10.0 | packaged by conda-forge | (default, Nov 10 2021, 13:20:59) [MSC v.1916 64 bit (AMD64)] 项目版本：v0.2.4 langchain版本：0.0.287. fastchat版本：0.2.28

当前使用的分词器：ChineseRecursiveTextSplitter INFO: Uvicorn running on http://192.168.1.43:7861 (Press CTRL+C to quit) 当前启动的LLM模型：['chatglm2-6b-int4'] @ cuda {'api_base_url': 'http://192.168.1.43:8888/v1', 'api_key': 'EMPTY', 'local_model_path': 'C:\chatGLM1\Langchain-Chatchat\model\chatglm2-6b-int4'} 当前Embbedings模型： m3e-base @ cuda

服务端运行信息： OpenAI API Server: http://192.168.1.43:8888/v1 (请确认llm_model_dict中配置的api_base_url与上面地址一致。) Chatchat API Server: http://192.168.1.43:7861 ==============================Langchain-Chatchat Configuration==============================

2023-10-06 21:56:14,996 - _client.py[line:1013] - INFO: HTTP Request: POST http://192.168.1.43:20001/list_models "HTTP/1.1 200 OK" INFO: 192.168.1.43:49612 - "POST /llm_model/list_models HTTP/1.1" 200 OK 2023-10-06 21:56:28,717 - _client.py[line:1013] - INFO: HTTP Request: POST http://192.168.1.43:20001/list_models "HTTP/1.1 200 OK" INFO: 192.168.1.43:49663 - "POST /llm_model/list_models HTTP/1.1" 200 OK INFO: 192.168.1.43:49669 - "POST /chat/knowledge_base_chat HTTP/1.1" 200 OK 2023-10-06 21:56:28,872 - SentenceTransformer.py[line:66] - INFO: Load pretrained SentenceTransformer: C:\chatGLM1\Langchain-Chatchat\model\m3e-base 2023-10-06 21:56:29,752 - faiss_cache.py[line:74] - INFO: loading vector store in 'samples' from disk. {'kb_name': 'samples', 'vs_type': 'faiss', 'embed_model': 'm3e-base', 'file_count': 1, 'create_time': datetime.datetime(2023, 9, 26, 15, 19, 21)} 2023-10-06 21:56:29,756 - loader.py[line:54] - INFO: Loading faiss with AVX2 support. 2023-10-06 21:56:29,756 - loader.py[line:58] - INFO: Could not load library with AVX2 support due to: ModuleNotFoundError("No module named 'faiss.swigfaiss_avx2'") 2023-10-06 21:56:29,756 - loader.py[line:64] - INFO: Loading faiss. 2023-10-06 21:56:29,773 - loader.py[line:66] - INFO: Successfully loaded faiss. Batches: 100%|███████████████████████████████████████████████████████████████████████████| 1/1 [00:05<00:00, 5.70s/it] 2023-10-06 21:56:35,779 - _client.py[line:1013] - INFO: HTTP Request: POST http://192.168.1.43:20001/list_models "HTTP/1.1 200 OK" INFO: 192.168.1.43:49684 - "POST /llm_model/list_models HTTP/1.1" 200 OK 2023-10-06 21:56:35 | INFO | stdout | INFO: 192.168.1.43:49693 - "POST /v1/chat/completions HTTP/1.1" 200 OK 2023-10-06 21:56:35,803 - util.py[line:67] - INFO: message='OpenAI API response' path=http://192.168.1.43:8888/v1/chat/completions processing_ms=None request_id=None response_code=200 2023-10-06 21:56:35 | INFO | httpx | HTTP Request: POST http://192.168.1.43:20002/worker_generate_stream "HTTP/1.1 200 OK" ERROR: Exception in ASGI application Traceback (most recent call last): File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\starlette\responses.py", line 277, in call await wrap(partial(self.listen_for_disconnect, receive)) File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\starlette\responses.py", line 273, in wrap await func() File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\starlette\responses.py", line 250, in listen_for_disconnect message = await receive() File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\uvicorn\protocols\http\h11_impl.py", line 538, in receive await self.message_event.wait() File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\asyncio\locks.py", line 213, in wait await fut asyncio.exceptions.CancelledError: Cancelled by cancel scope 1b76314b1c0

During handling of the above exception, another exception occurred:

• Exception Group Traceback (most recent call last): | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\uvicorn\protocols\http\h11_impl.py", line 408, in run_asgi | result = await app( # type: ignore[func-returns-value] | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\uvicorn\middleware\proxy_headers.py", line 84, in call | return await self.app(scope, receive, send) | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\fastapi\applications.py", line 290, in call | await super().call(scope, receive, send) | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\starlette\applications.py", line 122, in call | await self.middleware_stack(scope, receive, send) | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\starlette\middleware\errors.py", line 184, in call | raise exc | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\starlette\middleware\errors.py", line 162, in call | await self.app(scope, receive, _send) | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\starlette\middleware\exceptions.py", line 79, in call | raise exc | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\starlette\middleware\exceptions.py", line 68, in call | await self.app(scope, receive, sender) | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\fastapi\middleware\asyncexitstack.py", line 20, in call | raise e | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\fastapi\middleware\asyncexitstack.py", line 17, in call | await self.app(scope, receive, send) | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\starlette\routing.py", line 718, in call | await route.handle(scope, receive, send) | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\starlette\routing.py", line 276, in handle | await self.app(scope, receive, send) | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\starlette\routing.py", line 69, in app | await response(scope, receive, send) | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\starlette\responses.py", line 270, in call | async with anyio.create_task_group() as task_group: | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\anyio_backends_asyncio.py", line 664, in aexit | raise BaseExceptionGroup( | exceptiongroup.ExceptionGroup: unhandled errors in a TaskGroup (1 sub-exception) +-+---------------- 1 ---------------- | Traceback (most recent call last): | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\starlette\responses.py", line 273, in wrap | await func() | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\starlette\responses.py", line 262, in stream_response | async for chunk in self.body_iterator: | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\starlette\concurrency.py", line 63, in iterate_in_threadpool | yield await anyio.to_thread.run_sync(_next, iterator) | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\anyio\to_thread.py", line 33, in run_sync | return await get_async_backend().run_sync_in_worker_thread( | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\anyio_backends_asyncio.py", line 2106, in run_sync_in_worker_thread | return await future | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\anyio_backends_asyncio.py", line 833, in run | result = context.run(func, args) | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\starlette\concurrency.py", line 53, in _next | return next(iterator) | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\fastchat\serve\model_worker.py", line 238, in generate_stream_gate | for output in self.generate_stream_func( | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\torch\utils_contextlib.py", line 35, in generator_context | response = gen.send(None) | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\fastchat\model\model_chatglm.py", line 71, in generate_stream_chatglm | for total_ids in model.stream_generate(inputs, gen_kwargs): | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\torch\utils_contextlib.py", line 35, in generator_context | response = gen.send(None) | File "C:\Users\zhang/.cache\huggingface\modules\transformers_modules\chatglm2-6b-int4\modeling_chatglm.py", line 1143, in stream_generate | outputs = self( | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl | return forward_call(args, kwargs) | File "C:\Users\zhang/.cache\huggingface\modules\transformers_modules\chatglm2-6b-int4\modeling_chatglm.py", line 932, in forward | transformer_outputs = self.transformer( | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl | return forward_call(*args, *kwargs) | File "C:\Users\zhang/.cache\huggingface\modules\transformers_modules\chatglm2-6b-int4\modeling_chatglm.py", line 828, in forward | hidden_states, presents, all_hidden_states, all_self_attentions = self.encoder( | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl | return forward_call(args, kwargs) | File "C:\Users\zhang/.cache\huggingface\modules\transformers_modules\chatglm2-6b-int4\modeling_chatglm.py", line 638, in forward | layer_ret = layer( | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl | return forward_call(*args, kwargs) | File "C:\Users\zhang/.cache\huggingface\modules\transformers_modules\chatglm2-6b-int4\modeling_chatglm.py", line 542, in forward | attention_output, kv_cache = self.self_attention( | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl | return forward_call(*args, *kwargs) | File "C:\Users\zhang/.cache\huggingface\modules\transformers_modules\chatglm2-6b-int4\modeling_chatglm.py", line 374, in forward | mixed_x_layer = self.query_key_value(hidden_states) | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl | return forward_call(args, kwargs) | File "C:\Users\zhang/.cache\huggingface\modules\transformers_modules\chatglm2-6b-int4\quantization.py", line 502, in forward | output = W8A16Linear.apply(input, self.weight, self.weight_scale, self.weight_bit_width) | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\torch\autograd\function.py", line 506, in apply | return super().apply(*args, **kwargs) # type: ignore[misc] | File "C:\Users\zhang/.cache\huggingface\modules\transformers_modules\chatglm2-6b-int4\quantization.py", line 75, in forward | weight = extract_weight_to_half(quant_w, scale_w, weight_bit_width) | File "C:\Users\zhang/.cache\huggingface\modules\transformers_modules\chatglm2-6b-int4\quantization.py", line 287, in extract_weight_to_half | func = kernels.int4WeightExtractionHalf | AttributeError: 'NoneType' object has no attribute 'int4WeightExtractionHalf' +------------------------------------ 2023-10-06 21:56:37 | ERROR | stderr | ERROR: Exception in ASGI application 2023-10-06 21:56:37 | ERROR | stderr | Traceback (most recent call last): 2023-10-06 21:56:37 | ERROR | stderr | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\starlette\responses.py", line 277, in call 2023-10-06 21:56:37 | ERROR | stderr | await wrap(partial(self.listen_for_disconnect, receive)) 2023-10-06 21:56:37 | ERROR | stderr | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\starlette\responses.py", line 273, in wrap 2023-10-06 21:56:37 | ERROR | stderr | await func() 2023-10-06 21:56:37 | ERROR | stderr | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\starlette\responses.py", line 250, in listen_for_disconnect 2023-10-06 21:56:37 | ERROR | stderr | message = await receive() 2023-10-06 21:56:37 | ERROR | stderr | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\uvicorn\protocols\http\h11_impl.py", line 538, in receive 2023-10-06 21:56:37 | ERROR | stderr | await self.message_event.wait() 2023-10-06 21:56:37 | ERROR | stderr | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\asyncio\locks.py", line 213, in wait 2023-10-06 21:56:37 | ERROR | stderr | await fut 2023-10-06 21:56:37 | ERROR | stderr | asyncio.exceptions.CancelledError: Cancelled by cancel scope 1f5dc24cc10 2023-10-06 21:56:37 | ERROR | stderr | 2023-10-06 21:56:37 | ERROR | stderr | During handling of the above exception, another exception occurred: 2023-10-06 21:56:37 | ERROR | stderr | 2023-10-06 21:56:37 | ERROR | stderr | + Exception Group Traceback (most recent call last): 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\uvicorn\protocols\http\h11_impl.py", line 408, in run_asgi 2023-10-06 21:56:37 | ERROR | stderr | | result = await app( # type: ignore[func-returns-value] 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\uvicorn\middleware\proxy_headers.py", line 84, in call 2023-10-06 21:56:37 | ERROR | stderr | | return await self.app(scope, receive, send) 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\fastapi\applications.py", line 290, in call 2023-10-06 21:56:37 | ERROR | stderr | | await super().call(scope, receive, send) 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\starlette\applications.py", line 122, in call 2023-10-06 21:56:37 | ERROR | stderr | | await self.middleware_stack(scope, receive, send) 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\starlette\middleware\errors.py", line 184, in call 2023-10-06 21:56:37 | ERROR | stderr | | raise exc 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\starlette\middleware\errors.py", line 162, in call 2023-10-06 21:56:37 | ERROR | stderr | | await self.app(scope, receive, _send) 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\starlette\middleware\cors.py", line 83, in call 2023-10-06 21:56:37 | ERROR | stderr | | await self.app(scope, receive, send) 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\starlette\middleware\exceptions.py", line 79, in call 2023-10-06 21:56:37 | ERROR | stderr | | raise exc 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\starlette\middleware\exceptions.py", line 68, in call 2023-10-06 21:56:37 | ERROR | stderr | | await self.app(scope, receive, sender) 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\fastapi\middleware\asyncexitstack.py", line 20, in call 2023-10-06 21:56:37 | ERROR | stderr | | raise e 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\fastapi\middleware\asyncexitstack.py", line 17, in call 2023-10-06 21:56:37 | ERROR | stderr | | await self.app(scope, receive, send) 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\starlette\routing.py", line 718, in call 2023-10-06 21:56:37 | ERROR | stderr | | await route.handle(scope, receive, send) 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\starlette\routing.py", line 276, in handle 2023-10-06 21:56:37 | ERROR | stderr | | await self.app(scope, receive, send) 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\starlette\routing.py", line 69, in app 2023-10-06 21:56:37 | ERROR | stderr | | await response(scope, receive, send) 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\starlette\responses.py", line 270, in call 2023-10-06 21:56:37 | ERROR | stderr | | async with anyio.create_task_group() as task_group: 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\anyio_backends_asyncio.py", line 664, in aexit 2023-10-06 21:56:37 | ERROR | stderr | | raise BaseExceptionGroup( 2023-10-06 21:56:37 | ERROR | stderr | | exceptiongroup.ExceptionGroup: unhandled errors in a TaskGroup (1 sub-exception) 2023-10-06 21:56:37 | ERROR | stderr | +-+---------------- 1 ---------------- 2023-10-06 21:56:37 | ERROR | stderr | | Traceback (most recent call last): 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\httpcore_exceptions.py", line 10, in map_exceptions 2023-10-06 21:56:37 | ERROR | stderr | | yield 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\httpcore_async\http11.py", line 197, in _receive_event 2023-10-06 21:56:37 | ERROR | stderr | | event = self._h11_state.next_event() 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\h11_connection.py", line 469, in next_event 2023-10-06 21:56:37 | ERROR | stderr | | event = self._extract_next_receive_event() 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\h11_connection.py", line 419, in _extract_next_receive_event 2023-10-06 21:56:37 | ERROR | stderr | | event = self._reader.read_eof() # type: ignore[attr-defined] 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\h11_readers.py", line 204, in read_eof 2023-10-06 21:56:37 | ERROR | stderr | | raise RemoteProtocolError( 2023-10-06 21:56:37 | ERROR | stderr | | h11._util.RemoteProtocolError: peer closed connection without sending complete message body (incomplete chunked read) 2023-10-06 21:56:37 | ERROR | stderr | | 2023-10-06 21:56:37 | ERROR | stderr | | The above exception was the direct cause of the following exception: 2023-10-06 21:56:37 | ERROR | stderr | | 2023-10-06 21:56:37 | ERROR | stderr | | Traceback (most recent call last): 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\httpx_transports\default.py", line 60, in map_httpcore_exceptions 2023-10-06 21:56:37 | ERROR | stderr | | yield 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\httpx_transports\default.py", line 239, in aiter 2023-10-06 21:56:37 | ERROR | stderr | | async for part in self._httpcore_stream: 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\httpcore_async\connection_pool.py", line 347, in aiter 2023-10-06 21:56:37 | ERROR | stderr | | async for part in self._stream: 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\httpcore_async\http11.py", line 325, in aiter 2023-10-06 21:56:37 | ERROR | stderr | | raise exc 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\httpcore_async\http11.py", line 317, in aiter 2023-10-06 21:56:37 | ERROR | stderr | | async for chunk in self._connection._receive_response_body(**kwargs): 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\httpcore_async\http11.py", line 186, in _receive_response_body 2023-10-06 21:56:37 | ERROR | stderr | | event = await self._receive_event(timeout=timeout) 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\httpcore_async\http11.py", line 196, in _receive_event 2023-10-06 21:56:37 | ERROR | stderr | | with map_exceptions({h11.RemoteProtocolError: RemoteProtocolError}): 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\contextlib.py", line 153, in exit 2023-10-06 21:56:37 | ERROR | stderr | | self.gen.throw(typ, value, traceback) 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\httpcore_exceptions.py", line 14, in map_exceptions 2023-10-06 21:56:37 | ERROR | stderr | | raise to_exc(exc) from exc 2023-10-06 21:56:37 | ERROR | stderr | | httpcore.RemoteProtocolError: peer closed connection without sending complete message body (incomplete chunked read) 2023-10-06 21:56:37 | ERROR | stderr | | 2023-10-06 21:56:37 | ERROR | stderr | | The above exception was the direct cause of the following exception: 2023-10-06 21:56:37 | ERROR | stderr | | 2023-10-06 21:56:37 | ERROR | stderr | | Traceback (most recent call last): 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\starlette\responses.py", line 273, in wrap 2023-10-06 21:56:37 | ERROR | stderr | | await func() 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\starlette\responses.py", line 262, in stream_response 2023-10-06 21:56:37 | ERROR | stderr | | async for chunk in self.body_iterator: 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\fastchat\serve\openai_api_server.py", line 431, in chat_completion_stream_generator 2023-10-06 21:56:37 | ERROR | stderr | | async for content in generate_completion_stream(gen_params, worker_addr): 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\fastchat\serve\openai_api_server.py", line 594, in generate_completion_stream 2023-10-06 21:56:37 | ERROR | stderr | | async for raw_chunk in response.aiter_raw(): 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\httpx_models.py", line 985, in aiter_raw 2023-10-06 21:56:37 | ERROR | stderr | | async for raw_stream_bytes in self.stream: 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\httpx_client.py", line 146, in aiter 2023-10-06 21:56:37 | ERROR | stderr | | async for chunk in self._stream: 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\httpx_transports\default.py", line 238, in aiter 2023-10-06 21:56:37 | ERROR | stderr | | with map_httpcore_exceptions(): 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\contextlib.py", line 153, in exit 2023-10-06 21:56:37 | ERROR | stderr | | self.gen.throw(typ, value, traceback) 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\httpx_transports\default.py", line 77, in map_httpcore_exceptions 2023-10-06 21:56:37 | ERROR | stderr | | raise mapped_exc(message) from exc 2023-10-06 21:56:37 | ERROR | stderr | | httpx.RemoteProtocolError: peer closed connection without sending complete message body (incomplete chunked read) 2023-10-06 21:56:37 | ERROR | stderr | +------------------------------------ 2023-10-06 21:56:37,591 - utils.py[line:15] - ERROR: ClientPayloadError: Caught exception: Response payload is not completed

启动 2023-10-06 21:56:14,999 - _client.py[line:1013] - INFO: HTTP Request: POST http://192.168.1.43:7861/llm_model/list_models "HTTP/1.1 200 OK" 2023-10-06 21:56:28 | INFO | httpx | HTTP Request: POST http://192.168.1.43:7861/llm_model/list_models "HTTP/1.1 200 OK" received input message: {'history': [{'content': '你好', 'role': 'user'}], 'knowledge_base_name': 'samples', 'local_doc_url': False, 'model_name': 'chatglm2-6b-int4', 'query': '你好', 'score_threshold': 1.0, 'stream': True, 'temperature': 0.7, 'top_k': 3} 2023-10-06 21:56:28 | INFO | httpx | HTTP Request: POST http://192.168.1.43:7861/chat/knowledge_base_chat "HTTP/1.1 200 OK" 2023-10-06 21:56:35 | INFO | httpx | HTTP Request: POST http://192.168.1.43:7861/llm_model/list_models "HTTP/1.1 200 OK" {'docs': [...]} 2023-10-06 21:56:37.592 Uncaught app exception Traceback (most recent call last): File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\streamlit\runtime\scriptrunner\script_runner.py", line 541, in _run_script exec(code, module.dict) File "C:\chatGLM1\Langchain-Chatchat\webui.py", line 66, in pages[selected_page]"func" File "C:\chatGLM1\Langchain-Chatchat\webui_pages\dialogue\dialogue.py", line 171, in dialogue_page chat_box.update_msg(text, element_index=0, streaming=False) File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\streamlit_chatbox\messages.py", line 294, in update_msg assert self.chat_inited, "please call init_session first" AssertionError: please call init_session first

[jinghong123]

结果和上面一样，问话不答，模型不答，不知道哪里出现问题

[nanisnull]

是不是模型的问题，你现在用的是chatglm-6b-int4；你试试chatglm-6b呢

问题复现 1.python.exe init_database.py 2.python.exe startup.py --all-api 3.python.exe server\llm_api.py 4.streamlit run webui.py --server.port 7860

结果：

==============================Langchain-Chatchat Configuration============================== 操作系统：Windows-10-10.0.22621-SP0. python版本：3.10.0 | packaged by conda-forge | (default, Nov 10 2021, 13:20:59) [MSC v.1916 64 bit (AMD64)] 项目版本：v0.2.4 langchain版本：0.0.287. fastchat版本：0.2.28

当前使用的分词器：ChineseRecursiveTextSplitter 当前启动的LLM模型：['chatglm2-6b-int4'] @ cuda {'api_base_url': 'http://192.168.1.43:8888/v1', 'api_key': 'EMPTY', 'local_model_path': 'C:\chatGLM1\Langchain-Chatchat\model\chatglm2-6b-int4'} 当前Embbedings模型： m3e-base @ cuda ==============================Langchain-Chatchat Configuration==============================

database talbes created filling kb infos to database 2023-10-06 21:54:33,088 - faiss_cache.py[line:74] - INFO: loading vector store in 'samples' from disk. {'kb_name': 'samples', 'vs_type': 'faiss', 'embed_model': 'm3e-base', 'file_count': 1, 'create_time': datetime.datetime(2023, 9, 26, 15, 19, 21)} 2023-10-06 21:54:33,288 - SentenceTransformer.py[line:66] - INFO: Load pretrained SentenceTransformer: C:\chatGLM1\Langchain-Chatchat\model\m3e-base 2023-10-06 21:54:34,486 - loader.py[line:54] - INFO: Loading faiss with AVX2 support. 2023-10-06 21:54:34,487 - loader.py[line:58] - INFO: Could not load library with AVX2 support due to: ModuleNotFoundError("No module named 'faiss.swigfaiss_avx2'") 2023-10-06 21:54:34,487 - loader.py[line:64] - INFO: Loading faiss. 2023-10-06 21:54:34,564 - loader.py[line:66] - INFO: Successfully loaded faiss. 已将 samples/test.txt 添加到数据库 总计用时： 0:00:01.527891 (ChatGLM1) PS C:\chatGLM1\Langchain-Chatchat> C:\Users\zhang\anaconda3\envs\ChatGLM1\python.exe startup.py --all-api

==============================Langchain-Chatchat Configuration============================== 操作系统：Windows-10-10.0.22621-SP0. python版本：3.10.0 | packaged by conda-forge | (default, Nov 10 2021, 13:20:59) [MSC v.1916 64 bit (AMD64)] 项目版本：v0.2.4 langchain版本：0.0.287. fastchat版本：0.2.28

当前使用的分词器：ChineseRecursiveTextSplitter 当前启动的LLM模型：['chatglm2-6b-int4'] @ cuda {'api_base_url': 'http://192.168.1.43:8888/v1', 'api_key': 'EMPTY', 'local_model_path': 'C:\chatGLM1\Langchain-Chatchat\model\chatglm2-6b-int4'} 当前Embbedings模型： m3e-base @ cuda ==============================Langchain-Chatchat Configuration==============================

2023-10-06 21:54:45,187 - startup.py[line:541] - INFO: 正在启动服务： 2023-10-06 21:54:45,187 - startup.py[line:542] - INFO: 如需查看 llm_api 日志，请前往 C:\chatGLM1\Langchain-Chatchat\logs 2023-10-06 21:54:47 | ERROR | stderr | INFO: Started server process [20580] 2023-10-06 21:54:47 | ERROR | stderr | INFO: Waiting for application startup. 2023-10-06 21:54:47 | ERROR | stderr | INFO: Application startup complete. 2023-10-06 21:54:47 | ERROR | stderr | INFO: Uvicorn running on http://192.168.1.43:8888 (Press CTRL+C to quit) 2023-10-06 21:54:52 | WARNING | transformers_modules.chatglm2-6b-int4.quantization | Failed to load cpm_kernels:No module named 'cpm_kernels' C:\Users\zhang.cache\huggingface\modules\transformers_modules\chatglm2-6b-int4\quantization_kernels_parallel.c:1:0: warning: -fPIC ignored for target (all code is position independent) #include ^ 2023-10-06 21:54:53 | WARNING | transformers_modules.chatglm2-6b-int4.quantization | Load parallel cpu kernel failed C:\Users\zhang.cache\huggingface\modules\transformers_modules\chatglm2-6b-int4\quantization_kernels_parallel.so: Traceback (most recent call last): File "C:\Users\zhang/.cache\huggingface\modules\transformers_modules\chatglm2-6b-int4\quantization.py", line 148, in init kernels = ctypes.cdll.LoadLibrary(kernel_file) File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\ctypesinit.py", line 452, in LoadLibrary return self._dlltype(name) File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\ctypesinit.py", line 374, in init self._handle = _dlopen(self._name, mode) FileNotFoundError: Could not find module 'C:\Users\zhang.cache\huggingface\modules\transformers_modules\chatglm2-6b-int4\quantization_kernels_parallel.so' (or one of its dependencies). Try using the full path with constructor syntax.

C:\Users\zhang.cache\huggingface\modules\transformers_modules\chatglm2-6b-int4\quantization_kernels.c:1:0: warning: -fPIC ignored for target (all code is position independent) void compress_int4_weight(void weight, void out, int n, int m) ^ INFO: Started server process [12124] INFO: Waiting for application startup.

INFO: Application startup complete.

==============================Langchain-Chatchat Configuration============================== 操作系统：Windows-10-10.0.22621-SP0. python版本：3.10.0 | packaged by conda-forge | (default, Nov 10 2021, 13:20:59) [MSC v.1916 64 bit (AMD64)] 项目版本：v0.2.4 langchain版本：0.0.287. fastchat版本：0.2.28

当前使用的分词器：ChineseRecursiveTextSplitter INFO: Uvicorn running on http://192.168.1.43:7861 (Press CTRL+C to quit) 当前启动的LLM模型：['chatglm2-6b-int4'] @ cuda {'api_base_url': 'http://192.168.1.43:8888/v1', 'api_key': 'EMPTY', 'local_model_path': 'C:\chatGLM1\Langchain-Chatchat\model\chatglm2-6b-int4'} 当前Embbedings模型： m3e-base @ cuda

服务端运行信息： OpenAI API Server: http://192.168.1.43:8888/v1 (请确认llm_model_dict中配置的api_base_url与上面地址一致。) Chatchat API Server: http://192.168.1.43:7861 ==============================Langchain-Chatchat Configuration==============================

2023-10-06 21:56:14,996 - _client.py[line:1013] - INFO: HTTP Request: POST http://192.168.1.43:20001/list_models "HTTP/1.1 200 OK" INFO: 192.168.1.43:49612 - "POST /llm_model/list_models HTTP/1.1" 200 OK 2023-10-06 21:56:28,717 - _client.py[line:1013] - INFO: HTTP Request: POST http://192.168.1.43:20001/list_models "HTTP/1.1 200 OK" INFO: 192.168.1.43:49663 - "POST /llm_model/list_models HTTP/1.1" 200 OK INFO: 192.168.1.43:49669 - "POST /chat/knowledge_base_chat HTTP/1.1" 200 OK 2023-10-06 21:56:28,872 - SentenceTransformer.py[line:66] - INFO: Load pretrained SentenceTransformer: C:\chatGLM1\Langchain-Chatchat\model\m3e-base 2023-10-06 21:56:29,752 - faiss_cache.py[line:74] - INFO: loading vector store in 'samples' from disk. {'kb_name': 'samples', 'vs_type': 'faiss', 'embed_model': 'm3e-base', 'file_count': 1, 'create_time': datetime.datetime(2023, 9, 26, 15, 19, 21)} 2023-10-06 21:56:29,756 - loader.py[line:54] - INFO: Loading faiss with AVX2 support. 2023-10-06 21:56:29,756 - loader.py[line:58] - INFO: Could not load library with AVX2 support due to: ModuleNotFoundError("No module named 'faiss.swigfaiss_avx2'") 2023-10-06 21:56:29,756 - loader.py[line:64] - INFO: Loading faiss. 2023-10-06 21:56:29,773 - loader.py[line:66] - INFO: Successfully loaded faiss. Batches: 100%|███████████████████████████████████████████████████████████████████████████| 1/1 [00:05<00:00, 5.70s/it] 2023-10-06 21:56:35,779 - _client.py[line:1013] - INFO: HTTP Request: POST http://192.168.1.43:20001/list_models "HTTP/1.1 200 OK" INFO: 192.168.1.43:49684 - "POST /llm_model/list_models HTTP/1.1" 200 OK 2023-10-06 21:56:35 | INFO | stdout | INFO: 192.168.1.43:49693 - "POST /v1/chat/completions HTTP/1.1" 200 OK 2023-10-06 21:56:35,803 - util.py[line:67] - INFO: message='OpenAI API response' path=http://192.168.1.43:8888/v1/chat/completions processing_ms=None request_id=None response_code=200 2023-10-06 21:56:35 | INFO | httpx | HTTP Request: POST http://192.168.1.43:20002/worker_generate_stream "HTTP/1.1 200 OK" ERROR: Exception in ASGI application Traceback (most recent call last): File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\starlette\responses.py", line 277, in call await wrap(partial(self.listen_for_disconnect, receive)) File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\starlette\responses.py", line 273, in wrap await func() File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\starlette\responses.py", line 250, in listen_for_disconnect message = await receive() File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\uvicorn\protocols\http\h11_impl.py", line 538, in receive await self.message_event.wait() File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\asyncio\locks.py", line 213, in wait await fut asyncio.exceptions.CancelledError: Cancelled by cancel scope 1b76314b1c0

During handling of the above exception, another exception occurred:

• Exception Group Traceback (most recent call last): | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\uvicorn\protocols\http\h11_impl.py", line 408, in run_asgi | result = await app( # type: ignore[func-returns-value] | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\uvicorn\middleware\proxy_headers.py", line 84, in call | return await self.app(scope, receive, send) | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\fastapi\applications.py", line 290, in call | await super().call(scope, receive, send) | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\starlette\applications.py", line 122, in call | await self.middleware_stack(scope, receive, send) | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\starlette\middleware\errors.py", line 184, in call | raise exc | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\starlette\middleware\errors.py", line 162, in call | await self.app(scope, receive, _send) | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\starlette\middleware\exceptions.py", line 79, in call | raise exc | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\starlette\middleware\exceptions.py", line 68, in call | await self.app(scope, receive, sender) | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\fastapi\middleware\asyncexitstack.py", line 20, in call | raise e | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\fastapi\middleware\asyncexitstack.py", line 17, in call | await self.app(scope, receive, send) | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\starlette\routing.py", line 718, in call | await route.handle(scope, receive, send) | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\starlette\routing.py", line 276, in handle | await self.app(scope, receive, send) | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\starlette\routing.py", line 69, in app | await response(scope, receive, send) | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\starlette\responses.py", line 270, in call | async with anyio.create_task_group() as task_group: | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\anyio_backends_asyncio.py", line 664, in aexit | raise BaseExceptionGroup( | exceptiongroup.ExceptionGroup: unhandled errors in a TaskGroup (1 sub-exception) +-+---------------- 1 ---------------- | Traceback (most recent call last): | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\starlette\responses.py", line 273, in wrap | await func() | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\starlette\responses.py", line 262, in stream_response | async for chunk in self.body_iterator: | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\starlette\concurrency.py", line 63, in iterate_in_threadpool | yield await anyio.to_thread.run_sync(_next, iterator) | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\anyio\to_thread.py", line 33, in run_sync | return await get_async_backend().run_sync_in_worker_thread( | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\anyio_backends_asyncio.py", line 2106, in run_sync_in_worker_thread | return await future | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\anyio_backends_asyncio.py", line 833, in run | result = context.run(func, args) | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\starlette\concurrency.py", line 53, in _next | return next(iterator) | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\fastchat\serve\model_worker.py", line 238, in generate_stream_gate | for output in self.generate_stream_func( | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\torch\utils_contextlib.py", line 35, in generator_context | response = gen.send(None) | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\fastchat\model\model_chatglm.py", line 71, in generate_stream_chatglm | for total_ids in model.stream_generate(inputs, gen_kwargs): | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\torch\utils_contextlib.py", line 35, in generator_context | response = gen.send(None) | File "C:\Users\zhang/.cache\huggingface\modules\transformers_modules\chatglm2-6b-int4\modeling_chatglm.py", line 1143, in stream_generate | outputs = self( | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl | return forward_call(args, kwargs) | File "C:\Users\zhang/.cache\huggingface\modules\transformers_modules\chatglm2-6b-int4\modeling_chatglm.py", line 932, in forward | transformer_outputs = self.transformer( | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl | return forward_call(*args, *kwargs) | File "C:\Users\zhang/.cache\huggingface\modules\transformers_modules\chatglm2-6b-int4\modeling_chatglm.py", line 828, in forward | hidden_states, presents, all_hidden_states, all_self_attentions = self.encoder( | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl | return forward_call(args, kwargs) | File "C:\Users\zhang/.cache\huggingface\modules\transformers_modules\chatglm2-6b-int4\modeling_chatglm.py", line 638, in forward | layer_ret = layer( | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl | return forward_call(*args, kwargs) | File "C:\Users\zhang/.cache\huggingface\modules\transformers_modules\chatglm2-6b-int4\modeling_chatglm.py", line 542, in forward | attention_output, kv_cache = self.self_attention( | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl | return forward_call(*args, *kwargs) | File "C:\Users\zhang/.cache\huggingface\modules\transformers_modules\chatglm2-6b-int4\modeling_chatglm.py", line 374, in forward | mixed_x_layer = self.query_key_value(hidden_states) | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl | return forward_call(args, kwargs) | File "C:\Users\zhang/.cache\huggingface\modules\transformers_modules\chatglm2-6b-int4\quantization.py", line 502, in forward | output = W8A16Linear.apply(input, self.weight, self.weight_scale, self.weight_bit_width) | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\torch\autograd\function.py", line 506, in apply | return super().apply(*args, kwargs) # type: ignore[misc] | File "C:\Users\zhang/.cache\huggingface\modules\transformers_modules\chatglm2-6b-int4\quantization.py", line 75, in forward | weight = extract_weight_to_half(quant_w, scale_w, weight_bit_width) | File "C:\Users\zhang/.cache\huggingface\modules\transformers_modules\chatglm2-6b-int4\quantization.py", line 287, in extract_weight_to_half | func = kernels.int4WeightExtractionHalf | AttributeError: 'NoneType' object has no attribute 'int4WeightExtractionHalf' +------------------------------------ 2023-10-06 21:56:37 | ERROR | stderr | ERROR: Exception in ASGI application 2023-10-06 21:56:37 | ERROR | stderr | Traceback (most recent call last): 2023-10-06 21:56:37 | ERROR | stderr | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\starlette\responses.py", line 277, in call 2023-10-06 21:56:37 | ERROR | stderr | await wrap(partial(self.listen_for_disconnect, receive)) 2023-10-06 21:56:37 | ERROR | stderr | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\starlette\responses.py", line 273, in wrap 2023-10-06 21:56:37 | ERROR | stderr | await func() 2023-10-06 21:56:37 | ERROR | stderr | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\starlette\responses.py", line 250, in listen_for_disconnect 2023-10-06 21:56:37 | ERROR | stderr | message = await receive() 2023-10-06 21:56:37 | ERROR | stderr | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\uvicorn\protocols\http\h11_impl.py", line 538, in receive 2023-10-06 21:56:37 | ERROR | stderr | await self.message_event.wait() 2023-10-06 21:56:37 | ERROR | stderr | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\asyncio\locks.py", line 213, in wait 2023-10-06 21:56:37 | ERROR | stderr | await fut 2023-10-06 21:56:37 | ERROR | stderr | asyncio.exceptions.CancelledError: Cancelled by cancel scope 1f5dc24cc10 2023-10-06 21:56:37 | ERROR | stderr | 2023-10-06 21:56:37 | ERROR | stderr | During handling of the above exception, another exception occurred: 2023-10-06 21:56:37 | ERROR | stderr | 2023-10-06 21:56:37 | ERROR | stderr | + Exception Group Traceback (most recent call last): 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\uvicorn\protocols\http\h11_impl.py", line 408, in run_asgi 2023-10-06 21:56:37 | ERROR | stderr | | result = await app( # type: ignore[func-returns-value] 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\uvicorn\middleware\proxy_headers.py", line 84, in call 2023-10-06 21:56:37 | ERROR | stderr | | return await self.app(scope, receive, send) 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\fastapi\applications.py", line 290, in call 2023-10-06 21:56:37 | ERROR | stderr | | await super().call(scope, receive, send) 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\starlette\applications.py", line 122, in call 2023-10-06 21:56:37 | ERROR | stderr | | await self.middleware_stack(scope, receive, send) 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\starlette\middleware\errors.py", line 184, in call 2023-10-06 21:56:37 | ERROR | stderr | | raise exc 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\starlette\middleware\errors.py", line 162, in call 2023-10-06 21:56:37 | ERROR | stderr | | await self.app(scope, receive, _send) 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\starlette\middleware\cors.py", line 83, in call 2023-10-06 21:56:37 | ERROR | stderr | | await self.app(scope, receive, send) 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\starlette\middleware\exceptions.py", line 79, in call 2023-10-06 21:56:37 | ERROR | stderr | | raise exc 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\starlette\middleware\exceptions.py", line 68, in call 2023-10-06 21:56:37 | ERROR | stderr | | await self.app(scope, receive, sender) 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\fastapi\middleware\asyncexitstack.py", line 20, in call 2023-10-06 21:56:37 | ERROR | stderr | | raise e 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\fastapi\middleware\asyncexitstack.py", line 17, in call 2023-10-06 21:56:37 | ERROR | stderr | | await self.app(scope, receive, send) 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\starlette\routing.py", line 718, in call 2023-10-06 21:56:37 | ERROR | stderr | | await route.handle(scope, receive, send) 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\starlette\routing.py", line 276, in handle 2023-10-06 21:56:37 | ERROR | stderr | | await self.app(scope, receive, send) 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\starlette\routing.py", line 69, in app 2023-10-06 21:56:37 | ERROR | stderr | | await response(scope, receive, send) 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\starlette\responses.py", line 270, in call 2023-10-06 21:56:37 | ERROR | stderr | | async with anyio.create_task_group() as task_group: 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\anyio_backends_asyncio.py", line 664, in aexit 2023-10-06 21:56:37 | ERROR | stderr | | raise BaseExceptionGroup( 2023-10-06 21:56:37 | ERROR | stderr | | exceptiongroup.ExceptionGroup: unhandled errors in a TaskGroup (1 sub-exception) 2023-10-06 21:56:37 | ERROR | stderr | +-+---------------- 1 ---------------- 2023-10-06 21:56:37 | ERROR | stderr | | Traceback (most recent call last): 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\httpcore_exceptions.py", line 10, in map_exceptions 2023-10-06 21:56:37 | ERROR | stderr | | yield 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\httpcore_async\http11.py", line 197, in _receive_event 2023-10-06 21:56:37 | ERROR | stderr | | event = self._h11_state.next_event() 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\h11_connection.py", line 469, in next_event 2023-10-06 21:56:37 | ERROR | stderr | | event = self._extract_next_receive_event() 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\h11_connection.py", line 419, in _extract_next_receive_event 2023-10-06 21:56:37 | ERROR | stderr | | event = self._reader.read_eof() # type: ignore[attr-defined] 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\h11_readers.py", line 204, in read_eof 2023-10-06 21:56:37 | ERROR | stderr | | raise RemoteProtocolError( 2023-10-06 21:56:37 | ERROR | stderr | | h11._util.RemoteProtocolError: peer closed connection without sending complete message body (incomplete chunked read) 2023-10-06 21:56:37 | ERROR | stderr | | 2023-10-06 21:56:37 | ERROR | stderr | | The above exception was the direct cause of the following exception: 2023-10-06 21:56:37 | ERROR | stderr | | 2023-10-06 21:56:37 | ERROR | stderr | | Traceback (most recent call last): 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\httpx_transports\default.py", line 60, in map_httpcore_exceptions 2023-10-06 21:56:37 | ERROR | stderr | | yield 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\httpx_transports\default.py", line 239, in aiter 2023-10-06 21:56:37 | ERROR | stderr | | async for part in self._httpcore_stream: 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\httpcore_async\connection_pool.py", line 347, in aiter 2023-10-06 21:56:37 | ERROR | stderr | | async for part in self._stream: 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\httpcore_async\http11.py", line 325, in aiter 2023-10-06 21:56:37 | ERROR | stderr | | raise exc 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\httpcore_async\http11.py", line 317, in aiter 2023-10-06 21:56:37 | ERROR | stderr | | async for chunk in self._connection._receive_response_body(kwargs): 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\httpcore_async\http11.py", line 186, in _receive_response_body 2023-10-06 21:56:37 | ERROR | stderr | | event = await self._receive_event(timeout=timeout) 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\httpcore_async\http11.py", line 196, in _receive_event 2023-10-06 21:56:37 | ERROR | stderr | | with map_exceptions({h11.RemoteProtocolError: RemoteProtocolError}): 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\contextlib.py", line 153, in exit 2023-10-06 21:56:37 | ERROR | stderr | | self.gen.throw(typ, value, traceback) 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\httpcore_exceptions.py", line 14, in map_exceptions 2023-10-06 21:56:37 | ERROR | stderr | | raise to_exc(exc) from exc 2023-10-06 21:56:37 | ERROR | stderr | | httpcore.RemoteProtocolError: peer closed connection without sending complete message body (incomplete chunked read) 2023-10-06 21:56:37 | ERROR | stderr | | 2023-10-06 21:56:37 | ERROR | stderr | | The above exception was the direct cause of the following exception: 2023-10-06 21:56:37 | ERROR | stderr | | 2023-10-06 21:56:37 | ERROR | stderr | | Traceback (most recent call last): 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\starlette\responses.py", line 273, in wrap 2023-10-06 21:56:37 | ERROR | stderr | | await func() 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\starlette\responses.py", line 262, in stream_response 2023-10-06 21:56:37 | ERROR | stderr | | async for chunk in self.body_iterator: 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\fastchat\serve\openai_api_server.py", line 431, in chat_completion_stream_generator 2023-10-06 21:56:37 | ERROR | stderr | | async for content in generate_completion_stream(gen_params, worker_addr): 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\fastchat\serve\openai_api_server.py", line 594, in generate_completion_stream 2023-10-06 21:56:37 | ERROR | stderr | | async for raw_chunk in response.aiter_raw(): 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\httpx_models.py", line 985, in aiter_raw 2023-10-06 21:56:37 | ERROR | stderr | | async for raw_stream_bytes in self.stream: 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\httpx_client.py", line 146, in aiter 2023-10-06 21:56:37 | ERROR | stderr | | async for chunk in self._stream: 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\httpx_transports\default.py", line 238, in aiter 2023-10-06 21:56:37 | ERROR | stderr | | with map_httpcore_exceptions(): 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\contextlib.py", line 153, in exit 2023-10-06 21:56:37 | ERROR | stderr | | self.gen.throw(typ, value, traceback) 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\httpx_transports\default.py", line 77, in map_httpcore_exceptions 2023-10-06 21:56:37 | ERROR | stderr | | raise mapped_exc(message) from exc 2023-10-06 21:56:37 | ERROR | stderr | | httpx.RemoteProtocolError: peer closed connection without sending complete message body (incomplete chunked read) 2023-10-06 21:56:37 | ERROR | stderr | +------------------------------------ 2023-10-06 21:56:37,591 - utils.py[line:15] - ERROR: ClientPayloadError: Caught exception: Response payload is not completed

启动 2023-10-06 21:56:14,999 - _client.py[line:1013] - INFO: HTTP Request: POST http://192.168.1.43:7861/llm_model/list_models "HTTP/1.1 200 OK" 2023-10-06 21:56:28 | INFO | httpx | HTTP Request: POST http://192.168.1.43:7861/llm_model/list_models "HTTP/1.1 200 OK" received input message: {'history': [{'content': '你好', 'role': 'user'}], 'knowledge_base_name': 'samples', 'local_doc_url': False, 'model_name': 'chatglm2-6b-int4', 'query': '你好', 'score_threshold': 1.0, 'stream': True, 'temperature': 0.7, 'top_k': 3} 2023-10-06 21:56:28 | INFO | httpx | HTTP Request: POST http://192.168.1.43:7861/chat/knowledge_base_chat "HTTP/1.1 200 OK" 2023-10-06 21:56:35 | INFO | httpx | HTTP Request: POST http://192.168.1.43:7861/llm_model/list_models "HTTP/1.1 200 OK" {'docs': [...]} 2023-10-06 21:56:37.592 Uncaught app exception Traceback (most recent call last): File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\streamlit\runtime\scriptrunner\script_runner.py", line 541, in _run_script exec(code, module.dict) File "C:\chatGLM1\Langchain-Chatchat\webui.py", line 66, in pages[selected_page]"func" File "C:\chatGLM1\Langchain-Chatchat\webui_pages\dialogue\dialogue.py", line 171, in dialogue_page chat_box.update_msg(text, element_index=0, streaming=False) File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\streamlit_chatbox\messages.py", line 294, in update_msg assert self.chat_inited, "please call init_session first" AssertionError: please call init_session first

试试换个模型呢，你现在用的是chatglm-6b-int4，你试试chatglm-6b呢

[kernelLZ]

一样的问题，之前还可以运行，过了个节不知道怎么运行就出现这问题了

[jinghong123]

收到感谢

在 2023-10-09 17:57:59，"nanisnull" @.***> 写道：

是不是模型的问题，你现在用的是chatglm-6b-int4；你试试chatglm-6b呢

问题复现 1.python.exe init_database.py 2.python.exe startup.py --all-api 3.python.exe server\llm_api.py 4.streamlit run webui.py --server.port 7860

结果：

==============================Langchain-Chatchat Configuration============================== 操作系统：Windows-10-10.0.22621-SP0. python版本：3.10.0 | packaged by conda-forge | (default, Nov 10 2021, 13:20:59) [MSC v.1916 64 bit (AMD64)] 项目版本：v0.2.4 langchain版本：0.0.287. fastchat版本：0.2.28

当前使用的分词器：ChineseRecursiveTextSplitter 当前启动的LLM模型：['chatglm2-6b-int4'] @ cuda {'api_base_url': 'http://192.168.1.43:8888/v1', 'api_key': 'EMPTY', 'local_model_path': 'C:\chatGLM1\Langchain-Chatchat\model\chatglm2-6b-int4'} 当前Embbedings模型： m3e-base @ cuda ==============================Langchain-Chatchat Configuration==============================

database talbes created filling kb infos to database 2023-10-06 21:54:33,088 - faiss_cache.py[line:74] - INFO: loading vector store in 'samples' from disk. {'kb_name': 'samples', 'vs_type': 'faiss', 'embed_model': 'm3e-base', 'file_count': 1, 'create_time': datetime.datetime(2023, 9, 26, 15, 19, 21)} 2023-10-06 21:54:33,288 - SentenceTransformer.py[line:66] - INFO: Load pretrained SentenceTransformer: C:\chatGLM1\Langchain-Chatchat\model\m3e-base 2023-10-06 21:54:34,486 - loader.py[line:54] - INFO: Loading faiss with AVX2 support. 2023-10-06 21:54:34,487 - loader.py[line:58] - INFO: Could not load library with AVX2 support due to: ModuleNotFoundError("No module named 'faiss.swigfaiss_avx2'") 2023-10-06 21:54:34,487 - loader.py[line:64] - INFO: Loading faiss. 2023-10-06 21:54:34,564 - loader.py[line:66] - INFO: Successfully loaded faiss. 已将 samples/test.txt 添加到数据库 总计用时： 0:00:01.527891 (ChatGLM1) PS C:\chatGLM1\Langchain-Chatchat> C:\Users\zhang\anaconda3\envs\ChatGLM1\python.exe startup.py --all-api

==============================Langchain-Chatchat Configuration============================== 操作系统：Windows-10-10.0.22621-SP0. python版本：3.10.0 | packaged by conda-forge | (default, Nov 10 2021, 13:20:59) [MSC v.1916 64 bit (AMD64)] 项目版本：v0.2.4 langchain版本：0.0.287. fastchat版本：0.2.28

当前使用的分词器：ChineseRecursiveTextSplitter 当前启动的LLM模型：['chatglm2-6b-int4'] @ cuda {'api_base_url': 'http://192.168.1.43:8888/v1', 'api_key': 'EMPTY', 'local_model_path': 'C:\chatGLM1\Langchain-Chatchat\model\chatglm2-6b-int4'} 当前Embbedings模型： m3e-base @ cuda ==============================Langchain-Chatchat Configuration==============================

2023-10-06 21:54:45,187 - startup.py[line:541] - INFO: 正在启动服务： 2023-10-06 21:54:45,187 - startup.py[line:542] - INFO: 如需查看 llm_api 日志，请前往 C:\chatGLM1\Langchain-Chatchat\logs 2023-10-06 21:54:47 | ERROR | stderr | INFO: Started server process [20580] 2023-10-06 21:54:47 | ERROR | stderr | INFO: Waiting for application startup. 2023-10-06 21:54:47 | ERROR | stderr | INFO: Application startup complete. 2023-10-06 21:54:47 | ERROR | stderr | INFO: Uvicorn running on http://192.168.1.43:8888 (Press CTRL+C to quit) 2023-10-06 21:54:52 | WARNING | transformers_modules.chatglm2-6b-int4.quantization | Failed to load cpm_kernels:No module named 'cpm_kernels' C:\Users\zhang.cache\huggingface\modules\transformers_modules\chatglm2-6b-int4\quantization_kernels_parallel.c:1:0: warning: -fPIC ignored for target (all code is position independent) #include ^ 2023-10-06 21:54:53 | WARNING | transformers_modules.chatglm2-6b-int4.quantization | Load parallel cpu kernel failed C:\Users\zhang.cache\huggingface\modules\transformers_modules\chatglm2-6b-int4\quantization_kernels_parallel.so: Traceback (most recent call last): File "C:\Users\zhang/.cache\huggingface\modules\transformers_modules\chatglm2-6b-int4\quantization.py", line 148, in init kernels = ctypes.cdll.LoadLibrary(kernel_file) File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\ctypesinit.py", line 452, in LoadLibrary return self.dlltype(name) File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\ctypes_init.py", line 374, in init self._handle = _dlopen(self._name, mode) FileNotFoundError: Could not find module 'C:\Users\zhang.cache\huggingface\modules\transformers_modules\chatglm2-6b-int4\quantization_kernels_parallel.so' (or one of its dependencies). Try using the full path with constructor syntax.

C:\Users\zhang.cache\huggingface\modules\transformers_modules\chatglm2-6b-int4\quantization_kernels.c:1:0: warning: -fPIC ignored for target (all code is position independent) void compress_int4_weight(void weight, void out, int n, int m) ^ INFO: Started server process [12124] INFO: Waiting for application startup.

INFO: Application startup complete.

==============================Langchain-Chatchat Configuration============================== 操作系统：Windows-10-10.0.22621-SP0. python版本：3.10.0 | packaged by conda-forge | (default, Nov 10 2021, 13:20:59) [MSC v.1916 64 bit (AMD64)] 项目版本：v0.2.4 langchain版本：0.0.287. fastchat版本：0.2.28

当前使用的分词器：ChineseRecursiveTextSplitter INFO: Uvicorn running on http://192.168.1.43:7861 (Press CTRL+C to quit) 当前启动的LLM模型：['chatglm2-6b-int4'] @ cuda {'api_base_url': 'http://192.168.1.43:8888/v1', 'api_key': 'EMPTY', 'local_model_path': 'C:\chatGLM1\Langchain-Chatchat\model\chatglm2-6b-int4'} 当前Embbedings模型： m3e-base @ cuda

服务端运行信息： OpenAI API Server: http://192.168.1.43:8888/v1 (请确认llm_model_dict中配置的api_base_url与上面地址一致。) Chatchat API Server: http://192.168.1.43:7861 ==============================Langchain-Chatchat Configuration==============================

2023-10-06 21:56:14,996 - _client.py[line:1013] - INFO: HTTP Request: POST http://192.168.1.43:20001/list_models "HTTP/1.1 200 OK" INFO: 192.168.1.43:49612 - "POST /llm_model/list_models HTTP/1.1" 200 OK 2023-10-06 21:56:28,717 - _client.py[line:1013] - INFO: HTTP Request: POST http://192.168.1.43:20001/list_models "HTTP/1.1 200 OK" INFO: 192.168.1.43:49663 - "POST /llm_model/list_models HTTP/1.1" 200 OK INFO: 192.168.1.43:49669 - "POST /chat/knowledge_base_chat HTTP/1.1" 200 OK 2023-10-06 21:56:28,872 - SentenceTransformer.py[line:66] - INFO: Load pretrained SentenceTransformer: C:\chatGLM1\Langchain-Chatchat\model\m3e-base 2023-10-06 21:56:29,752 - faiss_cache.py[line:74] - INFO: loading vector store in 'samples' from disk. {'kb_name': 'samples', 'vs_type': 'faiss', 'embed_model': 'm3e-base', 'file_count': 1, 'create_time': datetime.datetime(2023, 9, 26, 15, 19, 21)} 2023-10-06 21:56:29,756 - loader.py[line:54] - INFO: Loading faiss with AVX2 support. 2023-10-06 21:56:29,756 - loader.py[line:58] - INFO: Could not load library with AVX2 support due to: ModuleNotFoundError("No module named 'faiss.swigfaiss_avx2'") 2023-10-06 21:56:29,756 - loader.py[line:64] - INFO: Loading faiss. 2023-10-06 21:56:29,773 - loader.py[line:66] - INFO: Successfully loaded faiss. Batches: 100%|███████████████████████████████████████████████████████████████████████████| 1/1 [00:05<00:00, 5.70s/it] 2023-10-06 21:56:35,779 - _client.py[line:1013] - INFO: HTTP Request: POST http://192.168.1.43:20001/list_models "HTTP/1.1 200 OK" INFO: 192.168.1.43:49684 - "POST /llm_model/list_models HTTP/1.1" 200 OK 2023-10-06 21:56:35 | INFO | stdout | INFO: 192.168.1.43:49693 - "POST /v1/chat/completions HTTP/1.1" 200 OK 2023-10-06 21:56:35,803 - util.py[line:67] - INFO: message='OpenAI API response' path=http://192.168.1.43:8888/v1/chat/completions processing_ms=None request_id=None response_code=200 2023-10-06 21:56:35 | INFO | httpx | HTTP Request: POST http://192.168.1.43:20002/worker_generate_stream "HTTP/1.1 200 OK" ERROR: Exception in ASGI application Traceback (most recent call last): File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\starlette\responses.py", line 277, in call await wrap(partial(self.listen_for_disconnect, receive)) File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\starlette\responses.py", line 273, in wrap await func() File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\starlette\responses.py", line 250, in listen_for_disconnect message = await receive() File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\uvicorn\protocols\http\h11_impl.py", line 538, in receive await self.message_event.wait() File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\asyncio\locks.py", line 213, in wait await fut asyncio.exceptions.CancelledError: Cancelled by cancel scope 1b76314b1c0

During handling of the above exception, another exception occurred:

Exception Group Traceback (most recent call last): | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\uvicorn\protocols\http\h11_impl.py", line 408, in run_asgi | result = await app( # type: ignore[func-returns-value] | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\uvicorn\middleware\proxy_headers.py", line 84, in call | return await self.app(scope, receive, send) | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\fastapi\applications.py", line 290, in call | await super().call(scope, receive, send) | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\starlette\applications.py", line 122, in call | await self.middleware_stack(scope, receive, send) | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\starlette\middleware\errors.py", line 184, in call | raise exc | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\starlette\middleware\errors.py", line 162, in call | await self.app(scope, receive, _send) | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\starlette\middleware\exceptions.py", line 79, in call | raise exc | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\starlette\middleware\exceptions.py", line 68, in call | await self.app(scope, receive, sender) | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\fastapi\middleware\asyncexitstack.py", line 20, in call | raise e | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\fastapi\middleware\asyncexitstack.py", line 17, in call | await self.app(scope, receive, send) | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\starlette\routing.py", line 718, in call | await route.handle(scope, receive, send) | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\starlette\routing.py", line 276, in handle | await self.app(scope, receive, send) | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\starlette\routing.py", line 69, in app | await response(scope, receive, send) | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\starlette\responses.py", line 270, in call | async with anyio.create_task_group() as task_group: | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\anyio_backends_asyncio.py", line 664, in aexit | raise BaseExceptionGroup( | exceptiongroup.ExceptionGroup: unhandled errors in a TaskGroup (1 sub-exception) +-+---------------- 1 ---------------- | Traceback (most recent call last): | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\starlette\responses.py", line 273, in wrap | await func() | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\starlette\responses.py", line 262, in stream_response | async for chunk in self.body_iterator: | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\starlette\concurrency.py", line 63, in iterate_in_threadpool | yield await anyio.to_thread.run_sync(_next, iterator) | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\anyio\to_thread.py", line 33, in run_sync | return await get_async_backend().run_sync_in_worker_thread( | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\anyio_backends_asyncio.py", line 2106, in run_sync_in_worker_thread | return await future | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\anyio_backends_asyncio.py", line 833, in run | result = context.run(func, args) | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\starlette\concurrency.py", line 53, in _next | return next(iterator) | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\fastchat\serve\model_worker.py", line 238, in generate_stream_gate | for output in self.generate_stream_func( | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\torch\utils_contextlib.py", line 35, in generator_context | response = gen.send(None) | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\fastchat\model\model_chatglm.py", line 71, in generate_stream_chatglm | for total_ids in model.stream_generate(inputs, gen_kwargs): | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\torch\utils_contextlib.py", line 35, in generator_context | response = gen.send(None) | File "C:\Users\zhang/.cache\huggingface\modules\transformers_modules\chatglm2-6b-int4\modeling_chatglm.py", line 1143, in stream_generate | outputs = self( | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl | return forward_call(args, kwargs) | File "C:\Users\zhang/.cache\huggingface\modules\transformers_modules\chatglm2-6b-int4\modeling_chatglm.py", line 932, in forward | transformer_outputs = self.transformer( | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl | return forward_call(*args, *kwargs) | File "C:\Users\zhang/.cache\huggingface\modules\transformers_modules\chatglm2-6b-int4\modeling_chatglm.py", line 828, in forward | hidden_states, presents, all_hidden_states, all_self_attentions = self.encoder( | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl | return forward_call(args, kwargs) | File "C:\Users\zhang/.cache\huggingface\modules\transformers_modules\chatglm2-6b-int4\modeling_chatglm.py", line 638, in forward | layer_ret = layer( | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl | return forward_call(*args, kwargs) | File "C:\Users\zhang/.cache\huggingface\modules\transformers_modules\chatglm2-6b-int4\modeling_chatglm.py", line 542, in forward | attention_output, kv_cache = self.self_attention( | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl | return forward_call(*args, *kwargs) | File "C:\Users\zhang/.cache\huggingface\modules\transformers_modules\chatglm2-6b-int4\modeling_chatglm.py", line 374, in forward | mixed_x_layer = self.query_key_value(hidden_states) | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl | return forward_call(args, kwargs) | File "C:\Users\zhang/.cache\huggingface\modules\transformers_modules\chatglm2-6b-int4\quantization.py", line 502, in forward | output = W8A16Linear.apply(input, self.weight, self.weight_scale, self.weight_bit_width) | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\torch\autograd\function.py", line 506, in apply | return super().apply(*args, **kwargs) # type: ignore[misc] | File "C:\Users\zhang/.cache\huggingface\modules\transformers_modules\chatglm2-6b-int4\quantization.py", line 75, in forward | weight = extract_weight_to_half(quant_w, scale_w, weight_bit_width) | File "C:\Users\zhang/.cache\huggingface\modules\transformers_modules\chatglm2-6b-int4\quantization.py", line 287, in extract_weight_to_half | func = kernels.int4WeightExtractionHalf | AttributeError: 'NoneType' object has no attribute 'int4WeightExtractionHalf' +------------------------------------ 2023-10-06 21:56:37 | ERROR | stderr | ERROR: Exception in ASGI application 2023-10-06 21:56:37 | ERROR | stderr | Traceback (most recent call last): 2023-10-06 21:56:37 | ERROR | stderr | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\starlette\responses.py", line 277, in call 2023-10-06 21:56:37 | ERROR | stderr | await wrap(partial(self.listen_for_disconnect, receive)) 2023-10-06 21:56:37 | ERROR | stderr | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\starlette\responses.py", line 273, in wrap 2023-10-06 21:56:37 | ERROR | stderr | await func() 2023-10-06 21:56:37 | ERROR | stderr | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\starlette\responses.py", line 250, in listen_for_disconnect 2023-10-06 21:56:37 | ERROR | stderr | message = await receive() 2023-10-06 21:56:37 | ERROR | stderr | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\uvicorn\protocols\http\h11_impl.py", line 538, in receive 2023-10-06 21:56:37 | ERROR | stderr | await self.message_event.wait() 2023-10-06 21:56:37 | ERROR | stderr | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\asyncio\locks.py", line 213, in wait 2023-10-06 21:56:37 | ERROR | stderr | await fut 2023-10-06 21:56:37 | ERROR | stderr | asyncio.exceptions.CancelledError: Cancelled by cancel scope 1f5dc24cc10 2023-10-06 21:56:37 | ERROR | stderr | 2023-10-06 21:56:37 | ERROR | stderr | During handling of the above exception, another exception occurred: 2023-10-06 21:56:37 | ERROR | stderr | 2023-10-06 21:56:37 | ERROR | stderr | + Exception Group Traceback (most recent call last): 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\uvicorn\protocols\http\h11_impl.py", line 408, in run_asgi 2023-10-06 21:56:37 | ERROR | stderr | | result = await app( # type: ignore[func-returns-value] 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\uvicorn\middleware\proxy_headers.py", line 84, in call 2023-10-06 21:56:37 | ERROR | stderr | | return await self.app(scope, receive, send) 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\fastapi\applications.py", line 290, in call 2023-10-06 21:56:37 | ERROR | stderr | | await super().call(scope, receive, send) 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\starlette\applications.py", line 122, in call 2023-10-06 21:56:37 | ERROR | stderr | | await self.middleware_stack(scope, receive, send) 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\starlette\middleware\errors.py", line 184, in call 2023-10-06 21:56:37 | ERROR | stderr | | raise exc 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\starlette\middleware\errors.py", line 162, in call 2023-10-06 21:56:37 | ERROR | stderr | | await self.app(scope, receive, _send) 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\starlette\middleware\cors.py", line 83, in call 2023-10-06 21:56:37 | ERROR | stderr | | await self.app(scope, receive, send) 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\starlette\middleware\exceptions.py", line 79, in call 2023-10-06 21:56:37 | ERROR | stderr | | raise exc 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\starlette\middleware\exceptions.py", line 68, in call 2023-10-06 21:56:37 | ERROR | stderr | | await self.app(scope, receive, sender) 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\fastapi\middleware\asyncexitstack.py", line 20, in call 2023-10-06 21:56:37 | ERROR | stderr | | raise e 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\fastapi\middleware\asyncexitstack.py", line 17, in call 2023-10-06 21:56:37 | ERROR | stderr | | await self.app(scope, receive, send) 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\starlette\routing.py", line 718, in call 2023-10-06 21:56:37 | ERROR | stderr | | await route.handle(scope, receive, send) 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\starlette\routing.py", line 276, in handle 2023-10-06 21:56:37 | ERROR | stderr | | await self.app(scope, receive, send) 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\starlette\routing.py", line 69, in app 2023-10-06 21:56:37 | ERROR | stderr | | await response(scope, receive, send) 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\starlette\responses.py", line 270, in call 2023-10-06 21:56:37 | ERROR | stderr | | async with anyio.create_task_group() as task_group: 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\anyio_backends_asyncio.py", line 664, in aexit 2023-10-06 21:56:37 | ERROR | stderr | | raise BaseExceptionGroup( 2023-10-06 21:56:37 | ERROR | stderr | | exceptiongroup.ExceptionGroup: unhandled errors in a TaskGroup (1 sub-exception) 2023-10-06 21:56:37 | ERROR | stderr | +-+---------------- 1 ---------------- 2023-10-06 21:56:37 | ERROR | stderr | | Traceback (most recent call last): 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\httpcore_exceptions.py", line 10, in map_exceptions 2023-10-06 21:56:37 | ERROR | stderr | | yield 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\httpcore_async\http11.py", line 197, in _receive_event 2023-10-06 21:56:37 | ERROR | stderr | | event = self._h11_state.next_event() 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\h11_connection.py", line 469, in next_event 2023-10-06 21:56:37 | ERROR | stderr | | event = self._extract_next_receive_event() 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\h11_connection.py", line 419, in _extract_next_receive_event 2023-10-06 21:56:37 | ERROR | stderr | | event = self._reader.read_eof() # type: ignore[attr-defined] 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\h11_readers.py", line 204, in read_eof 2023-10-06 21:56:37 | ERROR | stderr | | raise RemoteProtocolError( 2023-10-06 21:56:37 | ERROR | stderr | | h11._util.RemoteProtocolError: peer closed connection without sending complete message body (incomplete chunked read) 2023-10-06 21:56:37 | ERROR | stderr | | 2023-10-06 21:56:37 | ERROR | stderr | | The above exception was the direct cause of the following exception: 2023-10-06 21:56:37 | ERROR | stderr | | 2023-10-06 21:56:37 | ERROR | stderr | | Traceback (most recent call last): 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\httpx_transports\default.py", line 60, in map_httpcore_exceptions 2023-10-06 21:56:37 | ERROR | stderr | | yield 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\httpx_transports\default.py", line 239, in aiter 2023-10-06 21:56:37 | ERROR | stderr | | async for part in self._httpcore_stream: 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\httpcore_async\connection_pool.py", line 347, in aiter 2023-10-06 21:56:37 | ERROR | stderr | | async for part in self._stream: 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\httpcore_async\http11.py", line 325, in aiter 2023-10-06 21:56:37 | ERROR | stderr | | raise exc 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\httpcore_async\http11.py", line 317, in aiter 2023-10-06 21:56:37 | ERROR | stderr | | async for chunk in self._connection._receive_response_body(**kwargs): 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\httpcore_async\http11.py", line 186, in _receive_response_body 2023-10-06 21:56:37 | ERROR | stderr | | event = await self._receive_event(timeout=timeout) 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\httpcore_async\http11.py", line 196, in _receive_event 2023-10-06 21:56:37 | ERROR | stderr | | with map_exceptions({h11.RemoteProtocolError: RemoteProtocolError}): 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\contextlib.py", line 153, in exit 2023-10-06 21:56:37 | ERROR | stderr | | self.gen.throw(typ, value, traceback) 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\httpcore_exceptions.py", line 14, in map_exceptions 2023-10-06 21:56:37 | ERROR | stderr | | raise to_exc(exc) from exc 2023-10-06 21:56:37 | ERROR | stderr | | httpcore.RemoteProtocolError: peer closed connection without sending complete message body (incomplete chunked read) 2023-10-06 21:56:37 | ERROR | stderr | | 2023-10-06 21:56:37 | ERROR | stderr | | The above exception was the direct cause of the following exception: 2023-10-06 21:56:37 | ERROR | stderr | | 2023-10-06 21:56:37 | ERROR | stderr | | Traceback (most recent call last): 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\starlette\responses.py", line 273, in wrap 2023-10-06 21:56:37 | ERROR | stderr | | await func() 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\starlette\responses.py", line 262, in stream_response 2023-10-06 21:56:37 | ERROR | stderr | | async for chunk in self.body_iterator: 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\fastchat\serve\openai_api_server.py", line 431, in chat_completion_stream_generator 2023-10-06 21:56:37 | ERROR | stderr | | async for content in generate_completion_stream(gen_params, worker_addr): 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\fastchat\serve\openai_api_server.py", line 594, in generate_completion_stream 2023-10-06 21:56:37 | ERROR | stderr | | async for raw_chunk in response.aiter_raw(): 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\httpx_models.py", line 985, in aiter_raw 2023-10-06 21:56:37 | ERROR | stderr | | async for raw_stream_bytes in self.stream: 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\httpx_client.py", line 146, in aiter 2023-10-06 21:56:37 | ERROR | stderr | | async for chunk in self._stream: 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\httpx_transports\default.py", line 238, in aiter 2023-10-06 21:56:37 | ERROR | stderr | | with map_httpcore_exceptions(): 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\contextlib.py", line 153, in exit 2023-10-06 21:56:37 | ERROR | stderr | | self.gen.throw(typ, value, traceback) 2023-10-06 21:56:37 | ERROR | stderr | | File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\httpx_transports\default.py", line 77, in map_httpcore_exceptions 2023-10-06 21:56:37 | ERROR | stderr | | raise mapped_exc(message) from exc 2023-10-06 21:56:37 | ERROR | stderr | | httpx.RemoteProtocolError: peer closed connection without sending complete message body (incomplete chunked read) 2023-10-06 21:56:37 | ERROR | stderr | +------------------------------------ 2023-10-06 21:56:37,591 - utils.py[line:15] - ERROR: ClientPayloadError: Caught exception: Response payload is not completed

启动 2023-10-06 21:56:14,999 - _client.py[line:1013] - INFO: HTTP Request: POST http://192.168.1.43:7861/llm_model/list_models "HTTP/1.1 200 OK" 2023-10-06 21:56:28 | INFO | httpx | HTTP Request: POST http://192.168.1.43:7861/llm_model/list_models "HTTP/1.1 200 OK" received input message: {'history': [{'content': '你好', 'role': 'user'}], 'knowledge_base_name': 'samples', 'local_doc_url': False, 'model_name': 'chatglm2-6b-int4', 'query': '你好', 'score_threshold': 1.0, 'stream': True, 'temperature': 0.7, 'top_k': 3} 2023-10-06 21:56:28 | INFO | httpx | HTTP Request: POST http://192.168.1.43:7861/chat/knowledge_base_chat "HTTP/1.1 200 OK" 2023-10-06 21:56:35 | INFO | httpx | HTTP Request: POST http://192.168.1.43:7861/llm_model/list_models "HTTP/1.1 200 OK" {'docs': [...]} 2023-10-06 21:56:37.592 Uncaught app exception Traceback (most recent call last): File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\streamlit\runtime\scriptrunner\script_runner.py", line 541, in _run_script exec(code, module.dict) File "C:\chatGLM1\Langchain-Chatchat\webui.py", line 66, in pages[selected_page]"func" File "C:\chatGLM1\Langchain-Chatchat\webui_pages\dialogue\dialogue.py", line 171, in dialogue_page chat_box.update_msg(text, element_index=0, streaming=False) File "C:\Users\zhang\anaconda3\envs\ChatGLM1\lib\site-packages\streamlit_chatbox\messages.py", line 294, in update_msg assert self.chat_inited, "please call init_session first" AssertionError: please call init_session first

试试换个模型呢，你现在用的是chatglm-6b-int4，你试试chatglm-6b呢

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: @.***>

[qiyuanzhang520]

int4WeightExtractionHalf如有报错 可安装cpm_kernels库，如此问题解决 pip install cpm_kernels

[jinghong123]

太感谢了 祝您顺利

在 2023-10-18 16:17:54，"qiyuanzhang520" @.***> 写道：

int4WeightExtractionHalf如有报错 可安装cpm_kernels库，如此问题解决 pip install cpm_kernels

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: @.***>

[Sumail2020]

int4WeightExtractionHalf如有报错 可安装cpm_kernels库，如此问题解决 pip install cpm_kernels

还是没有解决模型不回答的问题