运行python startup.py -a时报错MetadataIncompleteBuffer

[k1ngr4m]

问题描述 / Problem Description 用简洁明了的语言描述这个问题 / Describe the problem in a clear and concise manner.

复现问题的步骤 / Steps to Reproduce

1. 执行python startup.py -a时报错

(chatchat) ubuntu@VM-4-7-ubuntu:~/chatchat/Langchain-Chatchat$ python startup.py -a

==============================Langchain-Chatchat Configuration============================== 操作系统：Linux-5.15.0-86-generic-x86_64-with-glibc2.35. python版本：3.10.14 (main, Mar 21 2024, 16:24:04) [GCC 11.2.0] 项目版本：v0.2.10 langchain版本：0.0.354. fastchat版本：0.2.35

当前使用的分词器：ChineseRecursiveTextSplitter 当前启动的LLM模型：['chatglm3-6b', 'zhipu-api', 'openai-api'] @ cpu {'device': 'cuda', 'host': '0.0.0.0', 'infer_turbo': False, 'model_path': '/home/ubuntu/chatchat/Langchain-Chatchat/models/chatglm3-6b', 'model_path_exists': True, 'port': 20002} {'api_key': '', 'device': 'auto', 'host': '0.0.0.0', 'infer_turbo': False, 'online_api': True, 'port': 21001, 'provider': 'ChatGLMWorker', 'version': 'glm-4', 'worker_class': <class 'server.model_workers.zhipu.ChatGLMWorker'>} {'api_base_url': 'https://api.openai.com/v1', 'api_key': '', 'device': 'auto', 'host': '0.0.0.0', 'infer_turbo': False, 'model_name': 'gpt-4', 'online_api': True, 'openai_proxy': '', 'port': 20002} 当前Embbedings模型： bge-large-zh-v1.5 @ cpu ==============================Langchain-Chatchat Configuration==============================

2024-04-10 14:34:03,023 - startup.py[line:655] - INFO: 正在启动服务： 2024-04-10 14:34:03,024 - startup.py[line:656] - INFO: 如需查看 llm_api 日志，请前往 /home/ubuntu/chatchat/Langchain-Chatchat/logs /home/ubuntu/anaconda3/envs/chatchat/lib/python3.10/site-packages/langchain_core/_api/deprecation.py:117: LangChainDeprecationWarning: 模型启动功能将于 Langchain-Chatchat 0.3.x重写,支持更多模式和加速启动，0.2.x中相关功能将废弃 warn_deprecated( ERROR: [Errno 98] error while attempting to bind on address ('0.0.0.0', 20001): address already in use 2024-04-10 14:34:10 | ERROR | stderr | INFO: Started server process [320335] 2024-04-10 14:34:10 | ERROR | stderr | INFO: Waiting for application startup. 2024-04-10 14:34:10 | ERROR | stderr | INFO: Application startup complete. 2024-04-10 14:34:10 | ERROR | stderr | ERROR: [Errno 98] error while attempting to bind on address ('0.0.0.0', 20000): address already in use 2024-04-10 14:34:10 | ERROR | stderr | INFO: Waiting for application shutdown. 2024-04-10 14:34:10 | ERROR | stderr | INFO: Application shutdown complete. 2024-04-10 14:34:11 | INFO | model_worker | Register to controller ERROR: [Errno 98] error while attempting to bind on address ('0.0.0.0', 21001): address already in use 2024-04-10 14:34:12 | INFO | model_worker | Loading the model ['chatglm3-6b'] on worker eb050c38 ... 2024-04-10 14:34:12 | WARNING | transformers_modules.chatglm3-6b.tokenization_chatglm | Setting eos_token is not supported, use the default one. 2024-04-10 14:34:12 | WARNING | transformers_modules.chatglm3-6b.tokenization_chatglm | Setting pad_token is not supported, use the default one. 2024-04-10 14:34:12 | WARNING | transformers_modules.chatglm3-6b.tokenization_chatglm | Setting unk_token is not supported, use the default one. Loading checkpoint shards: 0%| | 0/7 [00:00<?, ?it/s] Loading checkpoint shards: 0%| | 0/7 [00:00<?, ?it/s] 2024-04-10 14:34:12 | ERROR | stderr | 2024-04-10 14:34:12 | ERROR | stderr | Process model_worker - chatglm3-6b: 2024-04-10 14:34:12 | ERROR | stderr | Traceback (most recent call last): 2024-04-10 14:34:12 | ERROR | stderr | File "/home/ubuntu/anaconda3/envs/chatchat/lib/python3.10/multiprocessing/process.py", line 314, in _bootstrap 2024-04-10 14:34:12 | ERROR | stderr | self.run() 2024-04-10 14:34:12 | ERROR | stderr | File "/home/ubuntu/anaconda3/envs/chatchat/lib/python3.10/multiprocessing/process.py", line 108, in run 2024-04-10 14:34:12 | ERROR | stderr | self._target(*self._args, self._kwargs) 2024-04-10 14:34:12 | ERROR | stderr | File "/home/ubuntu/chatchat/Langchain-Chatchat/startup.py", line 389, in run_model_worker 2024-04-10 14:34:12 | ERROR | stderr | app = create_model_worker_app(log_level=log_level, kwargs) 2024-04-10 14:34:12 | ERROR | stderr | File "/home/ubuntu/chatchat/Langchain-Chatchat/startup.py", line 217, in create_model_worker_app 2024-04-10 14:34:12 | ERROR | stderr | worker = ModelWorker( 2024-04-10 14:34:12 | ERROR | stderr | File "/home/ubuntu/anaconda3/envs/chatchat/lib/python3.10/site-packages/fastchat/serve/model_worker.py", line 77, in init 2024-04-10 14:34:12 | ERROR | stderr | self.model, self.tokenizer = load_model( 2024-04-10 14:34:12 | ERROR | stderr | File "/home/ubuntu/anaconda3/envs/chatchat/lib/python3.10/site-packages/fastchat/model/model_adapter.py", line 348, in load_model 2024-04-10 14:34:12 | ERROR | stderr | model, tokenizer = adapter.load_model(model_path, kwargs) 2024-04-10 14:34:12 | ERROR | stderr | File "/home/ubuntu/anaconda3/envs/chatchat/lib/python3.10/site-packages/fastchat/model/model_adapter.py", line 826, in load_model 2024-04-10 14:34:12 | ERROR | stderr | model = AutoModel.from_pretrained( 2024-04-10 14:34:12 | ERROR | stderr | File "/home/ubuntu/anaconda3/envs/chatchat/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py", line 561, in from_pretrained 2024-04-10 14:34:12 | ERROR | stderr | return model_class.from_pretrained( 2024-04-10 14:34:12 | ERROR | stderr | File "/home/ubuntu/anaconda3/envs/chatchat/lib/python3.10/site-packages/transformers/modeling_utils.py", line 3850, in from_pretrained 2024-04-10 14:34:12 | ERROR | stderr | ) = cls._load_pretrained_model( 2024-04-10 14:34:12 | ERROR | stderr | File "/home/ubuntu/anaconda3/envs/chatchat/lib/python3.10/site-packages/transformers/modeling_utils.py", line 4259, in _load_pretrained_model 2024-04-10 14:34:12 | ERROR | stderr | state_dict = load_state_dict(shard_file) 2024-04-10 14:34:12 | ERROR | stderr | File "/home/ubuntu/anaconda3/envs/chatchat/lib/python3.10/site-packages/transformers/modeling_utils.py", line 506, in load_state_dict 2024-04-10 14:34:12 | ERROR | stderr | with safe_open(checkpoint_file, framework="pt") as f: 2024-04-10 14:34:12 | ERROR | stderr | safetensors_rust.SafetensorError: Error while deserializing header: MetadataIncompleteBuffer

[dosubot[bot]]

Hey there @k1ngr4m! 👋 Long time no debug, huh? Hope you're doing well amidst all these tech quirks!

The errors you're encountering, including the MetadataIncompleteBuffer error and the address binding error, can be resolved with specific steps:

1. Address Binding Error: This indicates the specified ports are already in use. Use lsof -i :<port_number> to identify if a port is in use. To resolve, either stop the service using that port or modify the configs.py file to use different ports for Langchain-Chatchat services.

2. MetadataIncompleteBuffer Error: This error is related to issues with loading model metadata, possibly due to corrupted model files, insufficient memory, or incorrect model configuration. To address this:

   • Verify the model files for chatglm3-6b, zhipu-api, openai-api, and bge-large-zh-v1.5 are correctly downloaded and accessible.
   • Check the integrity of the chatglm3-6b model files, especially the tokenizer, and ensure they are not corrupted.
   • Confirm there is enough memory to load the models, as they require significant resources.
   • Ensure the model configurations in configs.py and startup.py are correct and compatible with your model versions.

Additionally, be aware of deprecation warnings indicating upcoming changes in functionalities. Adjusting the logging verbosity in configs.py or startup.py might provide more detailed logs for further diagnosis.

If the issue persists after following these steps, please provide more detailed information about the actions you've taken and any new error messages.

Sources

• 启动失败
• [BUG] python startup.py -a后无法启动
• [BUG] RuntimeError: Internal: could not parse ModelProto from chatglm3-6b/tokenizer.model
• [BUG] Langchain-chatchat 一键启动项目 python startup.py -a 报错
• 执行python startup.py -a 报错
• startup.py

#### About Dosu

This response is meant to be useful and save you time. It is **not meant to be a precise solution**, but rather a starting point for your own research. Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant. If you want to continue the conversation, start your reply with @dosu-bot.

[github-actions[bot]]

这个问题已经被标记为 stale ，因为它已经超过 30 天没有任何活动。