是否可以在服务器上运行？

[yanyilin3344]

Traceback (most recent call last): File "/home/aidata/work/service/ChatLM-mini-Chinese-main/cli_demo.py", line 13, in chat_bot = ChatBot(infer_config=infer_config) File "/home/aidata/work/service/ChatLM-mini-Chinese-main/model/infer.py", line 46, in init model = load_and_quantize_model( File "/home/aisdb1/envs/chatmini/lib/python3.10/site-packages/accelerate/utils/bnb.py", line 193, in load_and_quantize_model return dispatch_model(model, device_map=device_map, offload_dir=offload_folder) File "/home/aisdb1/envs/chatmini/lib/python3.10/site-packages/accelerate/big_modeling.py", line 436, in dispatch_model model.to(device) File "/home/aisdb1/envs/chatmini/lib/python3.10/site-packages/transformers/modeling_utils.py", line 2460, in to return super().to(*args, **kwargs) File "/home/aisdb1/envs/chatmini/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1160, in to return self._apply(convert) File "/home/aisdb1/envs/chatmini/lib/python3.10/site-packages/torch/nn/modules/module.py", line 810, in _apply module._apply(fn) File "/home/aisdb1/envs/chatmini/lib/python3.10/site-packages/torch/nn/modules/module.py", line 833, in _apply param_applied = fn(param) File "/home/aisdb1/envs/chatmini/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1158, in convert return t.to(device, dtype if t.is_floating_point() or t.is_complex() else None, non_blocking) NotImplementedError: Cannot copy out of meta tensor; no data! 在服务器上测试效果时 出现这个报错

[charent]

你重新拉一下代码就可以了，我在Linux下测试没问题了。原来infer.py的4bit量化加载的代码有bug，已经改了。

[yanyilin3344]

感谢，已经成功了

[yanyilin3344]

按照3.7 cli_demo 可以运行出结果， api那里出了问题。按照3.1步骤走出现了这个报错。 Traceback (most recent call last): File "", line 1, in File "/home/aisdb1/envs/chatmini/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py", line 526, in from_pretrained config, kwargs = AutoConfig.from_pretrained( File "/home/aisdb1/envs/chatmini/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py", line 1082, in from_pretrained config_dict, unused_kwargs = PretrainedConfig.get_config_dict(pretrained_model_name_or_path, kwargs) File "/home/aisdb1/envs/chatmini/lib/python3.10/site-packages/transformers/configuration_utils.py", line 644, in get_config_dict config_dict, kwargs = cls._get_config_dict(pretrained_model_name_or_path, kwargs) File "/home/aisdb1/envs/chatmini/lib/python3.10/site-packages/transformers/configuration_utils.py", line 699, in _get_config_dict resolved_config_file = cached_file( File "/home/aisdb1/envs/chatmini/lib/python3.10/site-packages/transformers/utils/hub.py", line 429, in cached_file raise EnvironmentError( OSError: We couldn't connect to 'https://huggingface.co' to load this file, couldn't find it in the cached files and it looks like charent/ChatLM-mini-Chinese is not the path to a directory containing a file named config.json. Checkout your internet connection or see how to run the library in offline mode at 'https://huggingface.co/docs/transformers/installation#offline-mode'. 尝试把model_id 改成ChatLM-mini-Chinese本地路径，没有成功。 咱们这个模型是否可以介入langchain-chatchat 那个项目？

[charent]

第一个问题，如果你运行python cli_demo.py没问题，那么api也应该没问题，这两个使用的是同一套config配置。 我这边运行python api_demo.py正常的：

第二个问题，步骤3.1是给快速尝试模型的同学准备的，就是不想克隆GitHub仓库，单纯想试试模型。能通过python cli_demo.py运行起来就是没有问题了。你给的报错：

We couldn't connect to 'https://huggingface.co/' to load this file, couldn't find it in the cached files and it looks like charent/ChatLM-mini-Chinese is not the path to a directory containing a file named config.json.

应该是你的电脑没有科学上网，无法连接到https://huggingface.co/，改成本地路径的话，要保证你的路径中有configuration_chat_model.py和modeling_chat_model.py这两个文件，解释见另外一个issue：13#issuecomment-1897867963。

第三个问题，我尝试过接入langchain，效果并不好，可能是我在做sft的时候剔除了很多带input的训练数据，你可以自己再做一次sft看看。langchain-chatchat没试过，估计效果也不太行，得重新微调模型才行。

[yanyilin3344]

好的，感谢解答。