Closed ye7love7 closed 3 months ago
internlm/internlm-xcomposer-vl-7b 这个很久以前在以demo的形式支持过,并没有正式支持,所以没办法用serve部署或者使用pipeline的接口。
internlm/internlm-xcomposer2-vl-7b 使用server/pipeline接口支持后,就不打算再支持 internlm/internlm-xcomposer-vl-7b 这个旧的模型了。
使用sudo docker run --runtime nvidia --gpus all
-v ~/.cache/huggingface:/root/.cache/huggingface
-v /home/tskj/MOD:/home/MOD/
-p 23333:23333
--ipc=host
openmmlab/lmdeploy:v0.4.1
lmdeploy serve api_server /home/MOD/internlm-xcomposer2-vl-7b
报以下错误:OSError: We couldn't connect to 'https://huggingface.co/' to load this file, couldn't find it in the cached files and it looks like openai/clip-vit-large-patch14-336 is not the path to a directory containing a file named config.json.
Checkout your internet connection or see how to run the library in offline mode at 'https://huggingface.co/docs/transformers/installation#offline-mode'.
这个报错是网络问题,internlm/internlm-xcomposer2-vl-7b 加载是需要联网的,你如果用xcomposer2 提供的 transformers的代码加载的话还需要下载这个模型。LMDeploy 进行了重写,不需要下载模型,但是需要去下载config信息,你如果要完全离线的话,需要去改LMDeploy 这里的重写代码。将config = CLIPVisionConfig.from_pretrained(vision_tower_name)
这一行联网加载,使用本地加载的方式替代,内容的话使用 openai/clip-vit-large-patch14-336 里面的config.json。
Checklist
Describe the bug
0.4.1版本下,使用sudo docker run --runtime nvidia --gpus all \ -v ~/.cache/huggingface:/root/.cache/huggingface \ -v /home/tskj/MOD:/home/MOD/ \ -p 23333:23333 \ --ipc=host \ openmmlab/lmdeploy:v0.4.1 \ lmdeploy serve api_server /home/MOD/internlm-xcomposer-vl-7b internlm/internlm-xcomposer-vl-7b报以下错误: Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained. Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained. Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained. Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained. Traceback (most recent call last): File "/opt/py38/bin/lmdeploy", line 11, in
load_entry_point('lmdeploy', 'console_scripts', 'lmdeploy')()
File "/opt/lmdeploy/lmdeploy/cli/entrypoint.py", line 37, in run
args.run(args)
File "/opt/lmdeploy/lmdeploy/cli/serve.py", line 283, in api_server
run_api_server(args.model_path,
File "/opt/lmdeploy/lmdeploy/serve/openai/api_server.py", line 1191, in serve
VariableInterface.async_engine = pipeline_class(
File "/opt/lmdeploy/lmdeploy/serve/async_engine.py", line 206, in init
self._build_turbomind(model_path=model_path,
File "/opt/lmdeploy/lmdeploy/serve/async_engine.py", line 254, in _build_turbomind
self.engine = tm.TurboMind.from_pretrained(
File "/opt/lmdeploy/lmdeploy/turbomind/turbomind.py", line 396, in from_pretrained
return cls(model_path=pretrained_model_name_or_path,
File "/opt/lmdeploy/lmdeploy/turbomind/turbomind.py", line 170, in init
self.model_comm = self._from_hf(model_source=model_source,
File "/opt/lmdeploy/lmdeploy/turbomind/turbomind.py", line 279, in _from_hf
output_model = OUTPUT_MODELS.get(output_format)(
File "/opt/lmdeploy/lmdeploy/turbomind/deploy/target_model/fp.py", line 26, in init
super().init(input_model, cfg, to_file, out_dir)
File "/opt/lmdeploy/lmdeploy/turbomind/deploy/target_model/base.py", line 155, in init
self.cfg = self.get_config(cfg)
File "/opt/lmdeploy/lmdeploy/turbomind/deploy/target_model/fp.py", line 30, in get_config
final_cfg = super().get_config(cfg).dict
File "/opt/lmdeploy/lmdeploy/turbomind/deploy/target_model/base.py", line 180, in get_config
final_cfg.update(dict(head_num=head_num, vocab_size=_vocab_size))
UnboundLocalError: local variable 'head_num' referenced before assignment
使用sudo docker run --runtime nvidia --gpus all \
-v ~/.cache/huggingface:/root/.cache/huggingface \
-v /home/tskj/MOD:/home/MOD/ \
-p 23333:23333 \
--ipc=host \
openmmlab/lmdeploy:v0.4.1 \
lmdeploy serve api_server /home/MOD/internlm-xcomposer2-vl-7b
报以下错误:OSError: We couldn't connect to 'https://huggingface.co' to load this file, couldn't find it in the cached files and it looks like openai/clip-vit-large-patch14-336 is not the path to a directory containing a file named config.json.
Checkout your internet connection or see how to run the library in offline mode at 'https://huggingface.co/docs/transformers/installation#offline-mode'.
Reproduction
可能需要修改hugging face上的模型文件
Environment
Error traceback
No response