Internvl_Chat_v1-2-Plus/Internvl_Chat_v1-5 lora微调后，加载模型报错AssertionError，请问该如何解决？

[HarlynG]

pipe = pipeline(model_path = '/cpfs/data/user/ken/amd_adp/OpenGVLab/Internvl_Chat_v1-2-Plus_lora', model_name = "Internvl_Chat_v1-2-Plus_lora") trainable params: 122,880,000 || all params: 34,511,897,600 || trainable%: 0.35605112597459726 Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained. Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained. Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained. Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained. Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained. Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained. Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained. Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained. Traceback (most recent call last): File "", line 1, in File "/usr/local/lib/python3.10/site-packages/lmdeploy/api.py", line 94, in pipeline return pipeline_class(model_path, File "/usr/local/lib/python3.10/site-packages/lmdeploy/serve/vl_async_engine.py", line 17, in init super().init(model_path, **kwargs) File "/usr/local/lib/python3.10/site-packages/lmdeploy/serve/async_engine.py", line 206, in init self._build_turbomind(model_path=model_path, File "/usr/local/lib/python3.10/site-packages/lmdeploy/serve/async_engine.py", line 254, in _build_turbomind self.engine = tm.TurboMind.from_pretrained( File "/usr/local/lib/python3.10/site-packages/lmdeploy/turbomind/turbomind.py", line 396, in from_pretrained return cls(model_path=pretrained_model_name_or_path, File "/usr/local/lib/python3.10/site-packages/lmdeploy/turbomind/turbomind.py", line 170, in init self.model_comm = self._from_hf(model_source=model_source, File "/usr/local/lib/python3.10/site-packages/lmdeploy/turbomind/turbomind.py", line 279, in _from_hf output_model = OUTPUT_MODELS.get(output_format)( File "/usr/local/lib/python3.10/site-packages/lmdeploy/turbomind/deploy/target_model/fp.py", line 26, in init super().init(input_model, cfg, to_file, out_dir) File "/usr/local/lib/python3.10/site-packages/lmdeploy/turbomind/deploy/target_model/base.py", line 156, in init assert self.cfg.valid AssertionError

[czczup]

可能得把lora的参数merge了再调用lmdeploy

[HarlynG]

可能得把lora的参数merge了再调用lmdeploy

非常感谢！使用https://huggingface.co/OpenGVLab/InternVL-Chat-V1-2-Plus 上的推理代码，也可以正常加载lora微调后的模型

[McCarthyjfq]

请问各位大佬有没有merge lora的教程或者脚本