reward model合并出错

[dogrepairditch]

python merge_peft_adapter.py --model_type bloom \

--base_model_name_or_path merged-sft --peft_model_path outputs-rm-v1 --output_dir merged-rm/

Namespace(base_model_name_or_path='merged-sft', model_type='bloom', output_dir='merged-rm/', peft_model_path='outputs-rm-v1', resize_emb=False, tokenizer_path=None) Base model: merged-sft LoRA model: outputs-rm-v1 Loading LoRA for sequence classification model Some weights of BloomForSequenceClassification were not initialized from the model checkpoint at merged-sft and are newly initialized: ['score.weight'] You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference. Traceback (most recent call last): File "merge_peft_adapter.py", line 109, in main() File "merge_peft_adapter.py", line 92, in main lora_model = PeftModel.from_pretrained( File "/root/miniconda3/envs/medicalgpt/lib/python3.8/site-packages/peft/peft_model.py", line 306, in from_pretrained model.load_adapter(model_id, adapter_name, is_trainable=is_trainable, **kwargs) File "/root/miniconda3/envs/medicalgpt/lib/python3.8/site-packages/peft/peft_model.py", line 606, in load_adapter load_result = set_peft_model_state_dict(self, adapters_weights, adapter_name=adapter_name) File "/root/miniconda3/envs/medicalgpt/lib/python3.8/site-packages/peft/utils/save_and_load.py", line 158, in set_peft_model_state_dict load_result = model.load_state_dict(peft_model_state_dict, strict=False) File "/root/miniconda3/envs/medicalgpt/lib/python3.8/site-packages/torch/nn/modules/module.py", line 2152, in load_state_dict raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format( RuntimeError: Error(s) in loading state_dict for PeftModelForSequenceClassification: size mismatch for base_model.model.score.modules_to_save.default.weight: copying a param with shape torch.Size([1, 1024]) from checkpoint, the shape in current model is torch.Size([2, 1024]).

[dogrepairditch]

请问为啥合并reward_model的lora参数会报错呢，这个该怎么解决呀

[shibing624]

fixed. https://github.com/shibing624/MedicalGPT/commit/48f86830ded53dd843b0416dfd45b6e942205bc5