RuntimeError: The size of tensor a (4096) must match the size of tensor b (256) at non-singleton dimension 1

[lcykww]

基础模型：llama-7b-hf 我尝试将微调后的参数合并，但出现了错误

Traceback (most recent call last): File "/home/kww/test_model.py", line 24, in <module> merged_model = model.merge_and_unload(safe_merge=True) File "/home/kww/MoRA/peft-mora/src/peft/tuners/lora/model.py", line 721, in merge_and_unload return self._unload_and_optionally_merge( File "/home/kww/MoRA/peft-mora/src/peft/tuners/lora/model.py", line 375, in _unload_and_optionally_merge target.merge(safe_merge=safe_merge, adapter_names=adapter_names) File "/home/kww/MoRA/peft-mora/src/peft/tuners/lora/bnb.py", line 106, in merge w_data = output.to(lora_data.dtype).to(lora_data.device) + lora_data RuntimeError: The size of tensor a (4096) must match the size of tensor b (256) at non-singleton dimension 1

这是我的合并参数的代码 ` from transformers import AutoTokenizer, AutoModelForCausalLM, AutoConfig import torch from peft import PeftModel, LoraConfig, LoraModel, get_peft_model import os

lora_path = "/home/kww/MoRA-main/save_mora_tuning_r8/checkpoint-820" base_model = "/home/kww/MoRA-main/yahma/llama-7b-hf" model = AutoModelForCausalLM.from_pretrained(pretrained_model_name_or_path=base_model, device_map='auto', torch_dtype=torch.bfloat16, load_in_8bit=True)

lora_config = LoraConfig.from_pretrained(lora_path) print(f"lora_config:{lora_config}") model = get_peft_model(model, lora_config) for name, param in model.named_parameters(): print(f"{name}: {param.size()}") merged_model = model.merge_and_unload(safe_merge=True) merged_model.save_pretrained("/home/kww/merged_model")`

[kongds]

这个错误应该是现在的mora实现不支持8bit的merge，可以把load_in_8bit=True去掉试试。

[lcykww]

这个错误应该是现在的mora实现不支持8bit的merge，可以把load_in_8bit=True去掉试试。

可以合并了，感谢