wangzaistone
commented
7 months ago @weikuiyang216 have you solved this problem ? and how you solved it thanks ,some people meet too , would you like to give a pr for us and contribution for the project ? Thanks ~
Oops322
Traceback (most recent call last): File "/home/hp/wen/DB-GPT-Hub-main/dbgpt_hub/predict/predict.py", line 50, in
model = ChatModel()
File "/home/hp/wen/DB-GPT-Hub-main/dbgpt_hub/llm_base/chat_model.py", line 21, in init
self.model, self.tokenizer = load_model_and_tokenizer(
File "/home/hp/wen/DB-GPT-Hub-main/dbgpt_hub/llm_base/load_tokenizer.py", line 294, in load_model_and_tokenizer
model = init_adapter(model, model_args, finetuning_args, is_trainable, is_mergeable)
File "/home/hp/wen/DB-GPT-Hub-main/dbgpt_hub/llm_base/adapter.py", line 94, in init_adapter
model = PeftModel.from_pretrained(
File "/home/hp/miniconda3/envs/dbgpt_hub/lib/python3.10/site-packages/peft/peft_model.py", line 271, in from_pretrained
model.load_adapter(model_id, adapter_name, is_trainable=is_trainable, **kwargs)
File "/home/hp/miniconda3/envs/dbgpt_hub/lib/python3.10/site-packages/peft/peft_model.py", line 561, in load_adapter
load_result = set_peft_model_state_dict(self, adapters_weights, adapter_name=adapter_name)
File "/home/hp/miniconda3/envs/dbgpt_hub/lib/python3.10/site-packages/peft/utils/save_and_load.py", line 126, in set_peft_model_state_dict
load_result = model.load_state_dict(peft_model_state_dict, strict=False)
File "/home/hp/miniconda3/envs/dbgpt_hub/lib/python3.10/site-packages/torch/nn/modules/module.py", line 2138, in load_state_dict
load(self, state_dict)
File "/home/hp/miniconda3/envs/dbgpt_hub/lib/python3.10/site-packages/torch/nn/modules/module.py", line 2126, in load
load(child, child_state_dict, child_prefix)
File "/home/hp/miniconda3/envs/dbgpt_hub/lib/python3.10/site-packages/torch/nn/modules/module.py", line 2126, in load
load(child, child_state_dict, child_prefix)
File "/home/hp/miniconda3/envs/dbgpt_hub/lib/python3.10/site-packages/torch/nn/modules/module.py", line 2126, in load
load(child, child_state_dict, child_prefix)
[Previous line repeated 4 more times]
File "/home/hp/miniconda3/envs/dbgpt_hub/lib/python3.10/site-packages/torch/nn/modules/module.py", line 2120, in load
module._load_from_state_dict(
File "/home/hp/miniconda3/envs/dbgpt_hub/lib/python3.10/site-packages/bitsandbytes/nn/modules.py", line 256, in _load_from_state_dict
self.weight, state_dict = bnb.nn.Params4bit.from_state_dict(
File "/home/hp/miniconda3/envs/dbgpt_hub/lib/python3.10/site-packages/bitsandbytes/nn/modules.py", line 158, in from_state_dict
data = state_dict.pop(prefix.rstrip('.'))
KeyError: 'base_model.model.model.layers.0.self_attn.q_proj.weight'
我用train_sft.sh微调CodeLlama-7b-Instruct-hf后predict出现以上错误，我发现adapter_model.bin中保存的是base_model.model.model.layers.0.self_attn.q_proj.lora_A.weight，和base_model.model.model.layers.4.self_attn.q_proj.lora_B.weight。