search

HestiaSky / E4SRec

MIT License
35 stars 6 forks source link

ValueError: Target modules [gate_proj, down_proj, up_proj] not found in the base model. Please check the target modules and try again. #5

Closed jackaihfia2334 closed 5 months ago

jackaihfia2334 commented 6 months ago

I use huggyllama-7b as base model

Target modules [gate_proj, down_proj, up_proj] not found in the base model. Please check the target modules and try again. File "/usr/local/lib/python3.10/dist-packages/peft/tuners/tuners_utils.py", line 222, in inject_adapter raise ValueError( File "/usr/local/lib/python3.10/dist-packages/peft/tuners/tuners_utils.py", line 88, in init self.inject_adapter(self.model, adapter_name) File "/usr/local/lib/python3.10/dist-packages/peft/tuners/lora.py", line 274, in init super().init(model, config, adapter_name) File "/usr/local/lib/python3.10/dist-packages/peft/peft_model.py", line 111, in init self.base_model = PEFT_TYPE_TO_MODEL_MAPPING[peft_config.peft_type]( File "/usr/local/lib/python3.10/dist-packages/peft/peft_model.py", line 1658, in init super().init(model, peft_config, adapter_name) File "/usr/local/lib/python3.10/dist-packages/peft/mapping.py", line 106, in get_peft_model return MODEL_TYPE_TO_PEFT_MODEL_MAPPING[peft_config.task_type](model, peft_config, adapter_name=adapter_name) File "/data1/E4SRec/model.py", line 39, in init self.llama_model = get_peft_model(self.llama_model, peft_config) File "/data1/E4SRec/finetune.py", line 118, in train model = LLM4Rec( File "/usr/local/lib/python3.10/dist-packages/fire/core.py", line 691, in _CallAndUpdateTrace component = fn(*varargs, **kwargs) File "/usr/local/lib/python3.10/dist-packages/fire/core.py", line 475, in _Fire component, remaining_args = _CallAndUpdateTrace( File "/usr/local/lib/python3.10/dist-packages/fire/core.py", line 141, in Fire component_trace = _Fire(component, args, parsed_flag_args, context, name) File "/data1/E4SRec/finetune.py", line 245, in fire.Fire(train) File "/usr/lib/python3.10/runpy.py", line 86, in _run_code exec(code, run_globals) File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main (Current frame) return _run_code(code, main_globals, None, ValueError: Target modules [gate_proj, down_proj, up_proj] not found in the base model. Please check the target modules and try again.

HestiaSky commented 5 months ago

Please use garage-bAInd/Platypus2-xB instead of vanilla LLaMA2 since vanilla LLaMA2 models are not instruction-tuned.

KpiHang commented 3 months ago

@jackaihfia2334 想请教您一下，这个代码您怎么跑起来的，项目中提供的数据集似乎并不完善，我有些不懂是怎么处理的，比如这个：https://github.com/HestiaSky/E4SRec/issues/12

非常感谢，希望得到您的回复。