search

baichuan-inc / Baichuan2

A series of large language models developed by Baichuan Intelligent Technology
https://huggingface.co/baichuan-inc
Apache License 2.0
4.08k stars 293 forks source link

ValueError: Target module NormHead() is not supported. #340

Open AlexJJJChen opened 9 months ago

AlexJJJChen commented 9 months ago

Traceback (most recent call last): File "/home/jianc/benchmark/finetune_s2s_model.py", line 232, in model = get_peft_model(base_model, lora_config) File "/home/jianc/anaconda3/envs/benchmark-llm/lib/python3.10/site-packages/peft/mapping.py", line 133, in get_peft_model return MODEL_TYPE_TO_PEFT_MODEL_MAPPING[peft_config.task_type](model, peft_config, adapter_name=adapter_name) File "/home/jianc/anaconda3/envs/benchmark-llm/lib/python3.10/site-packages/peft/peft_model.py", line 1043, in init super().init(model, peft_config, adapter_name) File "/home/jianc/anaconda3/envs/benchmark-llm/lib/python3.10/site-packages/peft/peft_model.py", line 125, in init self.base_model = cls(model, {adapter_name: peft_config}, adapter_name) File "/home/jianc/anaconda3/envs/benchmark-llm/lib/python3.10/site-packages/peft/tuners/lora/model.py", line 111, in init super().init(model, config, adapter_name) File "/home/jianc/anaconda3/envs/benchmark-llm/lib/python3.10/site-packages/peft/tuners/tuners_utils.py", line 87, in init self.inject_adapter(self.model, adapter_name) File "/home/jianc/anaconda3/envs/benchmark-llm/lib/python3.10/site-packages/peft/tuners/tuners_utils.py", line 244, in inject_adapter self._create_and_replace(peft_config, adapter_name, target, target_name, parent, optional_kwargs) File "/home/jianc/anaconda3/envs/benchmark-llm/lib/python3.10/site-packages/peft/tuners/lora/model.py", line 181, in _create_and_replace new_module = self._create_new_module(lora_config, adapter_name, target, kwargs) File "/home/jianc/anaconda3/envs/benchmark-llm/lib/python3.10/site-packages/peft/tuners/lora/model.py", line 334, in _create_new_module raise ValueError( ValueError: Target module NormHead() is not supported. Currently, only the following modules are supported: torch.nn.Linear, torch.nn.Embedding, torch.nn.Conv2d, transformers.pytorch_utils.Conv1D.