ValueError: Target modules [q_proj,v_proj] not found in the base model

[fiyero]

Hi all,

Sorry Im new to hugging face/llama/ Alpaca, i encounter this error when run the finetune.py

Loading checkpoint shards: 100%|██████████| 33/33 [00:10<00:00, 3.06it/s] The tokenizer class you load from this checkpoint is not the same type as the class this function is called from. It may result in unexpected tokenization. The tokenizer class you load from this checkpoint is 'LLaMATokenizer'. The class this function is called from is 'LlamaTokenizer'. Traceback (most recent call last): File "D:\Learning\fine_tune_model\alpaca-lora\finetune.py", line 276, in fire.Fire(train) File "C:\Users\Patrick\Anaconda3\envs\finetune2\lib\site-packages\fire\core.py", line 141, in Fire component_trace = _Fire(component, args, parsed_flag_args, context, name) File "C:\Users\Patrick\Anaconda3\envs\finetune2\lib\site-packages\fire\core.py", line 475, in _Fire component, remaining_args = _CallAndUpdateTrace( File "C:\Users\Patrick\Anaconda3\envs\finetune2\lib\site-packages\fire\core.py", line 691, in _CallAndUpdateTrace component = fn(*varargs, **kwargs) File "D:\Google\Learning\LLM_chatGPT\fine_tune_model\alpaca-lora\finetune.py", line 177, in train model = get_peft_model(model, config) File "C:\Users\Patrick\Anaconda3\envs\finetune2\lib\site-packages\peft\mapping.py", line 147, in get_peft_model return MODEL_TYPE_TO_PEFT_MODEL_MAPPING[peft_config.task_type](model, peft_config) File "C:\Users\Patrick\Anaconda3\envs\finetune2\lib\site-packages\peft\peft_model.py", line 560, in init super().init(model, peft_config) File "C:\Users\Patrick\Anaconda3\envs\finetune2\lib\site-packages\peft\peft_model.py", line 79, in init self.base_model = LoraModel(peft_config, model) File "C:\Users\Patrick\Anaconda3\envs\finetune2\lib\site-packages\peft\tuners\lora.py", line 133, in init self._find_and_replace() File "C:\Users\Patrick\Anaconda3\envs\finetune2\lib\site-packages\peft\tuners\lora.py", line 199, in _find_and_replace raise ValueError( ValueError: Target modules [q_proj,v_proj] not found in the base model. Please check the target modules and try again.

It should be some stupid mistakes for newbie like me :(

Thanks!

Patrick

[AngainorDev]

Looks like you didn't feed a llama model as base_model.

What was the full command you used?

[pandanakami]

Hello.

From the description of the file path, are you in a Windows environment? I also got the same error in Windows environment, and I avoided this error by taking the following actions.

Add the following to the beginning of the train() function in finetune.py

def train(
   ...
):
    if isinstance(lora_target_modules, str):
        lora_target_modules = lora_target_modules.replace(
            "[", "").replace("]", "").split(","))

In my environment, the startup argument '[q_proj,v_proj]' was interpreted as a string, not an array, so I made it an array of 'q_proj' and 'v_proj' in the action.

Please try it.

[ndvbd]

Thanks @pandanakami , I think you have one excessive closing parentheses.

[Log4J-Ops]

I'm having the same problem, and I've tried the method above, but I still get the same error

[whatcheers]

I get this when Im trying to train a LORA in a windows 11 environment

https://github.com/oobabooga/text-generation-webui/issues/3655

[NotTheStallion]

I tried with a different model and it works fine `from peft import LoraConfig, TaskType

peft_config = LoraConfig(task_type=TaskType.SEQ_2_SEQ_LM, inference_mode=False, r=8, lora_alpha=32, lora_dropout=0.1)

from transformers import AutoModelForSeq2SeqLM

model = AutoModelForSeq2SeqLM.from_pretrained("bigscience/mt0-large")
from peft import get_peft_model

model = get_peft_model(model, peft_config)
model.print_trainable_parameters()` 

But when i try to do the same thing for Llama3-70B-Instruct is gives me the same error as you. (I am on Linux and i tried with and without a python environment and it doesn't change anything.)