How to finetune StableLM with LoRA?

[Itto1992]

StableLM looks GPTNeoX and has query_key_value parameters. I thought I could apply LoRA to StableLM by specifying target_modules='query_key_value', but I got the following error.

Traceback (most recent call last):
  File "/root/workspace/finetune.py", line 288, in <module>
    fire.Fire(train)
  File "/usr/local/lib/python3.10/dist-packages/fire/core.py", line 141, in Fire
    component_trace = _Fire(component, args, parsed_flag_args, context, name)
  File "/usr/local/lib/python3.10/dist-packages/fire/core.py", line 475, in _Fire
    component, remaining_args = _CallAndUpdateTrace(
  File "/usr/local/lib/python3.10/dist-packages/fire/core.py", line 691, in _CallAndUpdateTrace
    component = fn(*varargs, **kwargs)
  File "/root/workspace/finetune.py", line 189, in train
    model = get_peft_model(model, config)
  File "/usr/local/lib/python3.10/dist-packages/peft/mapping.py", line 112, in get_peft_model
    return MODEL_TYPE_TO_PEFT_MODEL_MAPPING[peft_config.task_type](model, peft_config)
  File "/usr/local/lib/python3.10/dist-packages/peft/peft_model.py", line 647, in __init__
    super().__init__(model, peft_config, adapter_name)
  File "/usr/local/lib/python3.10/dist-packages/peft/peft_model.py", line 91, in __init__
    self.base_model = PEFT_TYPE_TO_MODEL_MAPPING[peft_config.peft_type](
  File "/usr/local/lib/python3.10/dist-packages/peft/tuners/lora.py", line 132, in __init__
    self.add_adapter(adapter_name, self.peft_config[adapter_name])
  File "/usr/local/lib/python3.10/dist-packages/peft/tuners/lora.py", line 139, in add_adapter
    self._find_and_replace(adapter_name)
  File "/usr/local/lib/python3.10/dist-packages/peft/tuners/lora.py", line 225, in _find_and_replace
    raise ValueError(
ValueError: Target modules query_key_value not found in the base model. Please check the target modules and try again.

Is there any solution? Thank you in advance!

I confirmed the base model has `query_key_value` parameters in the attention modules.

``` In [5]: model.named_parameters Out[5]: ```

[Itto1992]

Sorry, I passed wrong arguments. query_key_value worked when the correct arguments are passed.