AttributeError: 'ModelConfig' object has no attribute 'lora_task_type'

[joshhu]

using the example script, shows the error AttributeError: 'ModelConfig' object has no attribute 'lora_task_type'

another error shows when I put the lora_task_type to command line args ValueError: Some specified arguments are not used by the HfArgumentParser: ['--lora_task_type', 'SEQ_CLS']

[younesbelkada]

hi @joshhu which version of TRL do you have? can you try on the latest TRL from pypi?

[joshhu]

hi @joshhu which version of TRL do you have? can you try on the latest TRL from pypi?

Thank you for replying. The version or TRL is 0.7.11. Also I tried to install the most recent version through pip install trl -U but got no luck.

[younesbelkada]

can you share the command you use for training?

[younesbelkada]

as well as the full traceback of the issue

[joshhu]

as well as the full traceback of the issue

Thank you for replying.

The command line is

(llm) (JoshDev)joshhu:trl/ (main✓) $ python examples/scripts/reward_modeling.py \                             [0:21:56]
    --model_name_or_path="gpt2"  \
    --output_dir="reward_modeling_anthropic_hh" \
    --per_device_train_batch_size=64 \
    --num_train_epochs=1 \
    --gradient_accumulation_steps=16 \
    --gradient_checkpointing=True \
    --learning_rate=1.41e-5 \
    --remove_unused_columns=False \
    --optim="adamw_torch" \
    --logging_steps=10 \
    --evaluation_strategy="steps" \
    --max_length=512  \
    --lora_task_type = "SEQ_CLS"

and the traceback is

Traceback (most recent call last):
  File "/home/joshhu/playground/trl/examples/scripts/reward_modeling.py", line 45, in <module>
    reward_config, model_config = parser.parse_args_into_dataclasses()
                                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/joshhu/miniconda3/envs/llm/lib/python3.11/site-packages/transformers/hf_argparser.py", line 347, in parse_args_into_dataclasses
    raise ValueError(f"Some specified arguments are not used by the HfArgumentParser: {remaining_args}")
ValueError: Some specified arguments are not used by the HfArgumentParser: ['--lora_task_type', '=', 'SEQ_CLS']

[mengdi-li]

Hi @joshhu, I got the same issue. It can be solved by installing TRL from the source code.

You need to first uninstall TRL of version 0.7.11: pip uninstall trl

and then install TRL from the souce code that was cloned from github: pip install -e path/to/local/trl

The version of the newly installed TRL should be 0.7.12.dev0 as of today

[joshhu]

Hi @joshhu, I got the same issue. It can be solved by installing TRL from the source code.

You need to first uninstall TRL of version 0.7.11: pip uninstall trl

and then install TRL from the souce code that was cloned from github: pip install -e path/to/local/trl

The version of the newly installed TRL should be 0.7.12.dev0 as of today

It now works, thank you.

[younesbelkada]

Thanks. ! closing the issue then