SFTConfig type wrong

[the-utkarshjain]

This issue is in context with run_sft.py. Might occur with other tasks too.

The SFTTrainer class expects SFTConfig to be an instance of trl.SFTConfig. However, transformers.TrainingArguments is being passed. Due to this, the code usually throws an error when private variable of trl.SFTConfig like dataset_num_proc, etc are not passed. The bigger issue is that the code works with transformers.TrainingArguments, so you can't even pass those variables through the YAML file, or else the code throws an error saying there are extra arguments which aren't used by the program.

[the-utkarshjain]

The solution is simple. Go to src/configs and add these lines:

1. import trl (at the top of the file)
2. Change class SFTConfig(transformers.TrainingArguments) to class SFTConfig(trl.SFTConfig)

Don't forget to install the handbook with python -m pip install -e . (note the -e option or else your changes won't be read).

[tanliboy]

Thank you, @the-utkarshjain ! It took me a while to figure out TRL has switched and doesn't support the subclass of TrainingArguments as its parameter right now.

[tanliboy]

We also need to fix the DPOConfig that has the same problem.

[snoels]

A quick fix is: pip install trl==0.8.2