search

unslothai / unsloth

Finetune Llama 3.2, Mistral, Phi, Qwen 2.5 & Gemma LLMs 2-5x faster with 80% less memory
https://unsloth.ai
Apache License 2.0
18.37k stars 1.28k forks source link

fix/sft-trainer #1276

Closed Erland366 closed 1 week ago

Erland366 commented 1 week ago

Since trl>=0.13.0.dev0, we need to move ['max_seq_length', 'dataset_num_proc', 'packing', 'dataset_text_field'] to SFTConfig .

Hence, we need to implement patch into SFTTrainer for backward compatibility. Here's an example of using old API on newest trl. Note that I give warning if it possible to user to move to SFTConfig instead.

image

And here's using the new API :

image

Erland366 commented 1 week ago

image Old API working in Colab using Old version

image

Old API working in Colab using new version

Erland366 commented 1 week ago

image

SFTTrainer old API

image

SFTTrainer new API

Erland366 commented 1 week ago

image

DPOTrainer old API

image

DPOTrainer new API

Erland366 commented 1 week ago

image

ORPO is also working

danielhanchen commented 1 week ago

@Erland366 I added an auto import so it patches all TRL modules - if it works, we can merge everything in :) Thanks for helping again! And great work!