`transformers.Trainer` variant of `UnslothTrainer`

unslothai / unsloth

Finetune Llama 3.1, Mistral, Phi & Gemma LLMs 2-5x faster with 80% less memory

https://unsloth.ai

Apache License 2.0

15.21k stars 1.02k forks source link

`transformers.Trainer` variant of `UnslothTrainer` #655

Open lmg-anon opened 2 months ago

lmg-anon commented 2 months ago

This isn't a major issue, but I'm unable to use code for transformers.Trainer with UnslothTrainer without modifying Unsloth's code. Currently, UnslothTrainer inherits from SFTTrainer. It would be nice if there was another class like UnslothTrainer that inherited from transformers.Trainer instead.

danielhanchen commented 2 months ago

Oh good point! I was thinking of doing a UnslothWrap(Trainer) type wrapper