Could you please upgrade the trl library to the latest version?

unslothai / unsloth

Finetune Llama 3.1, Mistral, Phi & Gemma LLMs 2-5x faster with 80% less memory

https://unsloth.ai

Apache License 2.0

15.81k stars 1.07k forks source link

Could you please upgrade the trl library to the latest version? #890

Open ArcherShirou opened 1 month ago

ArcherShirou commented 1 month ago

I am using LLama-Factory to train long text DPO, but enabling unsloth is not supported with the latest version of the trl library. The newest trl update includes many useful new alignment algorithms. Additionally, on a single-machine setup with 8 A100 GPUs, unsloth encounters an error stating 'does not support multi-GPU setups'. Could you please let me know if it’s possible to support the latest trl library and multi-GPU training? Thanks

danielhanchen commented 1 month ago

Will support the latest TRL in the next release hopefully! We're currently in a beta testing phase for multi GPU for some of our community members currently

danielhanchen commented 1 month ago

Upgraded Unsloth to make it work with all new TRL versions, Python 3.12 and Torch 2.4/2.5!

ArcherShirou commented 1 month ago

It worked! Thanks for the update. It's really a great piece of work.