search

erfanzar / EasyDeL

Accelerate your training with this open-source library. Optimize performance with streamlined training and serving options with JAX. 🚀
https://easydel.readthedocs.io/en/latest/
Apache License 2.0
167 stars 19 forks source link

Will this project support lora? #79

Closed IvoryTower800 closed 5 months ago

IvoryTower800 commented 5 months ago

Thanks for giving me an easy way finetuning model on TPU. I tried finetune a full model using this project on kaggle's tpu and it works.

I'm wondering whether there is a schedule for supporting finetune a lora or qlora model? it's a more practical way for those only have access to limited computing resources like me...

erfanzar commented 5 months ago

yes in the upcoming future and after finishing DPO, SFT Trainers the Lora process will start in FJFormer (EasyDel Computation Backend) and ill Update BITs too

IvoryTower800 commented 5 months ago

Awesome! I think lots of people are looking forward to it. Thank you!