Can you support DoRA? - Githubissues

NVIDIA / NeMo

A scalable generative AI framework built for researchers and developers working on Large Language Models, Multimodal, and Speech AI (Automatic Speech Recognition and Text-to-Speech)

https://docs.nvidia.com/nemo-framework/user-guide/latest/overview.html

Apache License 2.0

10.79k stars 2.26k forks source link

Can you support DoRA? #9520

Open SJmdy opened 6 days ago

SJmdy commented 6 days ago

Hi Team, Do you have any plans to support DoRA?

Paper: https://arxiv.org/abs/2402.09353 peft: https://github.com/huggingface/peft/pull/1474

Thank you