search

tloen / alpaca-lora

Instruct-tune LLaMA on consumer hardware
Apache License 2.0
18.64k stars 2.22k forks source link

Anyone tried Dynamic LoRA (DyLoRA) to speedup #142

Open doxav opened 1 year ago

doxav commented 1 year ago

I am currently training Alpaca-LoRA as it is, but already wondering if someone tried to speed this up with DyLORA ?

Paper: https://neurips2022-enlsp.github.io/papers/paper_37.pdf Code to replace LoRA by DyLoRA: https://github.com/huawei-noah/KD-NLP/tree/main/DyLoRA

gururise commented 1 year ago

Interested in hearing if anyone had success with this..