How are the pretrained weigts set to be fixed during training?

microsoft / LoRA

Code for loralib, an implementation of "LoRA: Low-Rank Adaptation of Large Language Models"

https://arxiv.org/abs/2106.09685

MIT License

10.43k stars 669 forks source link

Open HaiminZhang opened 2 months ago

HaiminZhang commented 2 months ago

Hi everyone, could anyone let me know how the pretrained weigts are set to be fixed during training?

HaiminZhang commented 2 months ago

I've figure it out.