search

microsoft / LoRA

Code for loralib, an implementation of "LoRA: Low-Rank Adaptation of Large Language Models"
https://arxiv.org/abs/2106.09685
MIT License
10.78k stars 688 forks source link

Is the output the entire model？ #139

Open licy02 opened 1 year ago

licy02 commented 1 year ago

We use lora , is the output the whole model

edwardjhu commented 1 year ago

Can you elaborate on what you meant by output?

licy02 commented 1 year ago

I had some issues with my previous statement. What I meant was the model weights after fine-tuning. In previous projects I've worked on, they would only store the LoRA weights after fine-tuning. However, in this project, it stored all weight parameters after fine-tuning, and I would like to inquire whether it was an issue with my fine-tuning or if it was originally intended to be this way.

zamalali commented 9 months ago

Hi @licy02,

In the Lora project, storing all weight parameters post-fine-tuning is intentional and differs from some previous projects. This is to provide the complete model state for flexibility and use-case coverage. Hope this helps!