cloneofsimo / lora

Using Low-rank adaptation to quickly fine-tune diffusion models.
https://arxiv.org/abs/2106.09685
Apache License 2.0
7.08k stars 481 forks source link

Can LoRA be applied to different base models? #244

Open CJ0109 opened 1 year ago

CJ0109 commented 1 year ago

I've always been curious about how LoRA works. For example, if I trained my LoRA parameters on SD-1-4, can I apply it directly to the base model of SD-1-5? According to my previous knowledge, this should not work, because this is equivalent to changing the model parameters of SD-1-5, which will cause the inference to fail. But that doesn't seem to be the case. When I use Chilloutmix, different LoRA models such as JapaneseDoll, KoreaDoll can be combined into the base model with different weights, and it works quite well.