search

RedAIGC / StoryMaker

StoryMaker: Towards consistent characters in text-to-image generation
576 stars 50 forks source link

Is LoRA applied to image prompt embed? #27

Open chrisway613 opened 1 month ago

chrisway613 commented 1 month ago

I saw the code implementation is conflict with the formula (8) in paper, which said the LoRA layers are also applied to image prompt embed: https://github.com/RedAIGC/StoryMaker/blob/8cb2b12ed1c689cb7827151ace748abd5c8b90a0/ip_adapter/attention_processor_faceid.py#L184

So, is a slip of pen in paper?

RedAIGC commented 1 month ago

I saw the code implementation is conflict with the formula (8) in paper, which said the LoRA layers are also applied to image prompt embed:

https://github.com/RedAIGC/StoryMaker/blob/8cb2b12ed1c689cb7827151ace748abd5c8b90a0/ip_adapter/attention_processor_faceid.py#L184

So, is a slip of pen in paper?

Thanks for the correction. The formula in the paper is indeed wrong. Lora only works on text embed. We will correct it in the future.

chrisway613 commented 1 month ago

Thanks for your reply~! (btw, and I also waiting for the training code.)