search

thuanz123 / realfill

Unofficial implementation of RealFill
MIT License
359 stars 27 forks source link

is it possible to train multiple pairs of reference images target image at once? #21

Closed hjhjhjhhjhjhhh closed 5 months ago

hjhjhjhhjhjhhh commented 5 months ago

Let's say i have 20 reference images, target image pairs, can i pass all of them as reference images? so that i only need to train the LoRA one time, and it is able to generate consistent images for all the target image. Sorry for asking unreasonable question.

thuanz123 commented 5 months ago

I guess you mean there are more than one scene in the 20 images, right ? If so, I dont think RealFill can perform well. However, you should try and see what interesting results come out

thuanz123 commented 5 months ago

You can try like one LoRA but multiples special tokens for all 20 images. Currently, like DreamBooth, RealFIll only use one special token "sks". However, you could try multiple special tokens for multiple scenes. For example, suppose there are 3 scenes in your 20 images, you could try to train one LoRA weight but with 3 special tokens like "sks", "xyz" and "t@t" corresponding to 3 scenes.

hjhjhjhhjhjhhh commented 5 months ago

That's an interesting idea! I'll give it a try. Grateful for the quick and high-quality response!

thuanz123 commented 5 months ago

Please keep me updated (if possible) because I'm also very curious about this too 😄