Open loboere opened 2 years ago
@loboere I'm not sure if you can have 2 init images. It explicitly says img2img in their scripts
Midjourney has a new remix feature that merges 2 images. But it'd be sweet if we could do something similar with SD.
Just a pure guess: In a text-to-image model like Stable Diffusion, the noisy latent image is de-noised while conditioned on text embeddings. So what Midjourney did here could be using the image embeddings of the two images to condition the de-noising steps.
using the image embeddings of the two images to condition the de-noising steps
Yeah, but stable diffusions releases was trained on conditioning by text only :( Here stable diffusion fine-tuned for "image variations" https://github.com/LambdaLabsML/lambda-diffusers#stable-diffusion-image-variations but need more training steps.
Maybe there is CLIP text embeddings <> image embeddings convertion existing...
Seed travel
I mean use 2 init images and Strength see the mix between two images