comfyanonymous / ComfyUI

The most powerful and modular diffusion model GUI, api and backend with a graph/nodes interface.
https://www.comfy.org/
GNU General Public License v3.0
56.38k stars 5.97k forks source link

Generate batch of images inside same latent space(fixed latent noise / seed?) #3198

Closed brandostrong closed 7 months ago

brandostrong commented 7 months ago

In Auto1111, in the txt2image section you have the ability to generate images within the same latent space. For example, with the seed and prompt static, changing the controlnet driving image will lead to a small difference in the output image. This is the same for a small change in prompt.

In Comfyui however, making a batch of latents but only changing the controlnet per latent will produce wildly different images, even with the same seed.

Is there any existing node that can copy Auto1111's behavior in this? It greatly improves consistency on batch animations, because the sampler isn't sampling from a new latent space each latent.

ltdrdata commented 7 months ago

What you are talking about means applying noise generated with the same seed to all frames within a batch. From what I remember, there is a feature that allows you to set it to work in such a way when sampling using https://github.com/Kosinkadink/ComfyUI-AnimateDiff-Evolved.

NeedsMoar commented 7 months ago

The LatentBatchSeedBehavior node set to fixed will produce very similar images across a batch, the default is random I think. It should be a built-in node.

brandostrong commented 7 months ago

What you are talking about means applying noise generated with the same seed to all frames within a batch. From what I remember, there is a feature that allows you to set it to work in such a way when sampling using https://github.com/Kosinkadink/ComfyUI-AnimateDiff-Evolved.

Hi, this is pretty much the answer I'm looking for. It's not exactly like auto1111's, there's more flicker and randomness, but it's pretty close. Thank you.

Lalimec commented 4 months ago

Is there a way to keep the seed literally fixed? That node doesnt seem to work correctly.

image image

ltdrdata commented 4 months ago

Make sure you are not using stochastic sampler. ( _ancestral, _sde)

Lalimec commented 4 months ago

So its not possible to use a stochastic sampler and get the same seeds in a batch you say? I have a wf which outputs a person in different ages, so i need to keep the seed same through out the images, to reduce unwanted changes. It is obviously possible if I gave ksampler a "list" of prompts and set the seed to fixed. But generating in batches is a little more faster, so I wanted to utilize that.

image

ltdrdata commented 4 months ago

In a stochastic sampler, randomness is added intermittently during the sampling process. The seed only determines the initial noise of the latent, but it does not control the randomness of the sampling process.

Lalimec commented 4 months ago

Interesting, do you know any non-stochastic sampler that works with turbo models? I was using dpm++ sde with these ones, since it's a lengthy generaion i would like to keep the step count as low as its possible.