Support for integrated SDXL base + refiner sampler

[MoonRide303]

Fooocus by lllyasviel introduced new type of sampler for SDXL, where both base and refiner model work together during single sampling pass - which allows refiner to use momentum from base, and improves quality of the final image (described with more details on Fooocus project page).

Fooocus looks like interesting project with nice optimizations, but it aims for simplicity - and hides most configuration options from the users. Implementation of this new sampler looks largely compatible with ComfyUI code, as it uses Comfy code. KSamplerWithRefiner itself (defined in samplers_advanced.py) looks pretty similar to KSampler defined in samplers.py. It would be nice to have it ported to ComfyUI, and available as integrated SDXL base + refiner sampler node, so people who don't mind playing with nodes could benefit from this optimization, too.

[hustille]

I did a quick and dirty port into my custom nodes. Seems to work, but on GPU it apparently needs --gpu-only --force-fp32, which is too much for my poor 8GB 1070, so I can only run it on CPU, which doesn't encourage extensive testing ...

With half precision it throws an error, no idea whether it really needs 32 bit or whether I did something wrong. Perhaps someone who knows what he's doing can take a look at it?

The node resides under sampling/; two sets of inputs for base and refiner, as you'd expect. No idea what VAE you are supposed to use for decoding. I included a sample workflow in my project.

[MoonRide303]

@hustille Fooocus itself using this sampler stays under 8 GB VRAM usage, so I am pretty sure it's doable.

[hustille]

Found a few bits I missed - it's working now.

[n00mkrad]

Found a few bits I missed - it's working now.

Yeah you returned a single object but Nodes need to return tuples :)

[hustille]

Don't sweat the small details ... At first I only copied the sampler class, but the sampler function the class uses had some more differences apart from adding a second model, so I copied that too (and missed the return value, which Fooocus handles differently).

Foocus also messes with the sizes in the clip. No idea what that is about, whether we need it, and whether it is effective the way I copied it.

[hustille]

I missed some more parts - since those patch basic ComfyUI behaviour, I split this node out into a new project, so those patches can easily be enabled or disabled on their own: ComfyUI_Fooocus_KSampler