search

Carzit / sd-webui-samplers-scheduler

a samplers scheduler for stable diffusion webui
MIT License
37 stars 1 forks source link

Have you considered mixing LCM? #4

Open aifartist opened 1 year ago

aifartist commented 1 year ago

I've been heavily involved in using LCM which I can use to produce 512x512 4 step images in 45ms on my 4090. I'm a retired performance specialist from Microsoft.

A couple of weeks ago I started exploring mixing not only schedulers but models and seem to have stumbled across a way to get even better quality than standard LDM but with some of the speed of LCM. I call the tool I wrote 'SDHybrid'. In the image below the left image is standard LDM 20 steps. The right image is 6 step LCM(extremely fast). The middle image is where I take the partially denoised latent and pass it to LCM for the remainder of the generation. NOTE: The tool is based on my RTSD tool where you can move the sliders and get realtime feedback to explore ideas.

Note that the statement:

only a single sampler algorithm can be specified on all sampling steps in the generative process.

isn't strictly true. Some samplers combination do work as is as I've discovered through rapid realtime experimentation. The mixing of not just a scheduler but also LDM and LCM was the breakthrough for me. I've posted about this on reddit and a few discord groups.

Any interest? I have a great setup. Ubuntu 22.04 i9-13900K and a 4090. I can apply TensorRT or torch.compile for super speed. Right now I'm writing a diffusers pipeline to do this Hybrid scheduler approach I've been using.

Screenshot from 2023-11-27 14-12-31

Carzit commented 1 year ago

For some compatibility reasons, I update this repository as a stable version and explore some new features in the extension repository that only supports sd webui >= 1.6. The Sampler Scheduler version that supports the LCM sampler is available at https://github.com/Carzit/sd-webui-samplers-scheduler-for-v1.6.