sd-webui-samplers-scheduler Seniorious for v1.6.x

Introduction

A samplers scheduler which can apply different sampler in diffrent generation steps,

I hope it will be helpful to achieve a balance between generation speed and image quality.

Our paper on arXiv: Sampler Scheduler for Diffusion Models

Requirement

SD WebUI Version >= 1.6.0

(For version <= 1.5.2, please use https://github.com/Carzit/sd-webui-samplers-scheduler)

How to use

This repository is a extension for sd webui. Just place it in the extension folder!😉

Choose the Sampler Seniorious or Seniorious Karras to enable the samplers scheduler.

*Seniorious uses nomal noise scheduler and Seniorious Karras uses the noise scheduler
recommended in Elucidating the Design Space of Diffusion-Based Generative Models (Karras et al., 2022)

This samplers scheduler provides 8 sampler units (Sampler 1-8).

You can choose what kind of sampler used in each unit(choose None to unable), and the inference steps for each unit.

The image generation process will follow the configurations of these 8 units in sequence.

Attention: The total steps should be equal to the sum of the steps in every unit!
Open the Check accordion and press the Check button to check total steps.
Total steps in Seniorious shows the sum of steps in your Sampler Scheduler Settings and Total steps required shows the steps you set in webui. They must be equal.

If you use Sampler Scheduler Seniorious, the Sampler Scheduler Config will be added to the png_info.

Available Samplers

14 kinds of mainstream samplers in k-diffusion are available:

• Euler
• Euler a
• Heun
• Heun++
• LMS
• DPM2
• DPM2 a
• DPM++ 2S a
• DPM++ SDE
• DPM++ 2M
• DPM++ 2M SDE
• DPM++ 3M SDE
• Restart
• LCM

You can also choose Skip to skip certain steps.

Heun++ is my test version.
Restart is a new sampler in SD WebUI since version 1.6, using the recommended hyperparameters recommended in Implements restart sampling in Restart Sampling for Improving Generative Processes.
LCM is a new sampler used in latent consistency model(LCM).

FID Result

I calculate the FID Score based on EDM.

config:

• seeds=0-49999
• network=https://nvlabs-fi-cdn.nvidia.com/edm/pretrained/edm-cifar10-32x32-cond-vp.pkl
• NFE=30

Discretely scheduling different samplers during the sampling process has proven to be effective at a practical level.

Examples

ODE Samplers

Shared Parameters:

• Prompt: best quality, masterpiece, 1girl, solo, standing, sky, portrait, looking down, floating light particles, sunshine, cloud, depth of field, field, wide shot
• Negative prompt: badhandv4, By bad artist -neg, EasyNegative, NegfeetV2, ng_deepnegative_v1_75t, verybadimagenegative_v1.3
• Steps: 30
• CFG scale: 7
• Seed: 2307198650
• Size: 512x768,
• Model: anything-v5-PrtRE

*Seniorious and Seniorious Karras use the same parameters in this example.

Put ODE and SDE Samplers Together

I recommend to use SDE in the early sampling steps and ODE in the later sampling steps to solve the inherent problems previously caused by using either singly.

About LCM Sampler

Like other implemented sampling algorithms, I added the LCM sampler as a sampler option to our Sampler Scheduler.
But obviously the iteration formula of LCM sampler $x{i+1} = D\theta(x{i}) + \sigma{i+1} \epsilon_i$ is different from those of traditional LDM samplers.
Ignoring its coupling with the LCM model and using it rashly as a plug-in sampler for LDM may cause some problems, such as blurring at the edge of the image.

Moreover, based on some of my experience, I recommend using LCM sampler at later sampling steps rather than at the beginning.
In figures above, with other parameters fixed, the left uses 12-step Euler a + 8-step LCM while the right uses 8-step LCM + 12-step Euler a.

More

The idea of this extension was inspired by Seniorious, a Carillon composed of different talismans.

Different talismans correspond in sequence to make Seniorious a powerful weapon, and so do the samplers in the samplers scheduler.

In the end, many thanks to Chtholly Nota Seniorious, the happiest girl in the world.