LaurentMazare / diffusers-rs

An implementation of the diffusers api in Rust
Apache License 2.0
539 stars 55 forks source link

Optimize scheduler defaults #39

Closed mspronesti closed 1 year ago

mspronesti commented 1 year ago

This PR aims at optimizing the scheduler defaults for stable diffusion pipelines.

Today I noticed by accident that the defaults values for beta_start, beta_end and beta_schedule that HF uses in all the schedulers that I implemented in the past 3 weeks are not optimal for stable diffusion. This is why some of the images generated were "not so good". For instance, as somebody was pointing out in my last PR, Euler Ancestral Discrete was not producing very good looking images (coherently with the results yielded by the Python version from huggingface). Here's a comparison of the "correct" defaults vs the current ones, using 21 inference steps and 1573789502 as seed

image image

Here's the same comparison for Euler Discrete using the same seed and 15 inference steps.

sd_final.

sd_final

Other produced images using Euler Ancestral Discrete:

image image

The same applies to DDPM and DPM Multistep. DDIM and Heun Discrete already had these hyperparameters "appropriately" defaulted.

Lastly, I did some minor tweaks in a separate commit, just in case you prefer to keep the commit history cleaner merging them without squashing (assuming you find these contributions valuable :) )

LaurentMazare commented 1 year ago

Thanks, the updated images look indeed a lot better.