search

vladmandic / automatic

SD.Next: Advanced Implementation of Stable Diffusion and other Diffusion-based generative image models
https://github.com/vladmandic/automatic
GNU Affero General Public License v3.0
5.7k stars 423 forks source link

[Feature]: New sampler: "Restart" #1537

Open Centurion-Rome opened 1 year ago

Centurion-Rome commented 1 year ago

Feature description

Saw on Reddit: https://www.reddit.com/r/StableDiffusion/comments/14n6vt7/new_sampler_restart/ and sounds interesting:

Generative processes that involve solving differential equations, such as diffusion models, frequently necessitate balancing speed and quality. ODE-based samplers are fast but plateau in performance while SDE-based samplers deliver higher sample quality at the cost of increased sampling time. We attribute this difference to sampling errors: ODE-samplers involve smaller discretization errors while stochasticity in SDE contracts accumulated errors. Based on these findings, we propose a novel sampling algorithm called Restart in order to better balance discretization errors and contraction.

Empirically, Restart sampler surpasses previous diffusion SDE and ODE samplers in both speed and accuracy. Restart not only outperforms the previous best SDE results, but also accelerates the sampling speed by 10-fold / 2-fold on CIFAR-10 / ImageNet.

In addition, it attains significantly better sample quality than ODE samplers within comparable sampling times. Moreover, Restart better balances text-image alignment/visual quality versus diversity than previous samplers in the large-scale text-to-image Stable Diffusion model pre-trained on LAION .

b5zhupg5

Results on Stable Diffusion v1.5: z2a639r4

More details

Paper: https://arxiv.org/abs/2306.14878 Github: https://github.com/newbeeer/diffusion_restart_sampling

Sampler implementation code: https://github.com/Newbeeer/diffusion_restart_sampling/blob/b1e0022406e619cafb118a69b70a90484fb84c94/diffuser/diffusers/pipelines/stable_diffusion/pipeline_stable_diffusion.py#L676-L817

Version Platform Description

No response

drax-xard commented 1 year ago

Yeah I wasn't sure where to post it so I made a discussion thread, probably should've started with a feature request duh.

vladmandic commented 1 year ago

Not too late 😁

drax-xard commented 1 year ago

Unofficial ComfyUI implementation: https://github.com/ssitu/ComfyUI_restart_sampling

lendrick commented 1 year ago

Now if only ComfyUI's samplers and A1111's samplers didn't take completely different arguments.

drax-xard commented 1 year ago

Oh, I've no idea how they implement stuff, haven't tried ComfyUI yet, I just thought it could be useful.

LankyPoet commented 1 year ago

Is this coming to SD.NEXT? This is one of the more unique samplers.

gohan2091 commented 11 months ago

How do I get this working in SD.Next?

vladmandic commented 11 months ago

How do I get this working in SD.Next?

is it marked as COMPLETE? no? then what are you asking for?

vladmandic commented 8 months ago

anyone interested in this, note that author still has this as status:

TODO: merge into the diffuser repo.

i cannot proceed as much as i'd like as this work is done in a separate/incompatible version of diffusers. if you're interested, create feature request on the authors git repo to actually merge his changes so others can actually use that sampler.

LankyPoet commented 8 months ago

Thank you. I am still interested in this so I opened an issue here: https://github.com/Newbeeer/diffusion_restart_sampling/issues/5

I encourage others to add their support so the author sees it.