search

crowsonkb / k-diffusion

Karras et al. (2022) diffusion models for PyTorch
MIT License
2.26k stars 372 forks source link

Add the newest DPM-Solver #40

Closed LuChengTHU closed 1 year ago

LuChengTHU commented 1 year ago

Thanks for your amazing work and interest in DPM-Solver! We have updated DPM-Solver v2.0, which supports four types of diffusion models: noise prediction model, data prediction model, v-prediction model, and score function.

Moreover, we supported both single-step and multi-step versions and the corresponding algorithms for the exponential integrators for both the noise prediction model and the data prediction model.

I'm glad to help if you want to further support our DPM-Solver in this repo :)

crowsonkb commented 1 year ago

Huh... DPM-Solver++(2S) has nice properties of not breaking at low NFE but I feel it's muddier/a little worse at medium NFE, is there some sort of compromise?

Thank you, Katherine Crowson

On Wed, Nov 2, 2022 at 2:36 AM Cheng Lu @.***> wrote:

Thanks for your amazing work and interest in DPM-Solver! We have updated DPM-Solver v2.0, which supports four types of diffusion models: noise prediction model, data prediction model, v-prediction model, and score function.

Moreover, we supported both single-step and multi-step versions and the corresponding algorithms for the exponential integrators for both the noise prediction model and the data prediction model.

I'm glad to help if you want to further support our DPM-Solver in this repo :)

— Reply to this email directly, view it on GitHub https://github.com/crowsonkb/k-diffusion/issues/40, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABDQ67QI4BZGU4OQLETWHL3WGIYYFANCNFSM6AAAAAARU37KQM . You are receiving this because you are subscribed to this thread.Message ID: @.***>

LuChengTHU commented 1 year ago

Hi @crowsonkb

I suggest using the multistep version DPM-Solver++ for guided sampling. e.g., All my experiments for accelerating stable-diffusion use the DPM-Solver++(2M). Please check this page: https://github.com/LuChengTHU/dpm-solver/tree/main/example_v2/stable-diffusion

I also implement a third-order multistep version in https://github.com/LuChengTHU/dpm-solver and https://github.com/huggingface/diffusers/pull/1132 . The third-order solver can further accelerate the results for medium and large NFEs.

If you want to support these algorithms, I'm happy to help :)

crowsonkb commented 1 year ago

I added 2S and 2M! :)

I tried implementing the 3M variant but it seemed worse/images looked kind of deep fried?

LuChengTHU commented 1 year ago

Hi @crowsonkb

My recent work https://arxiv.org/abs/2211.01095 shows that for guided sampling with large guidance scales, high-order solvers all fail to converge in a small number of steps. So in my experience, I think 3M is preferred for unconditional sampling but is unsuitable for conditional (guided) sampling.

LuChengTHU commented 1 year ago

I added 2S and 2M! :)

I tried implementing the 3M variant but it seemed worse/images looked kind of deep fried?

So quick and nice commit! Thank you for the support! I will close the issue.

Timothy2046 commented 1 year ago

I used the latest version of A1111 and K-diffusion, but the WEBUI still no DPM Solver and DPM Solver++ in the Sampling method list, why? what should i do to use DPM Solver