Efficient sampling via DPM-Solver

[Birch-san]

Hi 👋,

https://github.com/LuChengTHU/dpm-solver just got released (with pytorch code!) and demonstrates how to sample in around 10 steps.
https://arxiv.org/abs/2206.00927

not sure how this compares to the sampling efficiencies that were unlocked with the Elucidated approach. presumably they can't be combined? but 10 samples is very few; perhaps this would outperform Elucidated?

@article{lu2022dpm,
  title={DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps},
  author={Lu, Cheng and Zhou, Yuhao and Bao, Fan and Chen, Jianfei and Li, Chongxuan and Zhu, Jun},
  journal={arXiv preprint arXiv:2206.00927},
  year={2022}
}

Thanks as always!

[Birch-san]

Okay it's been pointed out that this is already a to-do in the readme 😛

[lucidrains]

@Birch-san haha yea, it is on my radar

has anyone in the community / discord tried it yet? wondering how well it works

[Birch-san]

if I'm understanding correctly, then yeah @marunine experimented with sampling approaches, including dpm-solver.
hopefully m9 can explain — I don't wanna misrepresent the conclusions.

[lucidrains]

if @marunine vouches for the approach (or Katherine for that matter), I will definitely add it

[LuChengTHU]

Hi, I'm the first author of DPM-Solver and DPM-Solver++. Our work can greatly accelerate the sampling of stable-diffusion (in only 10-20 steps); here is our newest repo: https://github.com/LuChengTHU/dpm-solver

If you want to test it in Imagen, I'm happy to help.

P.S. Katherine's k-diffusion has supported our newest DPM-Solver++, and it is also supported further by stable-diffusion-webui. In addition, my recent PR to diffusers has been merged into the main branch of diffusers, and it works very well for stable-diffusion.

[bonlime]

@LuChengTHU hi, i didn't know where to ping you, so decided to do it here. I really like the quality obtained by the new sampler, but got several questions.

1. automatic111 repo also implements DPM++ (+ Karras scheduling). Do I understand correctly that in order to implement the Karras scheduling with 🤗 diffusers I only need to rewrite set_timesteps function?
2. Previously I've used DDIM sampler which allows inversion of image to original noise. Does your multi-step (DPM++) support inverting images? I guess it should as it's a deterministic sampler but I'm not sure how would the usage of previous step affect the inversion quality. For the DDIM I've used equation from original paper (which is exactly the same as for generation process). What would be the equation for DPM++(2M)?

[santisy]

@lucidrains Any plan to add the DPM solver in the near future? This project is awesome but it seems that it lacks some quick sampler method.