crowsonkb / k-diffusion

Karras et al. (2022) diffusion models for PyTorch
MIT License
2.26k stars 372 forks source link

Do not merge! Credits for DEIS #39

Open qsh-zh opened 1 year ago

qsh-zh commented 1 year ago

Hi Katherine,

Thanks for your awesome work. I open the issue to ask for credits of DEIS sampler.

  1. We are actually the first public work that proposes the usage of the exponential integrator method for solving diffusion model ODE, several months easier than concurrent works DPM-solver and Tero's Elucidate work.

Concurrent DPM-Solver~(second order and third order) and Heun method in Elucidate work is in fact exponential RK method with specific underlying RK solvers,

Despite different motivations, equation 18 is the key for DPM-Solver and Heun in Elucidate work. And two changes of variables $t <-> \rho$ and $x <->y$ are equivalent to "rescaling" in Elucidate work. And we can use more RK solvers for eq-18. Empirical comparisons among several RK solvers are also included in DEIS.

image

  1. Besides, applying linear multistep with non-uniform step-size and lower order warming started in the transformed ODE is first proposed in DEIS. see $\rho AB$-DEIS.

I know it is a bot area, but it is frustrating to be "scooped" and lose credits for a young researcher. Can you consider some modifications and have some credits for DEIS work?

Thanks, qsh