tianweiy / DMD2

(NeurIPS 2024 Oral 🔥) Improved Distribution Matching Distillation for Fast Image Synthesis
Other
512 stars 28 forks source link

Is this method suits for velocity prediction? #27

Closed icelighting closed 4 months ago

icelighting commented 4 months ago

i don't understand whether the score function (gradient direction) just suit for the u(t, xt)? should i need to transfer the velocity to the u(t, xt)? as the velocity is the minus of noise and xt,

image
tianweiy commented 4 months ago

yes, it also works for velocity prediction but you will need a set of modifications to adapt the epsilon prediction to v.

Some places are listed below (it might not be the full set of modifications needed)

https://github.com/tianweiy/DMD2/blob/0f8a481716539af7b2795740c9763a7d0d05b83b/main/sd_unified_model.py#L152C31-L152C48

https://github.com/tianweiy/DMD2/blob/0f8a481716539af7b2795740c9763a7d0d05b83b/main/sd_unified_model.py#L287

https://github.com/tianweiy/DMD2/blob/0f8a481716539af7b2795740c9763a7d0d05b83b/main/sd_guidance.py#L199C1-L200C1

https://github.com/tianweiy/DMD2/blob/0f8a481716539af7b2795740c9763a7d0d05b83b/main/sd_guidance.py#L231

https://github.com/tianweiy/DMD2/blob/0f8a481716539af7b2795740c9763a7d0d05b83b/main/sd_guidance.py#L295

icelighting commented 4 months ago

thanks for your reply, i