search

tianweiy / DMD2

(NeurIPS 2024 Oral 🔥) Improved Distribution Matching Distillation for Fast Image Synthesis
Other
448 stars 25 forks source link

Why use the intermediate estimated x_0|t as the distribution matching input， but not the final image of generator? #45

Open vv12kant opened 1 month ago

vv12kant commented 1 month ago

As far as I understand，the input of ditribution mathing should be the final output of generator, but under the circumstance of multi-step generator, we use the intermediate estimated x_0|t as the input，why?

tianweiy commented 1 month ago

you are right. However, using the final output requires us to back propagate through the time (or step) which will adds more gpu memory consumption. It should be possible to do this with some system optimization but we didn't get chance to try because of this concern

vv12kant commented 4 weeks ago

Thanks. I think I understand something: the multi-step generator is like an LCM model. The difference is that LCM needs to align the final output x_0 of all steps on the ODE, while DMD needs to match the distribution to the pre-trained DM at all steps on the ODE.