Is the EMA implementation code in this repository wrong?

dome272 / Diffusion-Models-pytorch

Pytorch implementation of Diffusion Models (https://arxiv.org/pdf/2006.11239.pdf)

Apache License 2.0

1.09k stars 252 forks source link

Is the EMA implementation code in this repository wrong? #38

Open prowontheus opened 10 months ago

prowontheus commented 10 months ago

in the update_average function of EMA class in modules.py，the update step is: old self.beta + (1 - self.beta) new should it be "new self.beta + (1 - self.beta) old"?

draym28 commented 2 months ago

actually, the code is correct. the formula of EMA is (suppose the weight beta=0.9): new = beta old + (1-beta) current the new average should be "a lot of old parts + a few new parts".