search

clovaai / stargan-v2

StarGAN v2 - Official PyTorch Implementation (CVPR 2020)
Other
3.5k stars 659 forks source link

Moving average #60

Open stqc opened 4 years ago

stqc commented 4 years ago

Can you please explain how computing moving averages help ?

CharlesNord commented 4 years ago

I think this can help to stabilize the weight update. Since stochastic gradient descent could sometimes lead to the wrong optimization direction, making the training noisy. Averaging with the recent weights can alleviate this problem.

But I cannot find any paper to support this idea.

CharlesNord commented 4 years ago

I found the comment under this issue could be the answer:

https://github.com/clovaai/stargan-v2/issues/62