search

IshmaelBelghazi / ALI

Adversarially Learned Inference
MIT License
311 stars 80 forks source link

batch norm updates #13

Open edgarriba opened 7 years ago

edgarriba commented 7 years ago

Hi, any hint about how does batch norm update and why you do the following?

bn_updates = [(p, m * 0.05 + p * 0.95) for p, m in pop_updates]

Just trying to mimic the same behavior in Pytorch but not sure what are p and m and when this update happens during the execution.

IshmaelBelghazi commented 7 years ago

This line estimate the population statistics during training using a moving average. p and m stand for the parameters before and after the update respectively.