search

pytorch / contrib

Implementations of ideas from recent papers
391 stars 42 forks source link

[question] SWA BatchNorm momentum reset #29

Open vadimkantorov opened 4 years ago

vadimkantorov commented 4 years ago

Could you please comment on the particular way of decaying the BatchNorm momentum parameter for every mini-batch during BatchNorm parameters update
https://github.com/pytorch/contrib/blob/master/torchcontrib/optim/swa.py#L305 ?

(As far as I understand, BatchNorm momentum is usually constant)

Thanks!