search

Project-MONAI / MONAI

AI Toolkit for Healthcare Imaging
https://monai.io/
Apache License 2.0
5.73k stars 1.05k forks source link

Exponential moving average (EMA) handler #2514

Closed sandylaker closed 3 years ago

sandylaker commented 3 years ago

Is your feature request related to a problem? Please describe. Exponential moving averaging is a very import trick for training CNNs like EfficienNet, MobileNet-V3, or self-supervised learning models like MoCo, BYOL etc. Specifically, we keep an exponential moving smoothed version of weights, and use it for validation and inference.

Describe the solution you'd like We can implement this function as an ignite handler.

Describe alternatives you've considered

Additional context

sandylaker commented 3 years ago

I can work on this feature and send a PR

Nic-Ma commented 3 years ago

Hi @sandylaker ,

Cool, welcome your contribution!

Thanks.

wyli commented 3 years ago

I'd encourage a pull request directly to pytorch-ignite, because it’s a generic feature, widely-applicable. But if ignite doesn't want to integrate it for any reason, monai can take it...

Cc @vfdev-5

vfdev-5 commented 3 years ago

@wyli thanks a lot for pinging! Yes, that's can be a feature for pytorch ignite.

sandylaker commented 3 years ago

@wyli Sure, I will make a PR to ignite first.