search

jettify / pytorch-optimizer

torch-optimizer -- collection of optimizers for Pytorch
Apache License 2.0
3.01k stars 295 forks source link

[Feature Request] SAM optimizer #346

Open bojesomo opened 3 years ago

bojesomo commented 3 years ago

It would be nice to have SAM optimizer

{SHARPNESS-AWARE MINIMIZATION FOR EFFICIENTLY IMPROVING GENERALIZATION} https://arxiv.org/pdf/2010.01412v3.pdf

InCogNiTo124 commented 3 years ago

I second this request!

The referense implementation is here

eighttails commented 2 years ago

Implementation for PyTorch is here. It's working very well. https://github.com/davda54/sam