Closed sarthakpati closed 2 months ago
The AdEMAMix optimizer has been shown to work well for large models [ref].
It would be great to make that available on GaNDLF.
N.A.
There are a few implementations for torch:
Is your feature request related to a problem? Please describe.
The AdEMAMix optimizer has been shown to work well for large models [ref].
Describe the solution you'd like
It would be great to make that available on GaNDLF.
Describe alternatives you've considered
N.A.
Additional context
There are a few implementations for torch: