How about RAdam + LAMB + Lookahead?

mgrankin / over9000

Over9000 optimizer

Apache License 2.0

425 stars 57 forks source link

How about RAdam + LAMB + Lookahead? #18

Closed Whu-wxy closed 4 years ago

Whu-wxy commented 4 years ago

Do you have this experiment plan in the near future?

mgrankin commented 4 years ago

Feel free to try and share your findings.

stale[bot] commented 4 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.