chapter4: compared standard SGD with Adam Optimizer

fastai / fastbook

The fastai book, published as Jupyter Notebooks

Other

21.41k stars 8.3k forks source link

chapter4: compared standard SGD with Adam Optimizer #642

Closed slatteryjim closed 5 months ago

slatteryjim commented 5 months ago

Made a little graph to compare plain SGD (lr=2e-5) with plain Adam optimizer, on a very simple quaddratic function with 3 parameters. It's like a "guess the 3 numbers I'm thinking of" game.

Adam was a good improvement, but I wonder if there isn't some way to do much better.

review-notebook-app[bot] commented 5 months ago

Check out this pull request on

See visual diffs & provide feedback on Jupyter Notebooks.

Powered by ReviewNB

slatteryjim commented 5 months ago

Oops, wrong destination branch; intended this for my forked repo master branch.