search

joelberkeley / spidr

Accelerated machine learning with dependent types
Apache License 2.0
73 stars 4 forks source link

Implement the *Ultimate Optimizer* #344

Closed joelberkeley closed 1 year ago

joelberkeley commented 1 year ago

This Nuerips 2022 awards paper introduces the Ultimate Optimizer, which allows us to use autodiff to find hyperparameters, which could dramatically simplify (and speed up?) ML workflows. Implement it.

Requires SGD first?

joelberkeley commented 1 year ago

Paper review suggested that while it was good, it wasn't as useful as it might have sounded. Let's just stick with sgd and adam for now. I'll leave this open as an interesting future goal