Add a `nesterov` flag to `radam` optimizer

google-deepmind / optax

Optax is a gradient processing and optimization library for JAX.

https://optax.readthedocs.io

Apache License 2.0

1.56k stars 165 forks source link

Closed carlosgmartin closed 2 months ago

carlosgmartin commented 2 months ago

Currently, the following optimizers receive a nesterov flag:

Feature request: Add a nesterov flag to the following optimizers:

vroulet commented 2 months ago

Hello @carlosgmartin, Would you be willing to do a PR for this? For the record, this paper provides some insights on radam.

carlosgmartin commented 2 months ago

@vroulet #949