Activate/deactivate softplus for MADGRAD & choosing beta softplus

lessw2020 / Ranger21

Ranger deep learning optimizer rewrite to use newest components

Apache License 2.0

323 stars 46 forks source link

Activate/deactivate softplus for MADGRAD & choosing beta softplus #23

Closed TheZothen closed 3 years ago

TheZothen commented 3 years ago

Hi,

I forgot to add a way to deactivate softplus for MADGRAD in the last PR. I also added an easy way to choose the beta parameter of the softplus transform.

lessw2020 commented 3 years ago

Excellent - this definitely adds even finer grained control re: softplus. Much appreciated!