search

Sllambias / yucca

Apache License 2.0
17 stars 2 forks source link

add support for schedule-free optimizer #174

Open Sllambias opened 6 months ago

Sllambias commented 6 months ago

https://github.com/facebookresearch/schedule_free/blob/main/schedulefree/adamw_schedulefree.py

Sllambias commented 6 months ago

https://github.com/Lightning-AI/pytorch-lightning/discussions/19759

asbjrnmunk commented 6 months ago

Also see caveats, regarding choice of hyperparams, batch norm etc: https://github.com/facebookresearch/schedule_free?tab=readme-ov-file#caveats