Closed clayne closed 1 week ago
Both of these issues were due to a bug in schedulefree 1.2.7:
https://github.com/Nerogar/OneTrainer/issues/578 https://github.com/Nerogar/OneTrainer/issues/579 (although this one seems potentially less related)
and discussed here: https://discord.com/channels/1102003518203756564/1102013112124706827/1308522446575632388
I've checked the changelog for schedulefree 1.3 and the one major change is related to weight decay during learning rate warmup: https://github.com/facebookresearch/schedule_free?tab=readme-ov-file#releases
Other than that, everything else looks minor.
From:
requirements-global.txt:schedulefree==1.2.7 # schedule-free optimizers
To:
requirements-global.txt:schedulefree==1.3 # schedule-free optimizers
No response
Describe your use-case.
Both of these issues were due to a bug in schedulefree 1.2.7:
https://github.com/Nerogar/OneTrainer/issues/578 https://github.com/Nerogar/OneTrainer/issues/579 (although this one seems potentially less related)
and discussed here: https://discord.com/channels/1102003518203756564/1102013112124706827/1308522446575632388
I've checked the changelog for schedulefree 1.3 and the one major change is related to weight decay during learning rate warmup: https://github.com/facebookresearch/schedule_free?tab=readme-ov-file#releases
Other than that, everything else looks minor.
What would you like to see as a solution?
From:
To:
Have you considered alternatives? List them here.
No response