search

kohya-ss / sd-scripts

Apache License 2.0
5.33k stars 881 forks source link

Allow unknown schedule-free optimizers to continue to module loader #1799

Closed rockerBOO closed 4 days ago

rockerBOO commented 1 week ago

To support ProdigyPlusScheduleFree

pip install git+http://github.com/LoganBooker/prodigy-plus-schedule-free
--optimizer_type prodigyplus.ProdigyPlusScheduleFree
                    INFO     use prodigyplus.ProdigyPlusScheduleFree | {'weight_decay': 0.1}                                         train_util.py:4111
[Prodigy+ScheduleFree] Optimiser contains single param_group -- 'split_groups' has been disabled.

Related #1796

kohya-ss commented 6 days ago

Thank you for this! And I'm very sorry, but for the schedule-free optimizer, I received a new PR #1600, so I merged it into the sd3 branch (and refactored in #1605). Therefore, I plan to delete this branch in the future. Sorry for the confusion.

I think can make this fix to the sd3 branch myself, however I would be very grateful if you could make the same fix to the sd3 branch.

rockerBOO commented 4 days ago

No problem @kohya-ss, I'll make a new PR against the SD3 branch.