search

kohya-ss / sd-scripts

Apache License 2.0
5.31k stars 880 forks source link

what is the default value of d_coef if not specificed? (for prodigy optimizer) #1797

Closed Hugs288 closed 5 days ago

Hugs288 commented 6 days ago

sorry if this is a dumb question but i cant find an answer online

LoganBooker commented 5 days ago

@Hugs288 No worries -- the source code for the optimiser is available on GitHub: https://github.com/konstmish/prodigy

The specific lines that show all the default initialisation values are here: https://github.com/konstmish/prodigy/blob/009a7156779ae213c0bc13b4c2fe5a93fca00c15/prodigyopt/prodigy.py#L59-L65

For d_coef, the default value is 1.0.

Hugs288 commented 5 days ago

Thank you.