search

kohya-ss / sd-scripts

Apache License 2.0
4.92k stars 823 forks source link

Implement exponential moving averages (EMA) #1321

Open ddpasa opened 4 months ago

ddpasa commented 4 months ago

using EMA during training often leads to better results, and is a standard technique these days in training models.

ThereforeGames commented 4 months ago

--enable_ema

ddpasa commented 4 months ago

--enable_ema

I can't find this option in the current codebase. Was it taken out?

ThereforeGames commented 4 months ago

My apologies, I've been using the option for so long that I forgot it wasn't a standard part of the codebase. 😅

It's from this PR:

https://github.com/kohya-ss/sd-scripts/pull/893

There is also a newer (better?) implementation here, but that one appears to have been canceled.