search

bmaltais / kohya_ss

Apache License 2.0
9.42k stars 1.22k forks source link

Lora AdamW8bit Learning Rate seems to have changed #2563

Closed rafstahelin closed 4 months ago

rafstahelin commented 4 months ago

for a long time, I usually trained for a photographic subject model using 1e-4 (te lr at 1e-5)-- cosine with warmups. Recently this setting seems to overfit much more quickly. I have heard rumours that this may have changed recently. Does anyone have a similar experience? Not sure what to make of this

zethfoxster commented 4 months ago

ive only noticed it got super slow...use to train the same set of data in 15 mins, now training takes almost an hour..same data same setting