search

kohya-ss / sd-scripts

Apache License 2.0
5.31k stars 880 forks source link

unexpected keyword argument 'num_decay_steps' when using cosine or linear scheduler (sd3 branch) #1602

Closed Enyakk closed 2 months ago

Enyakk commented 2 months ago

This is in reference to: bmaltais/kohya_ss#2812

using the latest sd3 branch and trying to train a FLUX1.dev LoRA using AdamW and a cosine or linear scheduler produces the error. See the attached config for complete details.

I believe starting from this change there has been a problem introduced with the cosine and linear schedulers. This is true at least for FLUX dev1 LoRA training using torch 2.4.0: https://github.com/kohya-ss/sd-scripts/pull/1393

This is the config TOML that produces the error for me: config_lora.zip

kohya-ss commented 2 months ago

Thank you for reporting! I've fixed.