WARNING gradient_accumulation_steps is 3. accelerate does not support train_db.py:109

bmaltais / kohya_ss

Apache License 2.0

9.27k stars 1.2k forks source link

WARNING gradient_accumulation_steps is 3. accelerate does not support train_db.py:109 #2335

Open DarkViewAI opened 4 months ago

DarkViewAI commented 4 months ago

WARNING gradient_accumulation_steps is 3. accelerate does not support train_db.py:109 gradient_accumulation_steps when training multiple models (U-Net and Text Encoder), so something might be wrong WARNING gradient_accumulation_stepsが3に設定されています。accelerateは複数モデル（U train_db.py:112 -NetおよびText Encoder）の学習時にgradient_accumulation_stepsをサポートしていないため結果は未知数です

bmaltais commented 4 months ago

Can you share the .toml file for the training? look like the trainer is warning about radient_accumulation_steps being set to 3 and warning it will cause issues