Hi,
When I train with dreambooth my sample images get distorted after ~8000 steps even though I don't get any errors or high loss rate (~0.115). I tried training with xformers 0.16 and latest version of xformers with pytorch 2.0. I had the same issue with both.
1200 steps:
2400 steps:
8640 steps:
Hi, When I train with dreambooth my sample images get distorted after ~8000 steps even though I don't get any errors or high loss rate (~0.115). I tried training with xformers 0.16 and latest version of xformers with pytorch 2.0. I had the same issue with both. 1200 steps:
2400 steps:
8640 steps:
Constant with warmup (%10) LR: 0.0000015 Noise offset 0.1 Xformers & memory efficient attention Flip augmentation fp16 AdamW8Bits 6 epoch 12 steps per file