search

Linaqruf / kohya-trainer

Adapted from https://note.com/kohya_ss/n/nbf7ce8d80f29 for easier cloning
Apache License 2.0
1.82k stars 296 forks source link

RuntimeError: NaN is detected #298

Open oblivisheee opened 10 months ago

oblivisheee commented 10 months ago

Hello, im trying to train lora xl model, but, when i load dataset, and process latent buckets, it raise error, that NaN is detected. How i can fix it?

SilentCat096 commented 9 months ago

I met the same problem, setting Bucketing and Latents Caching - mixed_precision to no solved the problem. But now I am running into problem when the training starts.