Runtimeerror when dataset is too large

kohya-ss / sd-scripts

Apache License 2.0

5.31k stars 880 forks source link

Runtimeerror when dataset is too large #1790

Open JEFFSEVENTHSENSE opened 1 week ago

JEFFSEVENTHSENSE commented 1 week ago

RuntimeError: unable to mmap 4096 bytes from file : Cannot allocate memory (12) Is there a fix to this? I tried making num_workers =0 the training can go on but it is very slow.

JEFFSEVENTHSENSE commented 1 week ago

fixed by increasing vmax.map but now facing issue as each training step takes >45sec for 52k image dataset as compared to >1.3 sec for 100 image dataset

kohya-ss commented 6 days ago

Could you please share the stack trace of the error?