search

deepfakes / faceswap-playground

User dedicated repo for the faceswap project
306 stars 194 forks source link

CRITICAL An unexpected crash has occurred - Training #283

Closed SCB16 closed 5 years ago

SCB16 commented 5 years ago

Hello,

I am currently trying to carry out a face swap using the step by step guide provided on GitHub using the GUI. The extract process seems to work perfectly for both image and videos but when I then try to train using the extracted images I get a Critical Error and it crashes. I have tried reading the crash log but it doesn't make much sense to me.

I have attached the file and hopefully someone can figure it out for me. I have already tried git pull --all and pip install --upgrade -r requirements.txt and although it did update one thing I still get a Critical Error.

crash_report.2019.05.07.201128226255.log

Thanks in advanced

torzdf commented 5 years ago

OOM = Out of Memory. 3GB card is cutting it fine, especially on windows.

A few options:

1) Lower your batchsize 2) Use VRAM saving options (Memory Saving Gradients and/or ping-pong) 3) Use lowmem version of model or the lightweight model.

You definitely won't be able to train realface on that card.

SCB16 commented 5 years ago

OOM = Out of Memory. 3GB card is cutting it fine, especially on windows.

A few options:

  1. Lower your batchsize
  2. Use VRAM saving options (Memory Saving Gradients and/or ping-pong)
  3. Use lowmem version of model or the lightweight model.

You definitely won't be able to train realface on that card.

Hi torzdf,

Thanks for the reply, I had a feeling it was due to a resource issue but could't quite figure out what OOM was (obvious now you've said it but it was late). I will give a few of these options a go and see if I can get a result and might even try on my Linux OS.

When you say lower your batch size do you literally mean reduce the amount of face picks I put into the trainer from the extract stage?

Thanks again!

torzdf commented 5 years ago

No, the -bs (--batchsize) option. IE, how many images are processed through the model at a time.

SCB16 commented 5 years ago

Oh ok!

Thanks again, will give this a go and see what happens!

SCB16 commented 5 years ago

Fantastic, I lowered the batch size to 32 from 64 and used the memory saving gradients option along with lightweight trainer and it seems to be running now and showing me previews! Thanks again!