Convert fails with "out of memory"

[Kirin-kun]

I thought I would give it a shot again after the latest corrections, so I tried to make a video, but I never could merge the faces back.

The video was originally 1280x720, extracted with ffmpeg: dlib cnn extractor worked like a charm since the latest correction in the face detection. Learning worked too.

But when trying to convert, it failed with OOM. Images too big? So I tried to resize the extracted video to 900x506 and after extraction to redo the alignments, it failed again, with OOM.

I then resized them down to 640x360. OOM again!

Command used to try to merge:

python faceswap.py convert -i C:\fakes\data_A -o C:\fakes\data_A\merged --serializer json --alignments C:\fakes\data_A\faces\alignments.json -m C:\fakes\model -b 10 -e 10 -D cnn

With or without -D cnn, it fails the same.

I have a GTX 1060 6Gb, so I don't think it should be that bad.

[blvckmvgicdotexe]

I have the exact same problem with 1280x720 frames and a GTX 1060 6GB python faceswap.py convert -i C:\videoframes -o C:\videoframes\converted -v -m C:\model -t Original -c Masked -D hog -fr 5881-8140 -b 5 -S -M facehullandrect -e 0 Can't process any frame, always OOM: Error while calling cudaMalloc(&data, new_size*sizeof(float)) in file C:\env\faceswap\dlib\dlib\dnn\gpu_data.cpp:195. code: 2, reason: out of memory

[iperov]

@Kirin-kun --alignments c;\fakes\data_A\faces\alignments.json ;

no alignments was loaded, so used face extractor inside converter, this cause no mem for dlib+keras extractor+keras converter.

@blvck-mvgic same you didnt provide --alignments for convert

@Clorr I suggest disable converting if no alignments was loaded

[Kirin-kun]

It was a typo when I filled this issue. I changed the path here to shorten it.

It still fails, whether there's an alignment file or not.

And anyway, if the path to the file is mistyped, the program stops with "No such file or directory".

So, no, it doesn't use the face extractor inside the converter.

[blvckmvgicdotexe]

What if the dataset is made off multiple sources AND some images were deleted? I have multiple alignments.json files in different folders

[Kirin-kun]

I think/hope the converter loads an image, then searches the landmarks in the alignment file to know where to put the face and then proceeds to do its job. If it doesn't find it, it should copy it as-is.

That pictures present in the alignments file would be missing from the directory should be irrelevant.

Or the error comes from that it tries to run the face extractor inside the converter if it can't find it in the alignments file? Instead of ignoring it.

The first images of the dataset don't contain any face.

[blvckmvgicdotexe]

Ok this is what I did: 1) Extracted ALL target video frames with ffmpeg (instead of 1 frame per second as I did for training) 2) Deleted the frames I didn't need as I didn't wan't to wait hours to see the result and got something around 2500 frames which is like 1 minute and half of video, pretty fair 3) Ran python faceswap.py extract with target video folder to generate the alignments.json file using CNN (it worked perfectly btw unlike a week ago) 3) Ran convert on target video folder with frames without the --aligments parameter, it found the alignments by itself

Working perfectly fine Thanks @iperov, now it is working! Btw I'm no python programmer but wouldn't it be better to add a parameter that specifies whether the alignment with keras has to be run before the conversion so that tensorflow can do its magic without OOM? Would be basically what I did but automated

[Kirin-kun]

@blvck-mvgic conversion happened with 1280x720 frames?

[blvckmvgicdotexe]

@Kirin-kun yes, without any OOM

Also, GPU memory usage seems pretty normal, capped at 5.3GB as intended with tensorflow

The frames look completely fine, doing the extraction and alignment again on the new video frames did the trick.

[Kirin-kun]

Funnily, it looks like it's working when you do NOT give the alignments parameter and keep everything at default.

It found the alignment file by itself after the extraction saved it in the default location and happily converted.

Well, case closed.

[Kirin-kun]

When I let everything at default instead of giving the parameters, it worked.

[ppmdo]

@Kirin-kun It seems I have the same problem. I can extract using CNN with no problem, but on convert, it fails with an "out of memory error".

If you don't set any parameters it works because it uses the Hog detector.