Add video support

[Clorr]

I'll try to tackle that one with the simple cv2 support

[Ganonmaster]

Does OpenCV make it possible to interpolate face positions between frames? I've had several instances where I've tried to convert a series of frames, but it fails to detect faces in a few frames in between; causing a glitching effect where the target face turns on and off. (link semi-nsfw)

[Clorr]

Indeed this a common issue. Face detection is often blocked by compression artifacts. It is the case in still pics and is even more present in videos.

I don't think there is a built in interpolation, but you can try doing it manually. While parsing frames, just keep a track of 1 or 2 previous frames and if current frame has nothing detected, do a 'mean' with previous data.

Other possibility is to keep the result of the last detection, and keep frames without faces until you detect another face. If the new face is close to the last, apply the new detection (or the old, or an interpolation) to the buffered frames. (Don't forget to give a max size to your buffer and flush it if no new face has been detected for x frames)

I'll try to tackle that later on...

[Clorr]

For now, I have an issue with input video reading. It seems python-opencv has no video decoder so I had to compile the full version...

[Clorr]

As I see on your video, there are also little glitches due to the face generator that may generate different faces for 2 consecutive frames. I don't see a clear solution here for now.

[Clorr]

I'm giving up for now, opencv-python seems not to have video support, and the full opencv lib is a pain to build....

I'll try to handle sequences of photos instead (video will just be a sequence of frames so it will adapt easily to videos)

[Clorr]

Remark: ‘python-opencv‘ seems not to have video support built-in as default. Is ‘moviepy‘ a better alternative?

[Ganonmaster]

In the end, all these libraries boil down to is basic interaction with ffmpeg. I'm pretty sure python-OpenCV has this, but only if configured to use it. I would opt for the library with the most straightforward/pythonic interface. I would guess that is most likely moviepy, but I will look further into this after I've recovered from the new years hangover.

[IcyTv]

How about taking existing code from somewhere that extracts all frames from a video, running the face swap over those and then taking the output and combine it to a video?

[fetchlister]

@IcyTv Thats just writing a 4 line bash script to run the existing tools....