Boost efficiency by 300% by adding an interpolation engine

[JustMaier]

Basically once you put the images through an interpolation engine like RIFE (the interpolator used by FreeFlow), 10 Steps looks just as good as 30 steps in most cases. As such, I think it would be a massive bump in efficiency if we could integrate RIFE into this script so that when we generate the video we can automatically interpolate up to a target framerate from whatever number of Travel Steps they chose.

I could try to take a stab at it, but thought I'd put the issue out in case someone else was interested in picking it up.

[dfaker]

You could drop the moviepy dependency and use straight ffmpeg and the 'minterpolate' filter, zulko has a pretty good example of using numpy to pipe frames into a ffmpeg subprocess at https://github.com/Zulko/moviepy/blob/master/moviepy/video/io/ffmpeg_writer.py allowing you to drop the extra dependency as AUTOMATIC1111/stable-diffusion-webui already depends on numpy

[JustMaier]

I thought about using ffmpeg directly, but I didn't see it listed in the requirements file and so I wasn't sure if it was already in the project.

As for the minterpolate filter, I'll have to give that a try.

[JustMaier]

Ok, I tried minterpolate and the quality just isn't good enough imo:

• 30fps with minterpolate blend
• 30fps with minterpolate mci
• RIFE 30fps

[dfaker]

Yes, the RIFE example is far superior!

[yownas]

This is a really good idea. I'm not sure if I know how to implement it though. :(

If anyone wants to give it a try I would be glad if you were able to toggle it on/off. Part of interpolating between the seeds if to explore how Stable Diffusion react to the changes....but on the other hand sometimes you just want a cool/smooth video. :)

[yownas]

RIFE code added. :)

[JustMaier]

Awesome!