Deep Motion: A Convolutional Neural Network for Frame Interpolation

Use a DCNN to perform frame interpolation.

Paper: Deep Motion: A Convolutional Neural Network for Frame Interpolation

Based off of the U-Net architecture

Software Requirements

• Keras (tested on v1.1.2)
• TensorFlow (tested on v0.10.0)
• NumPy, SciPy, matplotlib
• OpenCV (tested on v3.1.0, but v2.X should work) (only needed for fps_convert.py)
• FFMPEG (only needed for batch samples generator for YouTube-8M videos)

Model Weights

Download the model weights here.

*Note that the weights are trained using the architecture defined in FI_unet.py/get_unet_2(), which requires input of shape=(6, 128, 384), due to the use of Batch Normalization (probably could do without that)

Training

Details in train.py. It's Keras, so don't worry ;)

Testing

For images, look at DO_TESTING section of train.py

For videos, you can use fps_convert.py to double/quadruple/etc the FPS of any video

Results

View the results at the end of the paper

Watch the presentation video

Presentation Slides