baowenbo / DAIN

Depth-Aware Video Frame Interpolation (CVPR 2019)
https://sites.google.com/view/wenbobao/dain
MIT License
8.19k stars 840 forks source link

Getting double the original fps instead of TARGET_FPS #119

Closed KarimMibrahim closed 3 years ago

KarimMibrahim commented 3 years ago

Hello, I am getting a strange problem where it seems that the output is I am getting is double the original frame rate instead of the target frame rate.

For example, the original fps is 25, and the video has 16559 frames. The target fps is 60 and the output should've been (16559/2560) = 39741.6 The output I get has 33116, which corresponds to a converted fps of 50. (33116/16559 25 =50)

I have tried this with a different video with the same result. I have tried with another fps (just selected 90 to see what happens) an the result was this time it is multiplied by 3 (where it should have been 3.6). Is this a precision problem where numbers are rounded somewhere? Am I missing something here or doing something wrong?

The experiments were done with the colab notebook code.

AlphaGit commented 3 years ago

Hi! The interpolation is done one frame at a time, so the amount on the frames on the video is not taken in consideration. This is, if the video original fps is 25, and the target fps is 60, the ratio is 60/25 = 2.4. This is rounded to 2, which means that DAIN will double the frames. (Will add 1 frame in between two other frames.)

If the target FPS was 75, then the ratio would be 3, resulting in DAIN generating 2 frames in between for every existing two frames.

Indeed, you're right: there is some level of rounding, but this can't be helped, since you cannot generate "2 and a half" frames. DAIN must generate an integer number of frames.