search

nihui / rife-ncnn-vulkan

RIFE, Real-Time Intermediate Flow Estimation for Video Frame Interpolation implemented with ncnn library
MIT License
821 stars 68 forks source link

Request ensemble mode support for rife-4.6. / operation problem '-n' and '-s'. #56

Closed Noir16 closed 1 year ago

Noir16 commented 2 years ago

Can't I just specify ensemble True, Fast False like vapoursynth extension in #50? This can be useful when you prioritize quality over speed.

I tried to directly put the ensemble mode model included in the vapoursynth extension, but an error layer MemoryData not exists or registered occurred.

Also, -n doesn't do frame interpolation, it only seems to increase the number of frames, and -s doesn't work properly. Same issue: #45

nihui commented 2 years ago

try -z option or even -x -z if quality is preferred

temporal tta is aka ensemble

Noir16 commented 2 years ago

Currently, to take advantage of -n values, interpolation was possible using an integer of [number of original frames]x[multiplier value]. ex) 30frames x 8 = 240frames = -n 240

However, there is no interpolation between the first frame and the last frame, since they do not refer to each other. (In videos, interpolation between the first frame and the last frame is unnecessary, but it is necessary when making animations such as gif and webp.)

Can you solve this?

Trimad commented 1 year ago

However, there is no interpolation between the first frame and the last frame, since they do not refer to each other.

As a stopgap, could you append a copy of the first frame to the end of the video before interpolating it?

Noir16 commented 1 year ago

As a stopgap, could you append a copy of the first frame to the end of the video before interpolating it?

Yes, can it. You can interpolate repeated images by input to the last frame name as 0 and the first frame name as 1 and giving appropriate values.

  1. For example, if the number of frames is 30 and multiplied by x4, rife-ncnn-vulkan -i [input a folder] -n 120 -m rife-v4.6 -o [output a folder] n= 120=30x4

  2. rife-ncnn-vulkan -i [A folder included files to last frame name as 0 and the first frame name as 1] -n 8 -m rife-v4.6 -o [output a folder] n= x2=4, x4=8, x8=16...

  3. If there are 8 files in the folder corresponding to 2, delete the files starting from 5~, and remove the first file to replace files 118, 119, and 120 in the folder corresponding to 1.

  4. Use an image2 video software like FFmpeg.

  5. Done.

Noir16 commented 1 year ago

Issue WIPed.