search

AmusementClub / vs-mlrt

Efficient CPU/GPU ML Runtimes for VapourSynth (with built-in support for waifu2x, DPIR, RealESRGANv2/v3, Real-CUGAN, RIFE, SCUNet and more!)
GNU General Public License v3.0
290 stars 19 forks source link

Is it possible to add other interpolation methods? #92

Open zelenooki87 opened 5 months ago

zelenooki87 commented 5 months ago

Is it possible to add other interpolation methods? I would assume IFRNet would be the easiest to implement because there is a Vulkan ported version. Is it possible to add some other interpolation from this benchmark page? https://github.com/AIVFI/Video-Frame-Interpolation-Rankings-and-Video-Deblurring-Rankings Thank you very much in advance

WolframRhodium commented 5 months ago

RIFE is the only model that is in the top rank in terms of perceptual quality with available pre-trained models, according to this ranking.

I would assume IFRNet would be the easiest to implement because there is a Vulkan ported version.

This is false.

zelenooki87 commented 5 months ago

Oh, that was my layman's assumption. Some videos I interpolated with ema-vfi and ifrnet in a conda environment seemed to me to be somewhat better than rife. In one of the previous issues I opened, you wrote to me that ema-vfi is too slow to consider adding it. Has anything changed in the meantime? You added swinir and scunet and big respect for that. :) Greetings

WolframRhodium commented 5 months ago

Now the GridSample operator is the bottleneck that affects all modern video frame interpolation models.

zelenooki87 commented 3 months ago

@WolframRhodium What about to implement new interpolation method? https://github.com/mulns/PerVFI

zelenooki87 commented 3 months ago

or this one? (seems better) https://github.com/JHLew/MoMo

WolframRhodium commented 3 months ago

You can do these yourself.