Vapoursynth: low performance with RIFE plugin

[chainikdn]

RIFE plugin: https://github.com/HomeOfVapourSynthEvolution/VapourSynth-RIFE-ncnn-Vulkan SVP integration: https://www.svp-team.com/wiki/RIFE_AI_interpolation

After a few recent updates its performance is now somehow usable for real-time playback. For example, rtx 2060 gives ~170 fps converting 480p and ~70fps converting 720p, so it should also work in a real time. And so it does in mpv.

But in MPC-HC it only gives 0.6-0.8 rate (480p) or even 0.25 rate (720p). "Frame rate" on the Status page shows low values including source frame rate value.

The minimal script w/o SVP:

import vapoursynth as vs
core = vs.core
core.num_threads = 4

core.std.LoadPlugin("RIFE.dll")

inp1 = VpsFilterSource.misc.SCDetect()
inp2 = inp1.resize.Bicubic(format=vs.RGBS, matrix_in_s="709")
smooth = core.rife.RIFE(inp2,model=9,gpu_id=0,gpu_thread=2,tta=False,sc=True)
smooth = smooth.resize.Point(format=inp1.format.id, matrix_s="709")

smooth.set_output()

[CrendKing]

Can't reproduce. I'm pretty much using your script (w/o SVP) in both mpv and MPC. Playing a 720p 25fps video. Both is smooth as butter (as 50fps). Below is a screenshot of the stats page.

Resource wise, mpv is using 46.8% of a single core (My Ryzen 5800 has 16 cores) and 81.44% of whole GPU (RTX 2070 Super). MPC + VPSF is using 80.4% of single core and 53.3% of whole GPU. So neither is hitting a throttle.

Could you profile on your end and find out where is the bottleneck? Could it be hardware specific?

[CrendKing]

Also, do you know what happens to https://github.com/HolyWu/vs-rife? I thought that one excited some people for a while. Is it dead? Does that have better performance?

[chainikdn]

Can't reproduce.

Hmmmm....

Does that have better performance?

after recent ncnn/Vulkan updates the perf difference is much smaller (like ~25%, comparing to x3-x4 times before). in exchange for huge DLC and several seconds (re-)init time

[chainikdn]

Could you try MPC-HC instead of MPC-BE?

[CrendKing]

several seconds (re-)init time

Noticed this. The first init time is long. Init time on seek is tolerable.

Could you try MPC-HC instead of MPC-BE?

Same result. The video is played perfectly. Same resource wise. Also tested EVR in MPC-HC and worked well.

[chainikdn]

Turns out that the list of Vulkan devices somehow depends on the application. On my laptop MPC-HC shows two devices - i.e. AMD IGP is id0 and RTX 2060 is id1, while MPC-BE & mpv can only see RTX 2060 as id0. No idea why, but it's definitely not AVSF's issue.