HolyWu / vs-rife

RIFE function for VapourSynth
MIT License
94 stars 7 forks source link

half the image is broken when using 4k content #16

Closed Selur closed 2 years ago

Selur commented 2 years ago

I get a broken output (see attachment), when using:

# Imports
import vapoursynth as vs
# getting Vapoursynth core
core = vs.core
# Loading Plugins
core.std.LoadPlugin(path="i:/Hybrid/64bit/vsfilters/MiscFilter/MiscFilters/MiscFilters.dll")
core.std.LoadPlugin(path="i:/Hybrid/64bit/vsfilters/SourceFilter/LSmashSource/vslsmashsource.dll")
# source: 'G:\TestClips&Co\files\MPEG-4 H.264\4k\Back to the Future (1985) 4k 10bit - 0.10.35-0.11.35.mkv'
# current color space: YUV420P10, bit depth: 10, resolution: 3840x2076, fps: 23.976, color matrix: 2020ncl, yuv luminance scale: limited, scanorder: progressive
# Loading G:\TestClips&Co\files\MPEG-4 H.264\4k\Back to the Future (1985) 4k 10bit - 0.10.35-0.11.35.mkv using LWLibavSource
clip = core.lsmas.LWLibavSource(source="G:/TestClips&Co/files/MPEG-4 H.264/4k/Back to the Future (1985) 4k 10bit - 0.10.35-0.11.35.mkv", format="YUV420P10", cache=0, fpsnum=24000, fpsden=1001, prefer_hw=1)
# Setting color matrix to 2020ncl.
clip = core.std.SetFrameProps(clip, _Matrix=9)
clip = clip if not core.text.FrameProps(clip,'_Transfer') else core.std.SetFrameProps(clip, _Transfer=9)
clip = clip if not core.text.FrameProps(clip,'_Primaries') else core.std.SetFrameProps(clip, _Primaries=9)
# Setting color range to TV (limited) range.
clip = core.std.SetFrameProp(clip=clip, prop="_ColorRange", intval=1)
# making sure frame rate is set to 23.976
clip = core.std.AssumeFPS(clip=clip, fpsnum=24000, fpsden=1001)
clip = core.misc.SCDetect(clip=clip,threshold=0.150)
from vsrife import RIFE
# adjusting color space from YUV420P10 to RGBS for VsTorchRIFE
clip = core.resize.Bicubic(clip=clip, format=vs.RGBS, matrix_in_s="2020ncl", range_s="limited")
# adjusting frame count&rate with RIFE (torch)
clip = RIFE(clip, scale=0.5, multi=3, device_type='cuda', device_index=0, fp16=True) # new fps: 71.928
# adjusting output color from: RGBS to YUV420P8 for x264Model
clip = core.resize.Bicubic(clip=clip, format=vs.YUV420P8, matrix_s="2020ncl", range_s="limited", dither_type="error_diffusion")
# set output frame rate to 71.928fps
clip = core.std.AssumeFPS(clip=clip, fpsnum=8991, fpsden=125)
# Output
clip.set_output()

tried different scale values, fp16 disabled, without scene change detection and other values for mult, nothing helped. https://github.com/HomeOfVapourSynthEvolution/VapourSynth-RIFE-ncnn-Vulkan works fine. 2k content also works fine. I tried different source filters and different files. Would be nice if this could be fixed.

attachment was too large: https://ibb.co/WGT9pvL

HolyWu commented 2 years ago

That's an issue in the upstream implementaion. See https://github.com/hzwer/arXiv2021-RIFE/issues/188 and https://github.com/hzwer/Practical-RIFE/issues/13. So just use ncnn-Vulkan version since it's not slower than the PyTorch version now.

Selur commented 2 years ago

Oh, okay thanks for looking into it. :)