Open mlbrnm opened 1 year ago
If the issue disappears with `fp16` disabled, it could be a precision problem?
If the issue disappears with
fp16
disabled, it could be a precision problem?
Unfortunately I am not that knowledgeable regarding AI/CUDA/etc and do not know how to test that, but I'd be happy to try if you have instructions/info.
Find test.py
in the svp installation and look into it.
Here is my config.
clip = RIFE(clip, multi=2, model=46, _implementation = 2, backend=BackendV2.TRT( device_id=0, fp16 = False, num_streams=2, output_format=1, use_cuda_graph=True, use_cudnn=False, workspace=None, static_shape = 1))
fp16=False + output_format=0 : slower than RIFE v1, but doesn't affect video quality. fp16=False + output_format=1 : Same as above. fp16=True + output_format=1: Faster than RIFE v1, but video quality looks really bad, especially on 4k videos.
I use the video to test https://drive.google.com/file/d/1dfR5TTnsQWX9PVIl-PVujFM_bGfEXUvJ/edit
no rife or RifeV1: https://cdn.discordapp.com/attachments/290709370600423424/1095138870187798578/LG_4K_HDR_Demo_-_New_York.ts_4_11_2023_7_09_06_AM.png
RIFEv2 + fp16 true https://cdn.discordapp.com/attachments/290709370600423424/1095138871580315688/LG_4K_HDR_Demo_-_New_York.ts_4_11_2023_7_09_37_AM.png
https://github.com/AmusementClub/vs-mlrt/issues/22#issuecomment-1447480396 I retested and the confirmed that v2 could not run with fp16. You doesn't use the flag force_fp16, so it might skip many things.
I also enabled fp16 in the same way, and when I frame interpolated a 2560x1440 video, a major jagged edge occurred. However, when I frame interpolate a 1920x1080 video with the same settings, no jaggedness occurs. Also, if fp16 is disabled, no jaggedness occurs.
Please see the last few posts on this thread:
https://www.svp-team.com/forum/viewtopic.php?id=6281&p=30
This was hypothesized by one person, then confirmed by 2 others on different setups using different media players.
Seems like the V2 models cause major jagged edges in certain situations.
https://i.imgur.com/gqoKFqP.png
https://cdn.discordapp.com/attachments/290709370600423424/1095130933562187847/image.png
But thanks for your great work on this in general. Love it.