tongyuantongyu / vs-NNVISR

Neural Network Video Interpolation / Super Resolution Filter for VapourSynth
BSD 3-Clause "New" or "Revised" License
46 stars 1 forks source link

Can't get it working,... #2

Closed Selur closed 1 year ago

Selur commented 1 year ago

Tried to test it by extracting everything into "F:/Hybrid/64bit/vsfilters/ResizeFilter/NNVISR" ending with:

F:\Hybrid\64bit\vsfilters\ResizeFilter\NNVISR\cublas64_12.dll
F:\Hybrid\64bit\vsfilters\ResizeFilter\NNVISR\cublasLt64_12.dll
F:\Hybrid\64bit\vsfilters\ResizeFilter\NNVISR\cudart64_12.dll
F:\Hybrid\64bit\vsfilters\ResizeFilter\NNVISR\cudnn64_8.dll
F:\Hybrid\64bit\vsfilters\ResizeFilter\NNVISR\cudnn_adv_infer64_8.dll
F:\Hybrid\64bit\vsfilters\ResizeFilter\NNVISR\cudnn_adv_train64_8.dll
F:\Hybrid\64bit\vsfilters\ResizeFilter\NNVISR\cudnn_cnn_infer64_8.dll
F:\Hybrid\64bit\vsfilters\ResizeFilter\NNVISR\cudnn_cnn_train64_8.dll
F:\Hybrid\64bit\vsfilters\ResizeFilter\NNVISR\cudnn_ops_infer64_8.dll
F:\Hybrid\64bit\vsfilters\ResizeFilter\NNVISR\cudnn_ops_train64_8.dll
F:\Hybrid\64bit\vsfilters\ResizeFilter\NNVISR\models
F:\Hybrid\64bit\vsfilters\ResizeFilter\NNVISR\nvblas64_12.dll
F:\Hybrid\64bit\vsfilters\ResizeFilter\NNVISR\nvinfer.dll
F:\Hybrid\64bit\vsfilters\ResizeFilter\NNVISR\nvinfer_builder_resource.dll
F:\Hybrid\64bit\vsfilters\ResizeFilter\NNVISR\nvinfer_dispatch.dll
F:\Hybrid\64bit\vsfilters\ResizeFilter\NNVISR\nvinfer_lean.dll
F:\Hybrid\64bit\vsfilters\ResizeFilter\NNVISR\nvinfer_plugin.dll
F:\Hybrid\64bit\vsfilters\ResizeFilter\NNVISR\nvinfer_vc_plugin.dll
F:\Hybrid\64bit\vsfilters\ResizeFilter\NNVISR\nvonnxparser.dll
F:\Hybrid\64bit\vsfilters\ResizeFilter\NNVISR\nvrtc-builtins64_121.dll
F:\Hybrid\64bit\vsfilters\ResizeFilter\NNVISR\nvrtc64_120_0.dll
F:\Hybrid\64bit\vsfilters\ResizeFilter\NNVISR\vs-nnvisr.dll
F:\Hybrid\64bit\vsfilters\ResizeFilter\NNVISR\models\fe_n2_2x2_l4_yuv1-1.onnx
F:\Hybrid\64bit\vsfilters\ResizeFilter\NNVISR\models\fe_n4_4x4_l1.onnx
F:\Hybrid\64bit\vsfilters\ResizeFilter\NNVISR\models\ff_n2_2x2_l4_yuv1-1.onnx
F:\Hybrid\64bit\vsfilters\ResizeFilter\NNVISR\models\ff_n4_4x4_l1.onnx

Problem is, when I use:

# Imports
import vapoursynth as vs
# getting Vapoursynth core
core = vs.core

# Loading Plugins
core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/SourceFilter/LSmashSource/vslsmashsource.dll")
core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/ResizeFilter/NNVISR/vs-nnvisr.dll")

# source: 'G:\TestClips&Co\files\test.avi'
# current color space: YUV420P8, bit depth: 8, resolution: 640x352, fps: 25, color matrix: 470bg, yuv luminance scale: limited, scanorder: progressive
# Loading G:\TestClips&Co\files\test.avi using LWLibavSource
clip = core.lsmas.LWLibavSource(source="G:/TestClips&Co/files/test.avi", format="YUV420P8", stream_index=0, cache=0, prefer_hw=0)

# Setting detected color matrix (470bg).
clip = core.std.SetFrameProps(clip, _Matrix=5)
# Setting color transfer info (470bg), when it is not set
clip = clip if not core.text.FrameProps(clip,'_Transfer') else core.std.SetFrameProps(clip, _Transfer=5)
# Setting color primaries info (), when it is not set
clip = clip if not core.text.FrameProps(clip,'_Primaries') else core.std.SetFrameProps(clip, _Primaries=5)
# Setting color range to TV (limited) range.
clip = core.std.SetFrameProp(clip=clip, prop="_ColorRange", intval=1)
# making sure frame rate is set to 25
clip = core.std.AssumeFPS(clip=clip, fpsnum=25, fpsden=1)
clip = core.std.SetFrameProp(clip=clip, prop="_FieldBased", intval=0) # progressive

clip = core.nnvisr.Super(clip, scale_factor=2, use_fp16=True, model="fe_n2_2x2_l4_yuv1-1.onnx", model_path="F:/Hybrid/64bit/vsfilters/ResizeFilter/NNVISR")

# set output frame rate to 25fps (progressive)
clip = core.std.AssumeFPS(clip=clip, fpsnum=25, fpsden=1)
# Output
clip.set_output()

I get:

Failed to evaluate the script:
Python exception: Failed to load F:/Hybrid/64bit/vsfilters/ResizeFilter/NNVISR/vs-nnvisr.dll. GetLastError() returned 1114.

Traceback (most recent call last):
File "src\cython\vapoursynth.pyx", line 3115, in vapoursynth._vpy_evaluate
File "src\cython\vapoursynth.pyx", line 3116, in vapoursynth._vpy_evaluate
File "C:\Users\Selur\Desktop\test_1.vpy", line 8, in 
core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/ResizeFilter/NNVISR/vs-nnvisr.dll")
File "src\cython\vapoursynth.pyx", line 2847, in vapoursynth.Function.__call__
vapoursynth.Error: Failed to load F:/Hybrid/64bit/vsfilters/ResizeFilter/NNVISR/vs-nnvisr.dll. GetLastError() returned 1114.

So the loading of the dll already fails. :(

I'm using portable Vapoursynth R63 (tried R62 too) on Windows 11 (64GB RAM, Ryzen 9 7950x) with a Geforce RTX4080 v535.98 studio drivers.

tongyuantongyu commented 1 year ago

This looks weird since error 1114 is ERROR_DLL_INIT_FAILED but NNVISR does not have dll initialization logic at all, so this could only come from some dependency library.

Can you install https://github.com/adamrehn/dll-diagnostics and run dlldiag deps vs-nnvisr.dll to see if the dependencies can be correctly loaded?

And by the way, you are not placing model files and calling NNVISR correctly. How to place model files is documented here and example to call NNVISR is provided here. We also don't have models trained on bt470bg colorspace, so you probably should convert your clip to bt709 first.

Selur commented 1 year ago

We also don't have models trained on bt470bg colorspace, so you probably should convert your clip to bt709 first. Thanks for that info, I'll keep that in mind when I get it running. Can you install https://github.com/adamrehn/dll-diagnostics and run dlldiag deps vs-nnvisr.dll to see if the dependencies can be correctly loaded? Can't get dlldiag working in my portable Vapoursynth. Using https://www.dependencywalker.com/ dependency_walker it seems to me that everything the dll needs is there.

Okay, I figured it out, vs-nnvisr.dll doesn't look next to it but only searches the PATH variable. So by using explicitly adding the dll folder to the PATH:

# Imports
import vapoursynth as vs
# getting Vapoursynth core
core = vs.core
import site
import os
import ctypes
# Adding torch dependencies to PATH
path = site.getsitepackages()[0]+'/../vsfilters/ResizeFilter/NNVISR'
ctypes.windll.kernel32.SetDllDirectoryW(path)
path = path.replace('\\', '/')
os.environ["PATH"] = path + os.pathsep + os.environ["PATH"]

the dll can be loaded.

Thanks

tongyuantongyu commented 1 year ago

Thanks for the info. This is not intended and a fix will be available later.