Error with onnxruntime_providers_shared.dll

kedaitinh12 commented 9 months ago

When I put onnxruntime_providers_shared.dll in C:\Program Files (x86)\AviSynth+\plugins64+ or System32. Both AvsPmod and Megui can't load the image and crash straightway. I need to fix it by deleting onnxruntime_providers_shared.dll before load the image and put again after loading the image

My script LoadPlugin("C:\Encode Tools\MeGUI-2944-64\tools\ffms\ffms2.dll") FFImageSource("C:\Users\ADMIN\Downloads\LCFome7.jpeg")

Asd-g commented 9 months ago

Try to put all mlrt_ort runtimes in own folder (for example in "C:\Program Files (x86)\AviSynth+\plugins64+\runtimes") and then create mlrt_ort_loader.avsi in "C:\Program Files (x86)\AviSynth+\plugins64+". Put in the created script:

# Uncomment if DirectML will be used.
#LoadDLL(".\runtimes\DirectML.dll")

# Since you're using CUDA
LoadDLL(".\runtimes\cudart64_12.dll")
LoadDLL(".\runtimes\cublasLt64_12.dll")
LoadDLL(".\runtimes\cublas64_12.dll")
LoadDLL(".\runtimes\cufft64_11.dll")
LoadDLL(".\runtimes\zlibwapi.dll") # cuDNN version 8.3.0+ depends on zlib as a shared library dependency
LoadDLL(".\runtimes\cudnn_ops_infer64_8.dll")
LoadDLL(".\runtimes\cudnn_cnn_infer64_8.dll")
LoadDLL(".\runtimes\cudnn_adv_infer64_8.dll")
LoadDLL(".\runtimes\cudnn64_8.dll")
LoadDLL(".\runtimes\cupti64_2023.3.1.dll")
LoadDLL(".\runtimes\onnxruntime_providers_shared.dll")
LoadDLL(".\runtimes\onnxruntime_providers_cuda.dll")

LoadDLL(".\runtimes\onnxruntime.dll")
LoadPlugin("mlrt_ort.dll")

Keep the file order the same.

kedaitinh12 commented 9 months ago

I got this error and AvsPmod crash right way

This is my mlrt_ort_loader.avsi file LoadDLL(".\runtimes\cudart64_12.dll") LoadDLL(".\runtimes\cublasLt64_12.dll") LoadDLL(".\runtimes\cublas64_12.dll") LoadDLL(".\runtimes\cufft64_11.dll") LoadDLL(".\runtimes\zlibwapi.dll") # cuDNN version 8.3.0+ depends on zlib as a shared library dependency LoadDLL(".\runtimes\cudnn_ops_infer64_8.dll") LoadDLL(".\runtimes\cudnn_cnn_infer64_8.dll") LoadDLL(".\runtimes\cudnn_adv_infer64_8.dll") LoadDLL(".\runtimes\cudnn64_8.dll") LoadDLL(".\runtimes\cupti64_2023.3.1.dll") LoadDLL(".\runtimes\DirectML.dll") LoadDLL(".\runtimes\onnxruntime.dll") LoadDLL(".\runtimes\onnxruntime_providers_cuda.dll") LoadDLL(".\runtimes\onnxruntime_providers_shared.dll") LoadPlugin("mlrt_ort.dll")

Asd-g commented 9 months ago

I explicitly wrote in my previous comment Keep the file order the same. LoadDLL(".\runtimes\onnxruntime_providers_cuda.dll") and LoadDLL(".\runtimes\onnxruntime_providers_shared.dll") should be before LoadDLL(".\runtimes\onnxruntime.dll").

kedaitinh12 commented 9 months ago

Still same My mlrt_ort_loader.avsi LoadDLL(".\runtimes\DirectML.dll") LoadDLL(".\runtimes\cudart64_12.dll") LoadDLL(".\runtimes\cublasLt64_12.dll") LoadDLL(".\runtimes\cublas64_12.dll") LoadDLL(".\runtimes\cufft64_11.dll") LoadDLL(".\runtimes\zlibwapi.dll") # cuDNN version 8.3.0+ depends on zlib as a shared library dependency LoadDLL(".\runtimes\cudnn_ops_infer64_8.dll") LoadDLL(".\runtimes\cudnn_cnn_infer64_8.dll") LoadDLL(".\runtimes\cudnn_adv_infer64_8.dll") LoadDLL(".\runtimes\cudnn64_8.dll") LoadDLL(".\runtimes\cupti64_2023.3.1.dll") LoadDLL(".\runtimes\onnxruntime_providers_cuda.dll") LoadDLL(".\runtimes\onnxruntime_providers_shared.dll") LoadDLL(".\runtimes\onnxruntime.dll") LoadPlugin("mlrt_ort.dll")

kedaitinh12 commented 9 months ago

I got other error while doing this script (i don't call mlrt_ov) LoadPlugin("C:\Encode Tools\MeGUI-2944-64\tools\ffms\ffms2.dll") FFVideoSource("C:\Users\ADMIN\Downloads\LCFome7.jpeg") Import("C:\Program Files (x86)\AviSynth+\plugins64+\mlrt_ort_loader.avsi") z_ConvertFormat(pixel_type="rgbps", colorspace_op="709:709:709:l=>rgb:709:709:f") mlrt_ort(network_path="C:/Program Files (x86)/AviSynth+/plugins64+/models/RealESRGAN/RealESRGAN_x4plus_anime_6B.onnx", builtin=false, fp16=true, provider="dml", device=1, tilesize_w=width/2, tilesize_h=height/2) propSet("_FieldBased",0) deep_resize(1920,1080,grain=0) z_ConvertFormat(pixel_type="yuv420p8", colorspace_op="rgb:709:709:f=>709:709:709:l") New File000000_1 And i need to delete mlrt_ov_loader.avsi and mlrt_ov.dll to get rid of this error, my mlrt_ov_loader.avsi LoadDLL(".\runtimes\gna.dll") LoadDLL(".\runtimes\openvino_auto_batch_plugin.dll") LoadDLL(".\runtimes\oopenvino_auto_plugin.dll") LoadDLL(".\runtimes\openvino_c.dll") LoadDLL(".\runtimes\openvino_gapi_preproc.dll") LoadDLL(".\runtimes\openvino_hetero_plugin.dll") LoadDLL(".\runtimes\openvino_intel_cpu_plugin.dll") LoadDLL(".\runtimes\openvino_intel_gna_plugin.dll") LoadDLL(".\runtimes\openvino_intel_gpu_plugin.dll") LoadDLL(".\runtimes\openvino_intel_hddl_plugin.dll") LoadDLL(".\runtimes\openvino_intel_myriad_plugin.dll") LoadDLL(".\runtimes\openvino_ir_frontend.dll") LoadDLL(".\runtimes\openvino_paddle_frontend.dll") LoadDLL(".\runtimes\openvino_tensorflow_frontend.dll") LoadDLL(".\runtimes\openvino_onnx_frontend.dll") LoadDLL(".\runtimes\openvino.dll") LoadDLL(".\runtimes\tbb.dll") LoadPlugin("mlrt_ov.dll")

Asd-g commented 9 months ago

As for mlrt_ov here there is example how the loader.avsi should be look. tbb.dll should be loaded before openvino.dll.

As for mlrt_ort here there is example how the loader.avsi should be look. Try onnxruntime_providers_shared.dll before onnxruntime_providers_cuda.dll

kedaitinh12 commented 9 months ago

As for mlrt_ov here there is example how the loader.avsi should be look. tbb.dll should be loaded before openvino.dll.

As for mlrt_ort here there is example how the loader.avsi should be look. Try onnxruntime_providers_shared.dll before onnxruntime_providers_cuda.dll

Still same error even I deleted LoadDLL(".\runtimes\DirectML.dll") LoadDLL(".\runtimes\DirectML.dll") LoadDLL(".\runtimes\cudart64_12.dll") LoadDLL(".\runtimes\cublasLt64_12.dll") LoadDLL(".\runtimes\cublas64_12.dll") LoadDLL(".\runtimes\cufft64_11.dll") LoadDLL(".\runtimes\zlibwapi.dll") # cuDNN version 8.3.0+ depends on zlib as a shared library dependency LoadDLL(".\runtimes\cudnn_ops_infer64_8.dll") LoadDLL(".\runtimes\cudnn_cnn_infer64_8.dll") LoadDLL(".\runtimes\cudnn_adv_infer64_8.dll") LoadDLL(".\runtimes\cudnn64_8.dll") LoadDLL(".\runtimes\cupti64_2023.3.1.dll") LoadDLL(".\runtimes\onnxruntime_providers_shared.dll") LoadDLL(".\runtimes\onnxruntime_providers_cuda.dll") LoadDLL(".\runtimes\onnxruntime.dll") LoadPlugin("mlrt_ort.dll")

kedaitinh12 commented 9 months ago

i got this error with mlrt_ov, this error crashes to the desktop of the AvsPmod until I delete onnxruntime_providers_shared.dll, I don't meet this crash anymore but the error still. I realized when I deleted onnxruntime_providers_shared.dll, the AvsPmod open faster My mlrt_ov_loader is like this LoadDLL(".\runtimes\gna.dll") LoadDLL(".\runtimes\openvino_auto_batch_plugin.dll") LoadDLL(".\runtimes\oopenvino_auto_plugin.dll") LoadDLL(".\runtimes\openvino_c.dll") LoadDLL(".\runtimes\openvino_gapi_preproc.dll") LoadDLL(".\runtimes\openvino_hetero_plugin.dll") LoadDLL(".\runtimes\openvino_intel_cpu_plugin.dll") LoadDLL(".\runtimes\openvino_intel_gna_plugin.dll") LoadDLL(".\runtimes\openvino_intel_gpu_plugin.dll") LoadDLL(".\runtimes\openvino_intel_hddl_plugin.dll") LoadDLL(".\runtimes\openvino_intel_myriad_plugin.dll") LoadDLL(".\runtimes\openvino_ir_frontend.dll") LoadDLL(".\runtimes\openvino_paddle_frontend.dll") LoadDLL(".\runtimes\openvino_tensorflow_frontend.dll") LoadDLL(".\runtimes\openvino_onnx_frontend.dll") LoadDLL(".\runtimes\tbb.dll") LoadDLL(".\runtimes\openvino.dll") LoadPlugin("mlrt_ov.dll")

I only make mlrt_ov work when pasting all runtimes of the Openvino direct to C:\Program Files (x86)\AviSynth+\plugins64+

Asd-g commented 9 months ago

Ok. I'll look if I can get access to some machine with NVIDIA.

About mlrt_ov - try the exact same loader:

LoadDLL("\runtimes\tbb.dll")
LoadDLL(".\runtimes\openvino.dll")
LoadPlugin("mlrt_ov.dll")

kedaitinh12 commented 9 months ago

Ok. I'll look if I can get access to some machine with NVIDIA.

About mlrt_ov - try the exact same loader:
LoadDLL("\runtimes\tbb.dll")
LoadDLL(".\runtimes\openvino.dll")
LoadPlugin("mlrt_ov.dll")
another error with only LoadDLL(".\runtimes\tbb.dll") LoadDLL(".\runtimes\openvino.dll") LoadPlugin("mlrt_ov.dll")

Asd-g commented 9 months ago

Do you have mlrt_ov.dll next to mlrt_ov_loader.avsi (in C:\Program Files (x86)\AviSynth+\plugins64+) ?

What I have and no issues:

C:\Program Files (x86)\AviSynth+\plugins64+\openvino_dll folder (with OpenVINO runtimes) that contains:

cache.json
docs
gna.dll
openvino.dll
openvino_auto_batch_plugin.dll
openvino_auto_plugin.dll
openvino_c.dll
openvino_gapi_preproc.dll
openvino_hetero_plugin.dll
openvino_intel_cpu_plugin.dll
openvino_intel_gna_plugin.dll
openvino_intel_gpu_plugin.dll
openvino_intel_hddl_plugin.dll
openvino_intel_myriad_plugin.dll
openvino_ir_frontend.dll
openvino_onnx_frontend.dll
openvino_paddle_frontend.dll
openvino_tensorflow_frontend.dll
pcie-ma2x8x.elf
plugins.xml
tbb.dll
usb-ma2x8x.mvcmd

C:\Program Files (x86)\AviSynth+\plugins64+\mlrt_ov.dll
C:\Program Files (x86)\AviSynth+\plugins64+\mlrt_ov_loader.avsi that cointains:

LoadDLL(".\openvino_dll\tbb.dll")
LoadDLL(".\openvino_dll\openvino.dll")
LoadPlugin("mlrt_ov.dll")

kedaitinh12 commented 9 months ago

OK, when I separate all runtimes .dll of OV and ORT. Openvino works now but ORT still has the same error in both Cuda and DML my mlrt_ort_loader.avsi LoadDLL(".\ort_dll\DirectML.dll") LoadDLL(".\ort_dll\cudart64_12.dll") LoadDLL(".\ort_dll\cublasLt64_12.dll") LoadDLL(".\ort_dll\cublas64_12.dll") LoadDLL(".\ort_dll\cufft64_11.dll") LoadDLL(".\ort_dll\zlibwapi.dll") # cuDNN version 8.3.0+ depends on zlib as a shared library dependency LoadDLL(".\ort_dll\cudnn_ops_infer64_8.dll") LoadDLL(".\ort_dll\cudnn_cnn_infer64_8.dll") LoadDLL(".\ort_dll\cudnn_adv_infer64_8.dll") LoadDLL(".\ort_dll\cudnn64_8.dll") LoadDLL(".\ort_dll\cupti64_2023.3.1.dll") LoadDLL(".\ort_dll\onnxruntime_providers_shared.dll") LoadDLL(".\ort_dll\onnxruntime_providers_cuda.dll") LoadDLL(".\ort_dll\onnxruntime.dll") LoadPlugin("mlrt_ort.dll")

Asd-g commented 9 months ago

Put DirectDML.dll in to own folder (for example, directml_dll). Put onnxruntime.dll in to own folder (for example onnxruntime_dll). Put the rest of runtimes dll in to own folder (for example cuda_dll).

What I have for mlrt_ort:

C:\Program Files (x86)\AviSynth+\plugins64+\onnxruntime_dll folder that contains onnxruntime.dll.
C:\Program Files (x86)\AviSynth+\plugins64+\directml_dll folder that contains DirectML.dll.
C:\Program Files (x86)\AviSynth+\plugins64+\mlrt_ort.dll
C:\Program Files (x86)\AviSynth+\plugins64+\mlrt_ort_loader.avsi that contains:

LoadDLL("C:\Program Files (x86)\AviSynth+\plugins64+\directml_dll\DirectML.dll")

LoadDLL("C:\Program Files (x86)\AviSynth+\plugins64+\onnxruntime_dll\onnxruntime.dll")
LoadPlugin("mlrt_ort.dll")

kedaitinh12 commented 9 months ago

Still have the same error and crash My mlrt_ort_loader.avsi LoadDLL(".\directml_dll\DirectML.dll") LoadDLL(".\ort_dll\cudart64_12.dll") LoadDLL(".\ort_dll\cublasLt64_12.dll") LoadDLL(".\ort_dll\cublas64_12.dll") LoadDLL(".\ort_dll\cufft64_11.dll") LoadDLL(".\ort_dll\zlibwapi.dll") # cuDNN version 8.3.0+ depends on zlib as a shared library dependency LoadDLL(".\ort_dll\cudnn_ops_infer64_8.dll") LoadDLL(".\ort_dll\cudnn_cnn_infer64_8.dll") LoadDLL(".\ort_dll\cudnn_adv_infer64_8.dll") LoadDLL(".\ort_dll\cudnn64_8.dll") LoadDLL(".\ort_dll\cupti64_2023.3.1.dll") LoadDLL(".\ort_dll\onnxruntime_providers_shared.dll") LoadDLL(".\ort_dll\onnxruntime_providers_cuda.dll") LoadDLL(".\onnxruntime_dll\onnxruntime.dll") LoadPlugin("mlrt_ort.dll")

I put onnxruntime_providers_shared.dll to C:\Program Files (x86)\AviSynth+\plugins64+\ort_dll and this file broke Ort Cuda, Ort DML, and Openvino to crash. The Openvino works when put separate all Openvino runtimes into the openvino_dll folder but it's crashes right away with onnxruntime_providers_shared.dll put into the ort_dll folder

SangenBR commented 9 months ago

I tested mlrt_ort and it was very slow compared to mlrt_ncnn

Asd-g commented 9 months ago

@SangenBR, mlrt_ort by default is using CPU. You need to specify provider="cuda" for CUDA or provider="dml" for DirectML.

@kedaitinh12, remove these lines LoadDLL(".\ort_dll\onnxruntime_providers_shared.dll") and LoadDLL(".\ort_dll\onnxruntime_providers_cuda.dll") from the mlrt_ort_loader.avsi (don't delete the files). Put the files onnxruntime_providers_shared.dll and onnxruntime_providers_cuda.dll next to onnxruntime.dll. So you will have:

C:\Program Files (x86)\AviSynth+\plugins64+\onnxruntime_dll folder that contains onnxruntime.dll, onnxruntime_providers_cuda.dll, onnxruntime_providers_shared.dll.
C:\Program Files (x86)\AviSynth+\plugins64+\directml_dll folder that contains DirectML.dll.
C:\Program Files (x86)\AviSynth+\plugins64+\mlrt_ort.dll.
C:\Program Files (x86)\AviSynth+\plugins64+\ort_dll that contains the cuda dlls.
C:\Program Files (x86)\AviSynth+\plugins64+\mlrt_ort_loader.avsi that contains:

LoadDLL(".\directml_dll\DirectML.dll")
LoadDLL(".\ort_dll\cudart64_12.dll")
LoadDLL(".\ort_dll\cublasLt64_12.dll")
LoadDLL(".\ort_dll\cublas64_12.dll")
LoadDLL(".\ort_dll\cufft64_11.dll")
LoadDLL(".\ort_dll\zlibwapi.dll") # cuDNN version 8.3.0+ depends on zlib as a shared library dependency
LoadDLL(".\ort_dll\cudnn_ops_infer64_8.dll")
LoadDLL(".\ort_dll\cudnn_cnn_infer64_8.dll")
LoadDLL(".\ort_dll\cudnn_adv_infer64_8.dll")
LoadDLL(".\ort_dll\cudnn64_8.dll")
LoadDLL(".\ort_dll\cupti64_2023.3.1.dll")
LoadDLL(".\onnxruntime_dll\onnxruntime.dll")
LoadPlugin("mlrt_ort.dll")

SangenBR commented 9 months ago

@SangenBR, mlrt_ort by default is using CPU. You need to specify provider="cuda" for CUDA or provider="dml" for DirectML.

@kedaitinh12, remove these lines LoadDLL(".\ort_dll\onnxruntime_providers_shared.dll") and LoadDLL(".\ort_dll\onnxruntime_providers_cuda.dll") from the mlrt_ort_loader.avsi (don't delete the files). Put the files onnxruntime_providers_shared.dll and onnxruntime_providers_cuda.dll next to onnxruntime.dll. So you will have:

C:\Program Files (x86)\AviSynth+\plugins64+\onnxruntime_dll folder that contains onnxruntime.dll, onnxruntime_providers_cuda.dll, onnxruntime_providers_shared.dll.

C:\Program Files (x86)\AviSynth+\plugins64+\directml_dll folder that contains DirectML.dll.

C:\Program Files (x86)\AviSynth+\plugins64+\mlrt_ort.dll.

C:\Program Files (x86)\AviSynth+\plugins64+\ort_dll that contains the cuda dlls.

C:\Program Files (x86)\AviSynth+\plugins64+\mlrt_ort_loader.avsi that contains:
LoadDLL(".\directml_dll\DirectML.dll")
LoadDLL(".\ort_dll\cudart64_12.dll")
LoadDLL(".\ort_dll\cublasLt64_12.dll")
LoadDLL(".\ort_dll\cublas64_12.dll")
LoadDLL(".\ort_dll\cufft64_11.dll")
LoadDLL(".\ort_dll\zlibwapi.dll") # cuDNN version 8.3.0+ depends on zlib as a shared library dependency
LoadDLL(".\ort_dll\cudnn_ops_infer64_8.dll")
LoadDLL(".\ort_dll\cudnn_cnn_infer64_8.dll")
LoadDLL(".\ort_dll\cudnn_adv_infer64_8.dll")
LoadDLL(".\ort_dll\cudnn64_8.dll")
LoadDLL(".\ort_dll\cupti64_2023.3.1.dll")
LoadDLL(".\onnxruntime_dll\onnxruntime.dll")
LoadPlugin("mlrt_ort.dll")

Yes I activated "cuda" but it's slow for me compared to mlrt_ncnn maybe it's because of my rtx 2060 video card, do you think about bringing trtexec to avs?

kedaitinh12 commented 9 months ago

Still same error

SangenBR commented 9 months ago

put everything in the plugins64 folder like I did but it seems that onnxruntime_providers_cuda.dll slows everything down

kedaitinh12 commented 9 months ago

put everything in the plugins64 folder like I did but it seems that onnxruntime_providers_cuda.dll slows everything down

I think issue relate to onnxruntime_providers_shared.dll

SangenBR commented 9 months ago

put everything in the plugins64 folder like I did but it seems that onnxruntime_providers_cuda.dll slows everything down

I think issue relate to onnxruntime_providers_shared.dll

I also noticed that this dll causes some slowness

Asd-g commented 8 months ago

I had to build onnxruntime instead of getting external versions.

Please redownload runtime files from Releases and follow the updated guide about the runtime files.

@SangenBR, I guess a recent NVIDIA from the high-end/enthusiast class should be give higher fps than mlrt_ncnn. And even with such card provider=dml probably will still give the best fps. Did you test provider=dml? "... do you think about bringing trtexec to avs?" - I don't have NVIDIA gpu so no plans about trtexec.

SangenBR commented 8 months ago

I downloaded and tested again and "CUDA" is still slow for me compared to mlrt_ncnn and "DML" and I get the error that is attached in the image 2024-02-11_21-23-20

Asd-g commented 8 months ago

Share the mlrt_ort_loader.avsi content and where are onnxruntime.dll and DirectML.dll located.

SangenBR commented 8 months ago

mlrt.zip

DirectML.dll and onnxruntime.dll are in the plugins64 dll folder and "CUDA" works a little faster with the last dlls you updated

Asd-g commented 8 months ago

I wanted mlrt_ort_loader.avsi something like the example here. For provider="dml" to work DirectML.dll must be loaded before onnxruntime.dll. This can be done with mlrt_ort_loader.avsi that contains:

LoadDLL("DirectML.dll") # first load DirectML.dll.
LoadDLL("onnxruntime.dll") # then load onnxruntime.dll.
LoadPlugin("mlrt_ort.dll") # finally load the avs plugin.

SangenBR commented 8 months ago

I don't have this mlrt_ort_loader.avsi and I looked in the download files and couldn't find it

SangenBR commented 8 months ago

same mistake

Asd-g commented 8 months ago

mlrt_ort.dll must loaded like this - LoadPlugin("mlrt_ort.dll").

Edit: You need to specify the full path if you will load the files in avspmod. Or create mlrt_ort_loader.avsi and place it in plugins64 (next to mlrt_ort.dll and other dlls).

kedaitinh12 commented 8 months ago

same mistake

Do like this guide https://github.com/Asd-g/avs-mlrt/tree/main/mlrt_ort#runtime-files

Asd-g commented 8 months ago

Or download the attached file and place it in plugins64.

mlrt_ort_loader.zip

SangenBR commented 8 months ago

I keep getting the same error when trying to use "DML"

Asd-g commented 8 months ago

You need to close and reopen AvsPmod in order to mlrt_ort_loader.avsi to take effect. Do not forget to delete the lines in avpsmod that loads the dlls.

SangenBR commented 8 months ago

You need to close and reopen AvsPmod in order to mlrt_ort_loader.avsi to take effect. Do not forget to delete the lines in avpsmod that loads the dlls.

I did it but I don't think that's the problem since it works with "CUDA"

Asd-g commented 8 months ago

Try with the attached onnxruntime.dll (replace the existing one with the atached version). It doesn't support CUDA so test it only for provider="dml".

onnxruntime_only_dml.zip

SangenBR commented 8 months ago

Try with the attached onnxruntime.dll (replace the existing one with the atached version). It doesn't support CUDA so test it only for provider="dml".

onnxruntime_only_dml.zip

the same error, don't worry, let it go, this problem might just be me

kedaitinh12 commented 8 months ago

New runtime files, onnxruntime_providers_shared.dll not crash now but when opening, the app take very much time to load

Asd-g commented 8 months ago

Thanks for the info.

@SangenBR, what version of Windows do you have? >= Windows 10 v1809?

Asd-g / avs-mlrt

Error with onnxruntime_providers_shared.dll #11