This project provides AviSynth+ ML filter runtimes for variety of platforms.
This is a partial port of the VapourSynth plugin vs-mlrt.
To simplify usage, wrappers are provided for all bundled models.
Custom models can be found in this doom9 thread.
Vulkan device (mlrt_ncnn)
Intel GPU (mlrt_ov(... device="GPU"))
NVIDIA GPU (mlrt_ort(... provider="cuda"))
DirectX 12-compatible hardware (mlr_ort(... provider="dml"))
Microsoft VisualC++ Redistributable Package 2022 (can be downloaded from here)
ncnn is a popular AI inference runtime. mlrt_ncnn provides a vulkan based runtime for some AI filters.
It includes support for on-the-fly ONNX to ncnn native format conversion so as to provide a unified interface across all runtimes provided by this project.
OpenVINO is an AI inference runtime developed by Intel, mainly targeting x86 CPUs and Intel GPUs.
The mlrt_ov plugin provides optimized pure CPU & Intel GPU runtime for some popular AI filters. Intel GPU supports Gen 8+ on Broadwell+ and the Arc series GPUs.
ONNX Runtime is an AI inference runtime with many backends.
The mlrt_ort plugin provides optimized CPU, CUDA GPU and DX12 GPU runtime for some popular AI filters.
Device specific benchmarks can be found here.
DirectML benchmarks can be found here.
Requirements:
mlrt_ncnn)mlrt_ov)mlrt_ort)# Build steps:
# Get the latest Vulkan SDK. (mlrt_ncnn)
# Get the latest ncnn release. (mlrt_ncnn)
# Get the latest OpenVINO release. (mlrt_ov)
# Get the latest Microsoft.ML.OnnxRuntime.DirectML. (mlrt_ort)
# Get the latest CUDA Toolkit. (mlrt_ort)
git clone --recurse-submodules --shallow-submodules --depth 1 https://github.com/Asd-g/avs-mlrt
cd avs-mlrt
# Building protobuf:
cd protobuf
cmake -B build_rel -G Ninja -DCMAKE_BUILD_TYPE=Release -Dprotobuf_BUILD_SHARED_LIBS=OFF -Dprotobuf_BUILD_TESTS=OFF -Dprotobuf_MSVC_STATIC_RUNTIME=OFF -DCMAKE_INSTALL_PREFIX=../install
cmake --build build_rel
cmake --install build_rel
# Building onnx:
cd ..\onnx
cmake -B build_rel -G Ninja -DCMAKE_BUILD_TYPE=Release -DONNX_USE_LITE_PROTO=ON -DONNX_USE_PROTOBUF_SHARED_LIBS=OFF -DONNX_GEN_PB_TYPE_STUBS=OFF -DONNX_ML=0 -DCMAKE_INSTALL_PREFIX=../install -DONNX_NAMESPACE=ONNX_NAMESPACE -DCMAKE_PREFIX_PATH=../install
cmake --build build_rel
cmake --install build_rel
# Building boost (optional):
#b2 --with-system --with-filesystem --with-chrono -q --toolset=msvc-14.3 address-model=64 variant=release link=static runtime-link=shared threading=multi --hash --prefix=.\bin\x64
#b2 --with-system --with-filesystem --with-chrono -q --toolset=msvc-14.3 address-model=64 variant=release link=static runtime-link=shared threading=multi --hash --prefix=.\bin\x64 install
# Building onnxruntime (mlrt_ort)
cd ..\mlrt_ort\onnxruntime
cmake -B build_rel -DCMAKE_BUILD_TYPE=Release -G Ninja -Donnxruntime_BUILD_SHARED_LIB=ON -Dprotobuf_BUILD_TESTS=OFF -Donnxruntime_BUILD_UNIT_TESTS=OFF -Donnxruntime_USE_CUDA=ON -Donnxruntime_CUDA_HOME="%CUDA_PATH%" -Donnxruntime_CUDNN_HOME=<...> -DCMAKE_INSTALL_PREFIX=..\..\install -DCMAKE_CUDA_ARCHITECTURES="50;61-real;75-real;86-real;89-real" -Dprotobuf_BUILD_SHARED_LIBS=OFF -Dprotobuf_MSVC_STATIC_RUNTIME=OFF -DONNX_USE_PROTOBUF_SHARED_LIBS=OFF cmake -Donnxruntime_USE_DML=ON -DFLATBUFFERS_BUILD_TESTS=OFF cmake
cmake --build build_rel
cmake --install build_rel
# Building avs-mlrt
# BUILD_MRLT_NCNN=ON (default)
# BUILD_MLRT_OV=ON (default)
# BUILD_MLRT_ORT=ON (default)
cd ..\..
cmake -B build -G Ninja -DCMAKE_PREFIX_PATH=".\install";<Vulkan_SDK_path>;<boost_path> -Dncnn_DIR=<...> -DOpenVINO_DIR=<...>