search

Blinue / Magpie

An all-purpose window upscaler for Windows 10/11.
GNU General Public License v3.0
9.17k stars 484 forks source link

Add Nvidia Maxine AI Video Upscaler (RTX Video Super Resolution) #560

Open Michael1297 opened 1 year ago

Michael1297 commented 1 year ago

Expected behavior 预期的功能

more information and tests https://github.com/AaronFeng753/Waifu2x-Extension-GUI/issues/449

General info: https://developer.nvidia.com/maxine SDK package download: https://catalog.ngc.nvidia.com/orgs/nvidia/teams/maxine/resources/maxine_windows_video_effects_sdk_ga (Nvidia account required to download) Usage guide: https://docs.nvidia.com/deeplearning/maxine/vfx-sdk-system-guide/index.html Github but use the SDK download instead because that comes with everything: https://github.com/NVIDIA/MAXINE-VFX-SDK SDK guide but probably not relevant to us: https://docs.nvidia.com/deeplearning/maxine/vfx-sdk-programming-guide/index.html

Alternative behavior (optional) 近似的功能（可选）

No response

hooke007 commented 1 year ago

Waifu2x-Extension-GUI is a fake semi-open-source project. Nothing valuable we could get. Currently there is no shader example. BTW, is RTX Video Super Resolution open?

Michael1297 commented 1 year ago

RTX VSR = Nvidia Maxine

Michael1297 commented 1 year ago

the tests are taken from https://github.com/AaronFeng753/Waifu2x-Extension-GUI/issues/449 https://imgsli.com/MTQ1MTAx/0/1 https://imgsli.com/MTQ1MTA2/0/1

sdk only ☹️ In Waifu2x-Extension-GUI, for the scaling algorithm to work, you need to install the sdk from https://www.nvidia.com/en-us/geforce/broadcasting/broadcast-sdk/resources/

hooke007 commented 1 year ago

As I said,

Waifu2x-Extension-GUI is a fake semi-open-source project. Nothing valuable we could get.

MagpieFX is compute shader which is based on DirectX 11, that's why we should know if it could be converted into a shader at first.

Blinue commented 1 year ago

After reviewing the documentation, I believe that integrating RTX Video Super Resolution is not impossible. Although it is not open source, it provides APIs to cooperate with D3D. I will give it a try.

GT500org commented 1 year ago

Interesting note from someone who uses Waifu2x-Extension-GUI for video upscaling, the RTX Video Super Resolution that it uses NVIDIA Maxine for seems to use CUDA cores for processing instead on Tensor cores. I don't know if that means it will work on GPU's that don't have Tensor cores (GTX GPU's for instance), although the experience may not be good on older GPU's with fewer/slower CUDA cores (I have an RTX 3070 Ti).

Insan1ty0ne commented 1 year ago

After reviewing the documentation, I believe that integrating RTX Video Super Resolution is not impossible. Although it is not open source, it provides APIs to cooperate with D3D. I will give it a try.

Would be amazing to see Nvidia's RTX Video Super Resolution (aka Nvidia Maxine) added to Magpie. It does wonders even on YouTube videos playing via. Google Chrome.

CberYellowstone commented 2 months ago

@Blinue RTX超分 是否可以进一步推进？NVIDIA于上月已经发布了完整的NVIDIA RTX Video SDK：rtx_video_sdk_v1.0.2.zip，提供了完整的文档以及dx11/dx12接口，实现起来应该不会很难 顺带一提，NVIDIA RTX Video SDK中还附带了TrueHDR这一接口，可以将SDR内容提升为HDR内容，在支持HDR的显示器上能大幅提高观感，能否一并实现？