search

chxy95 / HDRTVNet

ICCV2021 - A New Journey from SDRTV to HDRTV.
MIT License
128 stars 22 forks source link

Possibility of real-time mpv shader version? #11

Open SephirothFFKH opened 2 years ago

SephirothFFKH commented 2 years ago

Hi, HDRTVNet looks really great! I was wandering if it would be possible to make a shader version of HDRTVNet for real-time use as a mpv shader or a simplified version of it. Looking at the list of User Shaders for mpv (https://github.com/mpv-player/mpv/wiki/User-Scripts#user-shaders), all current Deep Learning shaders are only for spatial upscaling and HDRTVNet would be the first shader for color space upmapping and it would make HDRTVNet more practical to use.

Thank you very much

chxy95 commented 2 years ago

@SephirothFFKH Thanks for your suggestion very much! However, in actual I'm not familiar with the process of how to make a shader version and I don't have the time to do the project currently as well as in the near future. I would consider to make a simplified version but I'm not sure when this will be done. I would be very grateful if you can provide more practical advice and help~

SephirothFFKH commented 2 years ago

Thank you @chxy95 for your response. I definitely understand that you are busy right now and I thank you for your amazing work on HDRTVNet.

I have looked into how to produce deep learning shaders for mpv and I have found that the project Anime4K has done this for a spatial super resolution shader from Tensorflow here: https://github.com/bloc97/Anime4K/blob/master/tensorflow/Gen_Shader.ipynb Also, I have found a medium blog post on inference using shader here: https://medium.com/analytics-vidhya/inferring-a-super-resolution-neural-network-on-raspberry-pi-gpu-89b5456d21ef Alternatively, it also seems possible to do inference of a existing model in mpv without shader translation by using vapoursynth which is used in the project VSGAN here: https://github.com/rlaphoenix/VSGAN

I definitely understand that it is not something you can do right now or in the near future but if you ever return to HDRTVNet in the far future, this would be something very interesting to explore as real-time HDR content generation would be really great!

Thank you very much