search

ryanxingql / stdf-pytorch

Implementation of "Spatio-Temporal Deformable Convolution for Compressed Video Quality Enhancement" (AAAI'20).
https://www.aiide.org/ojs/index.php/AAAI/article/view/6697
Apache License 2.0
155 stars 20 forks source link

Question about the data format #6

Closed linjing7 closed 3 years ago

linjing7 commented 3 years ago

Hi, thank you very much for your reply in the afternoon, I'm sorry to trouble you again. I want to use optical flow in VQE, however, the input of the optical flow network has RGB channel instead of Y channel, so I think that I should make yuv channel lmdb instead of y lmdb, and then convert it to rgb channel before inputted to the optical flow network. So the only change of the origin code is set 'only_y=False' in the import_yuv fuction, is it right? BTW, there are several algorithm to convert data from yuv420 to rgb. Which one do you think is better?

1.R = Y + 1.402 (V - 128) G = Y - 0.34413 (U - 128) - 0.71414 (V - 128) B = Y + 1.772 (U - 128)

2.out_img = np.matmul(img, [[0.00456621, 0.00456621, 0.00456621], [0.00791071, -0.00153632, 0], [0, -0.00318811, 0.00625893]]) * 255.0 + [-276.836, 135.576, -222.921] The second one is from BasicSR~

ryanxingql commented 3 years ago

  1. Maybe right. You may first have a try. If you have RGB input such as PNG images, you can simply set the in_nc and out_nc being 3, and use disk IO instead of LMDB IO. See: https://github.com/RyanXingQL/PythonUtils/blob/4c2aa9b7ae9c72c8aa9200b5e3bb569a54c92f35/dataset.py#L266.

  2. To convert YUV420P to RGB, I suggest: YUV420P -> YUV444P -> RGB. See: https://github.com/RyanXingQL/PythonUtils/blob/4c2aa9b7ae9c72c8aa9200b5e3bb569a54c92f35/conversion.py#L177 https://github.com/RyanXingQL/PythonUtils/blob/4c2aa9b7ae9c72c8aa9200b5e3bb569a54c92f35/conversion.py#L132

linjing7 commented 3 years ago

Okay ,thank you very much. I'll have a try.