search

gabrieleilertsen / hdrcnn

HDR image reconstruction from a single exposure using deep CNNs
https://computergraphics.on.liu.se/hdrcnn/
BSD 3-Clause "New" or "Revised" License
508 stars 101 forks source link

how about video reconstruction #9

Closed guoyejun closed 6 years ago

guoyejun commented 6 years ago

Hi,

did you try this method to reconstruct a HDR video from a LDR video? Is it possible? thanks.

one thing is that for the typical video revolutions such as 720p (1280720) and 1080p (19201080), the size is not multiple of 32, there might be issue for the image resize.

gabrieleilertsen commented 6 years ago

Hi! Yes, I have tried that, reconstructing frame by frame. It works fine in some sequences, for example when you have some motion (fire, etc.). However, if you have saturated areas that do not move between frames, you can get a reconstruction that changes from one frame to the other due to small changes in the pixel intensities. Extending the method to handle video without temporal inconsistencies would be an interesting topic for future work.

And you are right about the resolution. One solution is to use zero padding to extend the resolution to the nearest height that is a multiple of 32, i.e. ceil(H/32)*32. Maybe it would be convenient to have this option in the reconstruction script, where it automatically performs zero padding and then clips the reconstructed image to the original resolution?

guoyejun commented 6 years ago

hope to soon see your new achievement to resolve the video reconstruction issues, :)

guoyejun commented 6 years ago

from video perspective, the format is usually YUV, while looks that this model accepts RGB format. Is there a way to let the model accept YUV format? thanks.

gabrieleilertsen commented 6 years ago

The trained parameters are for RGB, so you will have to convert the reconstructed images for video encoding. Alternatively, you can use the training script to learn reconstruction in a different color space. In that case, you will have to make sure to use a direct pixel-wise loss instead of the default illuminance/reflectance separated loss (set the flag --sep_loss=false), since the separated loss relies on converting RGB to illuminance + reflectance.