integrate into ffmpeg

[guoyejun]

Hi,

I tried to integrate this algorithm into ffmpeg, and just sent out the patch to wait for community reviews, see https://lists.ffmpeg.org/pipermail/ffmpeg-devel/2018-October/235222.html. any comments? thanks.

[gabrieleilertsen]

Hi! Interesting. So you can actually use Tensorflow from within ffmpeg, to run the reconstruction as a part of an ffmpeg command? How would the weights be supplied, externally or as a part of the ffmpeg library?

I am not that familiar with ffmpeg, so I cannot comment that much about the patch itself, but it sounds interesting to integrate the HDR reconstruction with ffmpeg. As for reconstructing video sequences, there will probably be situations where flickering will be an issue, since the reconstruction is only trained on static images and can change a lot for small changes in the input frames.

[guoyejun]

yes, it is implemented as a ffmpeg filter. As for the network model and weights, it can be saved in a single file within python script, and the saved file is used as the ffmpeg filter command line parameter.

I tried several videos and I do not see a visual difference, (neither more detail nor flickering). Let's take it as a next step. :)

To generate the model file, we need modify the original script a little.

• set name='y' for y_final within script at https://github.com/gabrieleilertsen/hdrcnn/blob/master/network.py
• add the following code to the script at https://github.com/gabrieleilertsen/hdrcnn/blob/master/hdrcnn_predict.py

graph = tf.graph_util.convert_variables_to_constants(sess, sess.graph_def, ["y"]) tf.train.write_graph(graph, '.', 'graph.pb', as_text=False)

the filter's parameter looks like: sdr2hdr=model_filename=/path_to_tensorflow_graph.pb:out_fmt=gbrp10le

[gabrieleilertsen]

Ok, sounds good. Thanks for the efforts!