search

JuheonYi / WESPE-TensorFlow

TensorFlow implementation of "Weakly Supervised Photo Enhancer for Digital Cameras"
53 stars 10 forks source link

tf.image.total_variation was wrongly used #2

Closed IPNUISTlegal closed 6 years ago

IPNUISTlegal commented 6 years ago

https://www.tensorflow.org/api_docs/python/tf/image/total_variation This can be used as a loss-function during optimization so as to suppress noise in images. But, your just use it to calculate image gradients. Maybe tf.image.image_gradients function is appropriate. https://github.com/JuheonYi/WESPE-TensorFlow/blob/8f94a0abf6687ab842a76b23f3d6134cb8da582d/DPED.py#L85 image thx for your reply!

JuheonYi commented 6 years ago

Hi. Thanks for your comment. I don't know if I fully understood your question, but currently I am using the tv_loss term as part of the loss to be optimized. (i.e., G_loss includes tv_loss, where G_optimizer minimizes the G_loss)

Could you elaborate on your comment?

Thanks!

IPNUISTlegal commented 6 years ago

You maybe wrongly calculate image gradients! you use tf.image.total_variation to calculate image gradients (dy, dx) ,right? image BUT, tf.image.total_variation in tensorflow seems to denoising for image! What do you think about tf.image.image_gradients to returns image gradients (dy, dx) for each color channel? thx

JuheonYi commented 6 years ago

From what I understand, in the TF document you mentioned, it says that tf.image.total_variation calculates the 2D anisotropic version of total variation, which is the same formula as the term you circled in red (After all, the name of the function itself indicates that it calculates the total variation)

Since total variation is a regularization that forces the neighboring pixels to have similar values, it can work as a denoising term.

I guess tf.image.image_gradients can work the same role. Please share the result when you try with the function.

Also, from my experience, it seems that TV loss term does not play a key role in performance. I guess one reason might be that the texture loss (calculated by GAN) also plays a similar role in reducing artifacts in the enhanced image.

Thanks!

IPNUISTlegal commented 6 years ago

Thanks for your detailed answer！ I just want to add some regularization to GAN loss function in image enhancement. Maybe i can't try your whole code , just try TV loss, whatever,appreciate that!