dyelax / Adversarial_Video_Generation

A TensorFlow Implementation of "Deep Multi-Scale Video Prediction Beyond Mean Square Error" by Mathieu, Couprie & LeCun.
MIT License
734 stars 184 forks source link

SSIM / MS-SSIM implementation #1

Open b3nk4n opened 8 years ago

b3nk4n commented 8 years ago

This is not really an issue, but might be helpful for you: While checking out your (awesome) code, I just discovered this TODO:

# TODO: Add SSIM error http://www.cns.nyu.edu/pub/eero/wang03-reprint.pdf

I was looking for an SSIM/MS-SSIM implementation in TensorFlow for a while. Because I could not discovery anything, I ported some Matlab/C++ code to TensorFlow. I've posted the code on StackOverflow, so feel free to use and improve it:

Stack Overflow: SSIM / MS-SSIM for TensorFlow

dyelax commented 8 years ago

Hey @bsautermeister, thanks for the link! I'll check that out and integrate it when I have a chance. Or, if you'd like to add it in yourself and submit a pull request, I'd be happy to accept.

b3nk4n commented 8 years ago

I will, as soon as I find time for that.

By the way: I'm not sure, but after comparing your PSNR implementation with it's formula in the paper and the PSNR Wikipedia-Definition for colored images:

For color images with three RGB values per pixel, the definition of PSNR is the same except the MSE is the sum over all squared value differences divided by image size and by three.

I think that you miss the by three. As you are using the formula for colored images and take the sum over all 3 dimension (_tf.reducesum(...,[1, 2, 3]), I think you should consider the 3rd dimension for your _numpixels as well:

num_pixels = tf.to_float(shape[1] * shape[2] * shape[3])

Or am I wrong? :)

EDIT

Probably the same thing for sharpness-differences error as well...

dyelax commented 8 years ago

Good catch. Fixed!

daji2112 commented 3 years ago

I know this is very very late but I wanted to get the SSIM @dyelax or @bsautermeister, did one of you got the chance to integrate it with the code, or Is there any other way to calculate the SSIM for this paper like for example, compare_ssim or tf.image.ssim, even can we calculate the psnr with tf.image.psnr..... I would really appreciate a reply. Thank you for this helpful code @dyelax