search

nekhtiari / image-similarity-measures

:chart_with_upwards_trend: Implementation of eight evaluation metrics to access the similarity between two images. The eight metrics are as follows: RMSE, PSNR, SSIM, ISSM, FSIM, SRE, SAM, and UIQ.
MIT License
585 stars 68 forks source link

Results are significantly different compared to ffmpeg implementation of SSIM and PSNR #28

Open ichlubna opened 3 years ago

ichlubna commented 3 years ago

I've noticed that the SSIM and PSNR values are much higher when using your tool than when trying ffmpeg method such as: ffmpeg -i first.png -i second.png -lavfi ssim -f null - Why is that? Thanks for the tool anyway ^_^.

sfalkena commented 3 years ago

Hi, I noticed this at first as well. It most certainly has to do with the data range that is being used in calculating the values. As a standard in this tool the value is set to 12 bits (4095) (check implementation here. I am not sure what ffmpeg uses, but I suspect it finds it from the datatype of the images. Hope this helps!

ichlubna commented 3 years ago

the value is set to 12 bits (4095)

Hmm would be nice to be able to change it as an argument or to have an automatic detection then. Otherwise the tool can't be really used when comparing for example compression methods etc. with already published results. Thanks for the answer anyway!

nekhtiari commented 3 years ago

@sfalkena thanks for the explanation. that is the reason for your inconsistent result @ichlubna. Fair point and nice suggestion, I will make a PR to make this a flexible argument.

nekhtiari commented 3 years ago

Sorry for taking this so long. Unfortunately, I am a bit busy at the moment with other projects. So it might take some time I can get back to this. In the meantime feel free to contribute or make PR if you feel like it. I would be more than happy :)

lumiaxth commented 1 year ago

Hi, can I ask that ffmpeg's results are reliable? Thx for your attention!

ichlubna commented 1 year ago

Hi, can I ask that ffmpeg's results are reliable? Thx for your attention!

I took ffmpeg as a generally accepted and widely used tool even among scientific community. Do you think the results are not correct? They were usually according to my expectations when I used it.

lumiaxth commented 1 year ago

Hi, can I ask that ffmpeg's results are reliable? Thx for your attention!

I took ffmpeg as a generally accepted and widely used tool even among scientific community. Do you think the results are not correct? They were usually according to my expectations when I used it.

Ok, got it. I'm new here, so I feel confused about two different data. Thanks for your kind reply!

grapemaidaa commented 8 months ago

What's "the value is set to 12 bits (4095)" mean? And where should I set it in this code?