Evaluate butteraugli on TID-2013 and publish results

google / butteraugli

butteraugli estimates the psychovisual difference between two images

Apache License 2.0

1.96k stars 137 forks source link

Evaluate butteraugli on TID-2013 and publish results #40

Open sp-1234 opened 6 years ago

sp-1234 commented 6 years ago

http://www.ponomarenko.info/tid2013.htm Metrics are ranked by their Spearman correlation with Mean Opinion Score.

Someone who has hardware powerful enough to make it practical could do this. It would be a real proof (or maybe disproof) that this metric is superior to others.

kornelski commented 6 years ago

I've done that with an earlier version of the library #15

For butteraugli this benchmark is problematic, because it also includes heavily distorted images, and butteraugli claims to only support comparison of very similar images.

sp-1234 commented 6 years ago

oops, I somehow overlooked that…

maybe it could make sense to exclude images which are too dissimilar, and look at the correlation for "good" images.