search

ocampor / image-quality

Image quality is an open source software library for Image Quality Assessment (IQA).
Apache License 2.0
394 stars 85 forks source link

Trying to Validate BRISQUE Output #31

Closed blue-j closed 3 years ago

blue-j commented 3 years ago

Describe the bug I can't say I am sure there is a bug. Installation is easy, and working fine, but the BRISQUE scores being output are not matching the ones from here: [https://www.learnopencv.com/image-quality-assessment-brisque/]

which is typically a very reliable source, written by the CEO of OpenCV.

He reports these BRISQUE scores for these images:

original-rotated-image.jpg 26.8286 JPEG2k-compressed.jpg 30.7417 compressed-heavy.jpg 33.0692 noise-version.jpg 79.8751 blur-vision.jpg 72.7044

while your library reports quite different numbers for me:

original-rotated-image.jpg 54.6798 JPEG2k-compressed.jpg 53.9756 compressed-heavy.jpg 55.0188 noise-version.jpg 70.2104 blur-vision.jpg 63.5786

Perhaps I am misunderstanding something, and your codes are not trying to produce the same output? I am a total amateur in the field of IQA. And perhaps it is his code that is in error. His metrics make more sense to me, looking at the imagery.

blur-vision noise-version compressed-heavy JPEG2k-compressed original-rotated-image

blue-j commented 3 years ago

Well, the plot thickens. Output from his script doesn't match the output reported in the blog. I will bring this to his attention. Your numbers are close enough to chalk up to slight variable differences at this point. I suppose this is good news! : )

HIS: blur-vision.jpg 63.9073 compressed-heavy.jpg 54.9763 JPEG2k-compressed.jpg 55.0107 noise-version.jpg 70.0947 original-rotated-image.jpg 54.2116

YOURS: blur-vision.jpg 63.5786 compressed-heavy.jpg 55.0188 JPEG2k-compressed.jpg 53.9756 noise-version.jpg 70.2104 original-rotated-image.jpg 54.6798

ocampor commented 3 years ago

@blue-j Thanks for your thorough response. The code does not come from OpenCV. The authors of the BRISQUE paper published the code in C++ and Matlab. My job was to port the algorithm to python and make it easy to install.

If you want to replicate 100% the results from the paper, I recommend you to take a look to the scaling step of the algorithm. The problem is that the original code uses the resize function of Matlab which is quite difficult to replicate in python.

I hope this information helps.