pavancm / Stitched-Image-Quality-Evaluator

Official implementation of Stitched Image Quality evaluator (SIQE)
17 stars 4 forks source link

Very long runtime #1

Open DiTo97 opened 3 years ago

DiTo97 commented 3 years ago

I followed all the guidelines in the README to install the necessary toolboxes, changed the paths accordingly, and launched the example.m script using the 5 sample images provided in the images folder, but I've been waiting for a quality score for more than 45 minutes now while it is still halfway through it. I've seen the comment about the inclusion of VLFeat to remove overlapping regions possibly being quite time consuming, but is it really 45-minute-long time consuming?

Feature extraction seems indeed to be quite the bottleneck. Could this warning be the key?

You should compile the MEX version of "pointOp.c", found in the MEX subdirectory of matlabPyrTools, and put it in your MATLAB path. It is MUCH faster.

MATLAB version: R2020b

pavancm commented 3 years ago

The warning indicates that mex version is not compiled and the program is using normal matlab scripts.If you are unfamiliar with mex, it basically converts certain C scripts into matlab executables making it run quite fast. It is advisable to use mex version rather than pure matlab version in order to obtain faster performance. Another sanity check you can do is to reduce the size of the images in the example.m script and then run the file. This can ensure whether everything is in order

DiTo97 commented 3 years ago

The warning indicates that mex version is not compiled and the program is using normal matlab scripts.If you are unfamiliar with mex, it basically converts certain C scripts into matlab executables making it run quite fast. It is advisable to use mex version rather than pure matlab version in order to obtain faster performance. Another sanity check you can do is to reduce the size of the images in the example.m script and then run the file. This can ensure whether everything is in order

I was just about to edit the original question.

I already compiled the C scripts with MEX, but still the _extract_featconstituent and the _extract_featstitched take forever to run, especially when computing the Bivariate distribution. Now that I do have the MEX executables it is certainly faster than before, yet in the realm of tens of minutes to run (26 mins the last time I tried), and nowhere close to the 1.0534 secs that is reported in Table XIII of the paper. I may reduce the time lag a little bit by introducing two batch jobs to extract the features from the constituents and the stitched image in parallel, but have you ever experienced something like while testing?

I will definitely try to reduce the size of the images a bit, and see the speedup that it will have this way.

Also, can you confirm that the output of the SVR is between -1 and 1?

FYI, I have a Dell laptop with Windows 10 Pro - 20H2, 32GB of RAM and 12GB of GPU.

Edit: What is your runtime with the samples from the images folder?

pavancm commented 3 years ago

Note that the time reported in the Table XIII of the paper is the time required for extracting features from a patch of size 100x100. Using very high resolution images can result in large number of patches, thus increasing processing time. Note that this reported time is just for feature extraction and does not consider the time required for removing overlapped regions.

The input to SVR is normalized between -1 and 1. The output is a quality score that can range from 0 (worst quality) to 100 (best quality)

I have a runtime of around 10-12 min for _extract_featstitched . This is obtained using slightly older version of Matlab (I think R2017 or R2018), so I am not sure the same will be valid for the latest version. Also by default the code uses parallel for loop during feature extraction, but in some machines it has been observed to reduce performance, so you might want to check whether parallel loop is actually helping your cause.

dsilvapiedra commented 6 months ago

Hi, I´m getting the same MEX warning. How do you compile and use pointOp.c? Also, it'd be nice to know what should be the quality_score for the example image. Thank you in advance

LHY12399 commented 2 months ago

Hi, I´m getting the same MEX warning. How do you compile and use pointOp.c? Also, it'd be nice to know what should be the quality_score for the example image. Thank you in advance

I just get 54.2,and can you tell me your score for the example ? Thank you.