search

hosang / detection-proposals

Evaluation of detection performance, recall, and repeatability of detection proposal algorithms
http://www.mpi-inf.mpg.de/departments/computer-vision-and-multimodal-computing/research/object-recognition-and-scene-understanding/how-good-are-detection-proposals-really/
GNU General Public License v3.0
141 stars 62 forks source link

Greedy Matching Algorithm #3

Closed ssakhavi closed 9 years ago

ssakhavi commented 9 years ago

In the greedy matching algorithm, I noticed that the algorithm finds the maximum value of a certain proposal and excludes that proposal for another ground-truth.

Isn't it better to solve a assignment problem instead so that you can get the maximum value of both proposals and have you tried it before? I'm guessing the nature of the "Greedy" is to avoid solving such a problem.

hosang commented 9 years ago

I have tried more elaborate matching algorithms, but they were too slow, especially for repeatability that was very noticeable.

An alternative could also be just to use the maximum overlap between each GT box and proposals and not exclude anything. This would make results slightly better, but it didn't change the overall trends when I tried. Using matching or not is just another choice during evaluation.

ssakhavi commented 9 years ago

Thanks for the reply

Actually what caused me to ask the question is that for a method we are developing, when we don't use the greedy algorithm, the performance is high but when we fit it to your code, the results are poorer.

After some analysis, I noticed some flaws in the method which can be the reason of mismatch between our own results and the results from the code.

Thanks again.

On Tue, May 5, 2015 at 2:02 PM, Jan Hosang notifications@github.com wrote:

I have tried more elaborate matching algorithms, but they were too slow, especially for repeatability that was very noticeable.

An alternative could also be just to use the maximum overlap between each GT box and proposals and not exclude anything. This would make results slightly better, but it didn't change the overall trends when I tried. Using matching or not is just another choice during evaluation.

Reply to this email directly or view it on GitHub https://github.com/hosang/detection-proposals/issues/3#issuecomment-98964529 .

Siavash Sakhavi http://www.linkedin.com/in/siavashsakhavi