Score for AP calculation

[lxa9867]

Hi,

Thank you for sharing such a work. I would like to ask what is used as score to compute AP_mask when just using MaskCut as reported in Table~7.

[frank-xwang]

Hi! Since MaskCut does not provide a score, we will need to use all the masks available to evaluate the AP_mask.

[lxa9867]

Is it equivalent to set all the scores to 1? I don't quite understand what do you mean by using all the masks available, can you please explain more?

[frank-xwang]

Certainly! As long as you do not set a threshold when evaluating AP_mask using COCO's official evaluation metric, the value used for the confidence score (whether it is 0.1, 1.0, or 0.000001) should not have any influence on the AP. You can check this blog, which gives a pretty good explanation of the evaluation metric.

[lxa9867]

Hi,

Thank you for your patient answers. Can you please let me know, without a score or a series of threshold, how to calculate the P-R curve? I am quite confused and still believe you cannot calculate AP if you don't have a score. Thank you

[frank-xwang]

As mentioned earlier, although the pseudo-masks produced by MaskCut do not have a confidence score (and we cannot get a confidence score), we can still assign a confidence score of 1.0 to all masks. The AP (average precision) is calculated using interpolated precision-recall, as explained in the cocoapi documentation. By assigning the same confidence score to all masks, we get an interpolated P-R curve that is a horizontal line. To understand the calculation method in more detail, you can refer to the cocoapi, which is the same tool we used to obtain the AP_mask metric in table 7.

[lxa9867]

Thank you so much for the detailed explanation! I understand it.