search

learn2phoenix / cvpr22_vplow_ow

MIT License
12 stars 4 forks source link

Questions about the metrics #3

Closed mcgeedunk closed 2 years ago

mcgeedunk commented 2 years ago

I have a question about the metrics (AuC and mAP). Based on the ICCV paper and the code, it seems that the cluster id is assigned by: 1) calculating the IoU between each bbox and groundtruth, 2) assign grountruth's id to each bbox, 3) voting based on the bboxes in this cluster. Then the purity AuC and mAP is calculated based on the assigned cluster id, right?

I am not sure whether my understanding is right, correct me if I am wrong.

rssaketh commented 2 years ago

Yes, you are correct. This metric does not reflect the over clustering because of which we are also reporting number of clusters for now. Traditional metrics like NMI which penalize over clustering cannot be used in this case as the recall is never 1 (which is not the case for image based clustering methods). Hope that answers your question