search

cocodataset / cocoapi

COCO API - Dataset @ http://cocodataset.org/
Other
6.07k stars 3.75k forks source link

Questions about cocoeval areaRng #289

Open ashnair1 opened 5 years ago

ashnair1 commented 5 years ago

The default area range for object detection evaluation is set as:

[[0 ** 2, 1e5 ** 2],   # all
[0 ** 2, 32 ** 2],      # small 
[32 ** 2, 96 ** 2],    # medium
[96 ** 2, 1e5 ** 2]]  #  large

I had a couple of questions regarding this:

  1. How were these limits chosen? I mean how was it decided that objects with area in between 0 and 32^2 will be categorised as small, objects with area in between 32^2 and 96^2 will be categorised as medium etc. This would most likely have been done based on some canonical image size. If so, what was that image size?

  2. Why is the upper limit of large object areas set to a really high value of 1e5^2? I believe it is to capture any objects larger than 96^2 but wouldn't that hurt the AP score? For example, in one of my projects I was using the default ranges for evaluating my object detector. Once I reduced the upper limit from 1e5^2 to 256^2, I got a boost in my AP score. This was most likely due to me not having any objects with area above 256^2. I am not sure about this finding, so kindly correct me if I'm mistaken.

tachikoma777 commented 5 years ago

Have you figure out the image size which its based on? I have similar questions.

ashnair1 commented 5 years ago

I haven't. But I think it doesn't matter. The metrics were calculated with respect to the test data so the area ranges might have been chosen so as to partition the objects appropriately.

For example, consider the 'small' area range - 0^2 to 32^2. The upper limit of 32^2 was probably selected so as to include all small objects in the coco test data.

tachikoma777 commented 5 years ago

Do you know how to change cocoapi to set threshold for small/medium/large AP evaluation? I create an issue here https://github.com/cocodataset/cocoapi/issues/325

Dicko87 commented 3 years ago

Hi guys, does anybody know how to get the total number of true positives, false positives, true negatives and false negatives using cocoeval?