annotations files not complete?

[FightForCS]

Hi, I want to evalute your pretrained model. I download "minival annotation" file and "Person detection results in COCO Minival" file from your link, but it seems that some images are missing. Here are my test codes:

val = json.load(open('./MSCOCO/annotations/person_keypoints_minival2014.json', 'r'))
train = json.load(open('./MSCOCO/annotations/person_keypoints_trainvalminusminival2014.json', 'r'))
det = json.load(open('./dets/person_detection_minival411_human553.json', 'r'))

det_image_ids = set([i['image_id'] for i in det])
val_image_ids = set([i['id'] for i in val['images']])
val_annot_ids = set([i['id'] for i in val['annotations']])
val_annot_img_ids = set([i['image_id'] for i in val['annotations']])

print(len(val_image_ids & val_annot_img_ids))
print(len(det_image_ids & val_annot_img_ids))

output is

2693
2692

it seems that many annotatated images(ground truth) do not exist in your detection result file and your minival. What is the reason? Thanks.

[chenyilun95]

There is no person in the rest of images. COCO minival dataset has 5000 images, which includes about 2000+ images with persons.

[FightForCS]

@chenyilun95 Thanks for your qucik reply, then why there are 4581 unique images containing person in your 'person_detection_minival411_human553.json' file? Is it becasue you have not apply NMS to the final detection results?

[chenyilun95]

Softnms has been applied in the detection result. We filter images whose person score is very low.

[FightForCS]

I see, so, we only need to filter low score person and run pretrained CPN on those filtered images?

[chenyilun95]

It’s OK. You should use nms to filter the detection boxes anyway. The code is without nms by default.

[FightForCS]

thx