keras-team / keras-cv

Industry-strength Computer Vision workflows with Keras
Other
1.01k stars 331 forks source link

COCO metrics only evaluated on image_ids with predictions #2504

Open digital-nomad-cheng opened 3 weeks ago

digital-nomad-cheng commented 3 weeks ago

https://github.com/keras-team/keras-cv/blob/94b0a551d03ee9de3d81663e2c1a680cb113f7f1/keras_cv/src/metrics/coco/pycoco_wrapper.py#L223-L227

The pycoco_wrapper only evaluates on images where there is a prediction, and leave out image_ids where there is not prediction. This is a serious bug if I understand it correctly. The coco metrics should be evaluated on all the images. Correct way to do this should be below, without filtering image_ids.

    coco_dt = coco_gt.loadRes(predictions=coco_predictions)
    coco_eval = COCOeval(coco_gt, coco_dt, iouType="bbox")