The pycoco_wrapper only evaluates on images where there is a prediction, and leave out image_ids where there is not prediction.
This is a serious bug if I understand it correctly. The coco metrics should be evaluated on all the images.
Correct way to do this should be below, without filtering image_ids.
https://github.com/keras-team/keras-cv/blob/94b0a551d03ee9de3d81663e2c1a680cb113f7f1/keras_cv/src/metrics/coco/pycoco_wrapper.py#L223-L227
The pycoco_wrapper only evaluates on images where there is a prediction, and leave out image_ids where there is not prediction. This is a serious bug if I understand it correctly. The coco metrics should be evaluated on all the images. Correct way to do this should be below, without filtering image_ids.