Keypoint Metrics - Matching predictions and ground truth

[fabianwindbacher]

I am working on Human Pose Estimation using the new keypoint annotations of this dataset.

I am seeing that the metrics expect the ground-truth keypoints and the predicted keypoints to match, i.e. we supply the KeypointsTensors in the same shape, in the same order.

I am confused as to how I match my predictions to the ground-truth, before feeding them to the metric functions. Particularly:

• My assumption would be that I have to use some sort of best match based on OKS. Is this true?
• How do I handle cases where I predict more persons than the GT? Or vice versa, how do I handle cases where I predict less people than the GT has.

For example: if I have more predictions than GT annotations, and I do some sort of best match calculation - discarding the worst predictions - the false positives would not be penalized, right?

[alexgorban]

Hi, sorry for the long overdue response.

Thanks for pointing this out. Out current set of metrics is focused on measuring quality of keypoint detection only, not object detection, so they expect objects to be already matched. We will consider adding metrics which measure both stages in the future. Meanwhile:

1. You can use objects bounding boxes to match objects or use ground truth object bounding boxes.
2. Since our ground truth has labels only for a subset of objects in the scene I suggest to exclude predicted objects without corresponding ground truth match from metric computation.

Please feel free to re-open this issue if you have further questions.