search

mkocabas / pose-residual-network

Code for the Pose Residual Network introduced in 'MultiPoseNet: Fast Multi-Person Pose Estimation using Pose Residual Network (ECCV 2018)' paper
https://arxiv.org/abs/1807.04067
346 stars 92 forks source link

Weird evaluation code in evaluate.py #22

Open jackyjsy opened 5 years ago

jackyjsy commented 5 years ago

Start from Line #140 in evaluation.py, it seems to me that you are using the groundtruth keypoints to obtain you keypoint estimation, which should not happen when you evaluate your PRN network. This issue makes it an improper evaluation.

A brief summary is that, first, you generate indexes from the old_weights_bbox (groundtruth). Then. you seem to utilize the index to place a window around that groundtruth position and calculate your estimated scores. Then the output keypoints are summarized from the scores.

I found the issue in the PyTorch version. There was another guy found the same issue Issue #17. Then I came here and found the same issue in this Keras version. @mkocabas please respond to our concerns. Thanks!

abster95 commented 5 years ago

@jackyjsy Seems like they're presenting the Both GT part from the Table3 of their work. Here they show how well their PRN network would assign keypoints to person instances if the keypoint and the person segmentation subnet gave perfect outputs.

I feel like @mkocabas and others should have been a bit more explicit with their intent for this repository.