[Bug] the dimensions of the keypoints have bug

[zbc-l]

Prerequisite

• [X] I have searched Issues and Discussions but cannot get the expected help.
• [X] The bug has not been fixed in the latest version(https://github.com/open-mmlab/mmpose).

Environment

Reproduces the problem - code sample

def _calc_distances(preds: np.ndarray, gts: np.ndarray, mask: np.ndarray, norm_factor: np.ndarray) -> np.ndarray: """Calculate the normalized distances between preds and target.

Note:
    - instance number: N
    - keypoint number: K
    - keypoint dimension: D (normally, D=2 or D=3)

Args:
    preds (np.ndarray[N, K, D]): Predicted keypoint location.
    gts (np.ndarray[N, K, D]): Groundtruth keypoint location.
    mask (np.ndarray[N, K]): Visibility of the target. False for invisible
        joints, and True for visible. Invisible joints will be ignored for
        accuracy calculation.
    norm_factor (np.ndarray[N, D]): Normalization factor.
        Typical value is heatmap_size.

Returns:
    np.ndarray[K, N]: The normalized distances. \
        If target keypoints are missing, the distance is -1.
"""
N, K, _ = preds.shape
# set mask=0 when norm_factor==0
_mask = mask.copy()
_mask[np.where((norm_factor == 0).sum(1))[0], :] = False

distances = np.full((N, K), -1, dtype=np.float32)
# handle invalid values
norm_factor[np.where(norm_factor <= 0)] = 1e6
distances[_mask] = np.linalg.norm(
     ((preds - gts) / norm_factor[:, None, :])[_mask], axis=-1)
return distances.T

Reproduces the problem - command or script

python tools/test.py /data/rtmo-s_8xb32-600e_coco-640x640.py /tools/work_dirs/rtmo-s_8xb32-600e_coco-640x640/best_coco_AP_epoch_599.pth

Reproduces the problem - error message

12/26 16:09:45 - mmengine - INFO - Evaluating PCKAccuracy (normalized by "bbox_size")... Traceback (most recent call last): File "D:/code/python/MMPose_Tutorials-main/mmpose-dev-1.x/mmpose-dev-1.x/tools/test.py", line 166, in main() File "D:/code/python/MMPose_Tutorials-main/mmpose-dev-1.x/mmpose-dev-1.x/tools/test.py", line 162, in main runner.test() File "C:\Users\lynnd\Anaconda3\envs\openmmlab\lib\site-packages\mmengine\runner\runner.py", line 1823, in test metrics = self.test_loop.run() # type: ignore File "C:\Users\lynnd\Anaconda3\envs\openmmlab\lib\site-packages\mmengine\runner\loops.py", line 438, in run metrics = self.evaluator.evaluate(len(self.dataloader.dataset)) File "C:\Users\lynnd\Anaconda3\envs\openmmlab\lib\site-packages\mmengine\evaluator\evaluator.py", line 79, in evaluate _results = metric.evaluate(size) File "C:\Users\lynnd\Anaconda3\envs\openmmlab\lib\site-packages\mmengine\evaluator\metric.py", line 133, in evaluate _metrics = self.compute_metrics(results) # type: ignore File "D:\code\python\MMPose_Tutorials-main\mmpose-dev-1.x\mmpose-dev-1.x\mmpose\evaluation\metrics\keypoint_2d_metrics.py", line 182, in computemetrics , pck, _ = keypoint_pck_accuracy(pred_coords, gt_coords, mask, File "D:\code\python\MMPose_Tutorials-main\mmpose-dev-1.x\mmpose-dev-1.x\mmpose\evaluation\functional\keypoint_eval.py", line 98, in keypoint_pck_accuracy distances = _calc_distances(pred, gt, mask, norm_factor) File "D:\code\python\MMPose_Tutorials-main\mmpose-dev-1.x\mmpose-dev-1.x\mmpose\evaluation\functional\keypoint_eval.py", line 41, in _calc_distances ((preds - gts) / norm_factor[:, None, :])[_mask], axis=-1) ValueError: operands could not be broadcast together with shapes (112,84,2) (10,84,2)

Additional information

Hello, when I was trying to use rtmo to train my own dataset, I found that the predicted keypoints dimensions were inconsistent when testing nme, which did not occur with rtmpose before.

[Ben-Louis]

PCKAccuracy is only suitable for top-down algorithms as it necessitates a one-to-one match between GTs and predictions. However, RTMO is a one-stage algorithm where its prediction does not directly correspond to GTs. Therefore, it is recommended to use retrieval-based metrics such as mAP to evaluate such algorithms.

[zbc-l]

PCKAccuracy is only suitable for top-down algorithms as it necessitates a one-to-one match between GTs and predictions. However, RTMO is a one-stage algorithm where its prediction does not directly correspond to GTs. Therefore, it is recommended to use retrieval-based metrics such as mAP to evaluate such algorithms.

Thank you for your answer, so epe nme can't use these evaluation indicators, right?

[Ben-Louis]

Yes, they can not be used either