Why the keypoints whose coordinates are out of the input shape are reserved when generating the label heatmaps?

[aBlueMouse]

Hi, I find when you generate the heatmaps, you throw away the points whose coordinates are less than 0, while how about those ones whose coordinates are bigger than the input shape, you replace them with the boundary's coordinates: label[i][j << 1 | 1] = min(label[i][j << 1 | 1], ori_size[0] - 1) label[i][j << 1] = min(label[i][j << 1], ori_size[1] - 1) I wonder why you reserve these keypoints and generate the heatmaps different from the keypoints' original location. Thanks!

[chenyilun95]

I think that's a bug we overlooked. But this kind of case may occur rarely and it won't hurt performance directly if it happens when testing.

[lilhope]

When computing loss, use the Tensor 'valid' multiple 'label', so the point out of boundary will not contribute to the loss, it won't hurt the performance.

[aBlueMouse]

I agree with the author's opinion, this situation is so few that it won't make harm, I just want to make sure whether you have some reasons. @chenyilun95 May you explain more details, I'm not quite catch what you mean, how to choose the 'valid label'? @lilhope Thanks a lot!

[lilhope]

Sorry I did not explain it clearly. In practice, I notice that keypoints out of boundary was mainly come from the rotate augmentation. In dataset.py, line 118 ~ 119, the author use a valid variable to label the out of boundary keypoints as Invisible. In network.py, line 128 and line 132, the 'valid' act as the mask to make label[i,:]=0 where valid[i] < 0, so the label of the out of boundary keypoint was all zero.
So I think the out of boundary label won't harm the performance.

[aBlueMouse]

@lilhope I get it, thanks!