search

mks0601 / I2L-MeshNet_RELEASE

Official PyTorch implementation of "I2L-MeshNet: Image-to-Lixel Prediction Network for Accurate 3D Human Pose and Mesh Estimation from a Single RGB Image", ECCV 2020
MIT License
710 stars 130 forks source link

Should I consider 1d heatmap over 3d heatmap for non-mesh human pose estimation? #41

Closed daun-io closed 3 years ago

daun-io commented 3 years ago

Thanks for the great paper, I read that your paper suggests 1d heatmap over 3d heatmap since number of mesh vertex is much larger and therefore predicting 3d heatmaps of all mesh vertices become computationally infeasible. However what if I only need is the PoseNet and not the MeshNet and therefore I'm only predicting relatively less number of joints (such as under 20) is there any further advantage by using 1d heatmap instead of 3d heatmap? image or what if I want to train on 2d human pose with 1d heatmap coordinate representation, can it be better than 2d heatmap in terms of performance or maybe gpu memory? I would like to know your insights.

mks0601 commented 3 years ago

If you consider only small number of joints (about 20), then I think using 2D or 3D heatmap would be better. You can see comparison between 1D/2D/3D heatmap in Table 2 of the paper. However, if you consider whole-body pose estimation, there are more than 100 keypoints, which can be a good place to apply lixel-based 1D heatmap.