search

hongsukchoi / Pose2Mesh_RELEASE

Official Pytorch implementation of "Pose2Mesh: Graph Convolutional Network for 3D Human Pose and Mesh Recovery from a 2D Human Pose", ECCV 2020
MIT License
666 stars 69 forks source link

Question regarding augmentation #25

Closed Shubhendu-Jena closed 3 years ago

Shubhendu-Jena commented 3 years ago

Hi,

I noticed something peculiar in the augmentation part and hence, I'd be graetful if you could clarify this for me. Basically, in your dataset class (for example in Human36M/dataset.py), when you do rotation augmentation for the image (variable rot in degrees in your code), you're not doing the same to your mesh data (mesh_cam on line 352). Instead, only the 2D and 3D joints (joint_img and smpl_joint_cam) are augmented as follows :

"joint_img, trans = j2d_processing(joint_img.copy(), (cfg.MODEL.input_shape[1], cfg.MODEL.input_shape[0]), bbox, rot, 0, None)" on line 368 "joint_cam = j3d_processing(joint_cam, rot, flip, self.flip_pairs)" on line 373

Shouldn't the augmentation be also done for mesh_cam?

Thanks

hongsukchoi commented 3 years ago

Hi, it’s a good point.

Actually I don’t do any augmentation to train Pose2Mesh. Only do flip augmentation for training PoseNet. So there’s no aug code for mesh_cam.

Shubhendu-Jena commented 3 years ago

Ah, got it! However, for the "Joint coordinate loss", the joints are regressed from the mesh. So, if the mesh isn't augmented, wouldn't it result in the wrong joints being regressed and used for computing the "Joint coordinate loss"? (line 131 Pose2Mesh_RELEASE/lib/core/base.py)

hongsukchoi commented 3 years ago

Good! The flip aug is only done in pre-training PoseNet. When training Pose2Mesh in an end-to-end manner, there is no augmentation. So the joints are also never flipped.

Shubhendu-Jena commented 3 years ago

I see. Was this done on purpose? Does it result in better convergence?

hongsukchoi commented 3 years ago

Yes, the model gave the best accuracy without augmentation. But you can try the augmentation referring to the dataloader codes in https://github.com/mks0601/I2L-MeshNet_RELEASE.

Shubhendu-Jena commented 3 years ago

Ah, okay. Thanks a lot for the responses. They helped me a lot in understanding your work better. I'll take a look at https://github.com/mks0601/I2L-MeshNet_RELEASE to try the augmentation :) Closing issue now.