search

mks0601 / I2L-MeshNet_RELEASE

Official PyTorch implementation of "I2L-MeshNet: Image-to-Lixel Prediction Network for Accurate 3D Human Pose and Mesh Estimation from a Single RGB Image", ECCV 2020
MIT License
710 stars 130 forks source link

Another mesh for visualization #35

Closed Zrrr1997 closed 3 years ago

Zrrr1997 commented 3 years ago

Hello,

first of all thanks for the great work! I was able to run the code without any problems and the pose estimation seems to be working fine.

Is it possible to use another mesh for the visualization - for example: a humanoid cartoon character? And what would be the steps to make this possible? I suppose I could still use the human mesh for the estimation and somehow transfer the estimated pose to another mesh via keypoints, but would that be a good idea? I would appreciate some pointers in the right direction!

mks0601 commented 3 years ago

As my network is trained on human images, it won't work well on cartoon images :( However, if you can extract 2D pose from the cartoon character, you can use my another work Pose2Mesh (paper: https://arxiv.org/abs/2008.09047, code: https://github.com/hongsukchoi/Pose2Mesh_RELEASE). It predicts 3D mesh from 2D pose.

Zrrr1997 commented 3 years ago

I dont want to use cartoon images as input. What I meant was to use another 3D model only for the rendering in the end. The problem is how to transfer the estimated pose for the SMPL mesh to another one? Is that possible or reasonable?

P.S. Thank you for the fast reponse and the paper :)

mks0601 commented 3 years ago

Ah sorry for the misunderstanding. For that, you can use SMPL pose parameter predicted in param stage of this repo. If the pose of another 3D model is defined in the same way of that of SMPL, you can just forward the predicted SMPL pose parameter to the another 3D model layer.

Zrrr1997 commented 3 years ago

Thank you! I think I can just use a texture for the SMPL model, but I do not know how to add that to your code and whether there are any ready-to-use textures online... However that is another issue so I am closing this one!