search

miraymen / pix2surf

Learning to Transfer Texture from Clothing Images to 3D Humans, CVPR 2020
Other
342 stars 62 forks source link

Generating Surface from SMPL Shape and Pose Values #3

Open sahil00199 opened 4 years ago

sahil00199 commented 4 years ago

Thank you for providing an easy to run demo of your work. However, I am trying to generate the clothed 3D models using the SMPL shape and pose parameters as input (as described in the paper). For this, the .obj files that are provided for the demo need to be generated from the SMPL parameters. I am unable to correctly obtain and model the garment-specific deformation (Denoted by $D$ in Section 3.1 of the manuscript). I tried changing the v_template and v_personal values in the Smpl model (defined in prep_data/utils/ch_smpl.py) for the vertices specific to the garment in question to the values given in the assets/init_meshes directory. This leads to a garment surface with accurate texture and correct pose and shape deformations but when overlaid on the unclothed body, the garment disappears under the body at several points. What is the correct way to model the cloth deformation and obtain the 3D (pose and shape dependent) coordinates for each garment?

miraymen commented 4 years ago

This is a well known issue with garment and body interpenetration. Please see this paper and its supplementary for a discussion.

Please see this repository for a possible solution.

sahil00199 commented 4 years ago

Thank you for your response. I did have a look at the resources you pointed to.

If I understand correctly, there is a displacement that is added to the unposed body vertices to obtain the unposed garment vertices which then undergo the usual skinning transformation.

It is, however, not clear to me where can I obtain these displacements. The MGN repository you pointed to has the unposed garment vertices (in the target shape) as part of the dataset whose displacement with respect to an unclothed body with the same shape is taken to be the garment displacement. While this can be used to obtain displacements for pants and shorts, this is not possible for shirts since I don't think the shirts garment type here maps to any of the 5 garment types in the MGN repository (TShirtNoCoat has 7702 vertices, ShirtNoCoat has 9273 vertices whereas the shirts garment type in this repo has 8168 vertices). It would, hence, be extremely useful if the garment displacements are available.

melih-unsal commented 3 years ago

hi @sahil00199, how did you change the pose and shape parameters? The demo code needs obj file instead of smpl parameters.