search

akanazawa / hmr

Project page for End-to-end Recovery of Human Shape and Pose
Other
1.55k stars 391 forks source link

Cant create a female mesh #37

Closed sree295 closed 6 years ago

sree295 commented 6 years ago

I have tried looking for shape space as told by you in https://github.com/akanazawa/hmr/issues/15 but I couldn't understand. Would you please elaborate on how to use female SMPL here?

akanazawa commented 6 years ago

Hi,

Basically the model is not trained for the female SMPL shape space. The 10D beta values mean different things in the female, male, and the neutral model. This code base is trained with neutral model, therefore you can't just copy over the beta of neutral model and use it for female.

If you just want a more female shape, which sounded like what the person was asking for, the pose is the same in all gendered models, so you can transfer the pose but use the default female shape.

So basically

  1. get the pose using HMR, throw away the shape.
  2. load female SMPL model (download it from the SMPL site, please read their README, use load_model function with the female model's pkl file.), set the pose to be the pose you got from HMR, set your shape however you want (or just use the default 0.) and that will give you a more female shape.

Again these approaches will not give you the female or male body shape in the picture, because the model was not trained on gendered models.

If you want to train a model that works with female or male model, you would have to train the model again, but only with female/male images or either have to have the gender annotated for each training image which is not very practical. This is why I used the neutral model.

Hope this helps,

Angjoo

akanazawa commented 6 years ago

Actually, if you really want, it would be a good idea to use the pose returned by HMR as the initialization to run the code for SMPLify: http://smplify.is.tue.mpg.de/

In SMPLify you can select the gender, and optimize for the pose and the shape of the person in the image, given that you provide detected 2D keypoints.

Starting from HMR's pose will be a good initialization for the optimization.

Best,

Angjoo

SergeiGorbatiuk commented 5 years ago

@akanazawa , I did not quite get what you mean by initializing SMPLify by HMR's pose. I know that SMPLify consumes output from DeepCut on a picture, a picture itself, and can be told what gender to use. What exactly do you call an HMR pose and won't there be a mismatch in different joint systems. Or am I missing something? :)

akanazawa commented 5 years ago

Hi,

So SMPLify is an iterative optimization approach, it requires some initialization to start the optimization. The idea is to use the predicted pose and shape as initialization. So HMR returns these parameters, so you can change SMPLify so that it starts from that. Here you want to use the gendered model, so you can use just the pose.

Hope this clears things up!

Best,

Angjoo

SergeiGorbatiuk commented 5 years ago

@akanazawa, thanks a lot, I will try this out;)