How to fuse MANO pose and SMPL pose into SMPL-X or SMPL+H?

[ZhengdiYu]

https://github.com/vchoutas/smplx/issues/63#issuecomment-780128592 This comment said:

SMPL-H and SMPL have the same pose and shape params (unlike SMPL-X and SMPL). So you can simply ignore the hand pose params and use the other pose and shape parameters in SMPL. It is that simple.

My problem is: I now have a set of MANO parameter, and a set of SMPL parameter, what is the quickest way to use them in SMPL-X?

1. Does this mean that we can directly feed MANO and SMPL's θ/β into SMPL+H to get correct results?
2. Do I need to first convert SMPL to SMPL-X or should I first use the parameter to get a SMPL+H, and then from SMPL+H to SMPL-X

I tried to directly input MANO's theta into the SMPLX model, it outputs like this:

I've also tried to replace hand vertices by using the mano index, but the hand is a flat plane, is it due to the flat-hand_mean? But I can't find it in smplx

[vchoutas]

Yes, you can directly feed MANO and SMPL pose, shape to SMPL+H.

I suspect that your problem has to do with the flat_hand_mean option. The MANO poses use flat_hand_mean=True and you use flat_hand_mean=False in SMPL+H, or vice-versa.

flat_hand_mean controls the rest pose of the hand. If it true, then you simply have an open "flat" hand. If it is False, then the rest pose will be the mean pose computed from the MANO training set.

[ZhengdiYu]

Yes, you can directly feed MANO and SMPL pose, shape to SMPL+H.

I suspect that your problem has to do with the flat_hand_mean option. The MANO poses use flat_hand_mean=True and you use flat_hand_mean=False in SMPL+H, or vice-versa.

flat_hand_mean controls the rest pose of the hand. If it true, then you simply have an open "flat" hand. If it is False, then the rest pose will be the mean pose computed from the MANO training set.

Yes, Thank you. That's extactly the point. I have set flat_hand_mean correctly now. And it looks reasonable now. But actually I was feeding MANO into SMPL-X instead of SMPL+H, it still looks reasonable.

So, one more question, can I feed MANO into SMPL-X？ I now understand that I can't SMPL into SMPL-X, but what about MANO?

[buaacyw]

Hi @ZhengdiYu! I also have the same problem. I found that SMPL+H only has 10 betas dimensions. However, each hand of MANO and SMPL has 10 betas dimensions. So we have 30 betas dimensions with SMPL + 2 MANO hands. How do you input the betas to SMPL+H? I guess maybe you only input the SMPL 10 betas dimensions to the SMPL+H? But in my data, the MANO hand prediction result is far more better than SMPL. So I think it will be better to use the hand shape produced by MANO. Is there a method? Thanks!

[MichaelJBlack]

You can download up to 300 shape components for SMPL+H. AMASS, for example, ships with 16. When using SMPL+H, you don't also use MANO's own shape space. The hand shape in SMPL+H is captured by the full body shape space (ie, your 10 betas). When using MANO by itself, it has its own shape space but this hand-specific space is not related to the SMPL+H space. You can use the hand pose parameters predicted using your MANO method directly with SMPL-H but not the shape parameters.

[Whiplash-18]

You can download up to 300 shape components for SMPL+H. AMASS, for example, ships with 16. When using SMPL+H, you don't also use MANO's own shape space. The hand shape in SMPL+H is captured by the full body shape space (ie, your 10 betas). When using MANO by itself, it has its own shape space but this hand-specific space is not related to the SMPL+H space. You can use the hand pose parameters predicted using your MANO method directly with SMPL-H but not the shape parameters.

Hello, if I want use the SMPL of 300 principle component , how can I use this version, change the dims of beta and use the the v1.1.0 model. Is it work? Looking for your reply! Thanks!