Retargeting : How to transfer the SMPL motion data to DeepMimic to RL?

[Zju-George]

I've build both models( SMPL and DeepMimic) in Unity, and restore them in T-pose. But the T-pose is very different, especially in arms. Besides, their joint count is different( 24 vs 14), So in SFV paper, how to transfer the data from SMPL into DeepMimic?

Any idea would be very helpful!

[akanazawa]

I think we just used the subset of SMPL joints that corresponds with the joints that the DeepMimic model has.

[nghorbani]

@Zju-George did you found the answer to this question?

[lucasjinreal]

Even though with subset skeleton, there still has some issue SMPL's bone rotate is different.

[akanazawa]

ah yes, you need these sets of files, and i just pushed this to the repo! (note its an old research code, someone has ran it but no guarantees, good luck!) https://github.com/akanazawa/motion_reconstruction/tree/master/hmr2bvh

[lucasjinreal]

@akanazawa hi, thanks for pointing this code. Wanna ask does this code rotates SMPL's bone to general way ? Does it able to driven Mixamo model without chaning mixamo model to same as smpl?

[akanazawa]

hi not sure what you mean by general way. it rotates SMPL's bone to match the deepmimic skeleton structure,

On Sat, Sep 17, 2022 at 1:54 AM JinTian @.***> wrote:

@akanazawa https://github.com/akanazawa hi, thanks for pointing this code. Wanna ask does this code rotates SMPL's bone to general way ?

— Reply to this email directly, view it on GitHub https://github.com/akanazawa/motion_reconstruction/issues/4#issuecomment-1250032139, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAHBXWG62XEYCC3DSNECSG3V6WBOBANCNFSM4II3HDHQ . You are receiving this because you were mentioned.Message ID: @.***>

-- Angjoo Kanazawa Assistant Professor Department of Electrical Engineering and Computer Sciences University of California, Berkeley https://people.eecs.berkeley.edu/~kanazawa/ She/Her

[lucasjinreal]

@akanazawa Hi, the SMPL skeleton bone orientation is up, while mixamo most character bone are along with it's child.

this caused SMPL output quaternion can not directly applied to target skeleton.

I am not mean rotation SMPL's bone, but SMPL model outputs pose (rotation matrix or quaternion)

[akanazawa]

yup you'd have to figure out the correct orientation to match the mixamo character!

Best,

A

[lucasjinreal]

@akanazawa do u know is there a way to do it?>