search

DeepMotionEditing / deep-motion-editing

An end-to-end library for editing and rendering motion of 3D characters with deep learning [SIGGRAPH 2020]
BSD 2-Clause "Simplified" License
1.55k stars 255 forks source link

Is it possible to use my own mocap dataset or videopose3d estimation result for training #75

Closed zitongzhan closed 3 years ago

zitongzhan commented 4 years ago

Hello,

As I glance through the code, I guess the function for converting my own mocap dataset or videopose3d estimation result to .bvh file is not provided. Typical mocap or pose estimation result only contains the xyz coordinate of each joint, but rotations are not provided. Do you have any suggestions for me to achieve this goal? Like, using pose estimation skeletons in retargeting. I see someone have mentioned this before in this post. Thanks in advance!

kfiraberman commented 4 years ago

Converting 3D positions (output of typical pose estimation methods) to rotations is an ill-posed problem. Usually, people use Inverse Kinematics (IK) to perform such a step. The IK provided here can be used as a baseline for such an implementation, but note that there is no such use case in our code since we only use it to correct foot contact positions. I believe that the results you want to achieve (3D positions -> 3D rotations) are depended on the quality of the 3D positions you have.