search

zju3dv / EasyMocap

Make human motion capture easier.
Other
3.72k stars 457 forks source link

Hidden Coordinate Transformations? #311

Closed mizeller closed 1 year ago

mizeller commented 1 year ago

TLDR; Are there any "hidden" transformations applied to the outpu data in the EasyMocap pipeline? If so, could you provide some details/hints on where to look specifically? Thanks in advance!

We were trying to combine the EasyMocap pipeline with Nerfstudio and collected some data using (a) 2 static, paired GoPros for the mocap data and (b) the same GoPro to capture the environment (dynamically).

Afterwards, we computed the camera extrinsics for all frames of (a) and (b) together using COLMAP, to ensure that they are in the same frame of reference. The goal is to place the synthetic human accurately in the reconstructed 3D scene. The 3D reconstruction (using NERFSTUDIO) is working and returns nice point clouds of the environment into which we want to place the human.

We're currently struggling to merge the .ply point cloud with the output .bvh of EasyMocap. We loaded both output files in Blender and it seems like the digital human has a completely different frame of reference. So far we could not find the reason when digging through the code. Any assistance would be much appreciated! :-)

chingswy commented 1 year ago

Hello, this .bvh file is exported by SMPL_maya.fbx, which is totally different with the provided SMPL.pkl.