eth-ait / aitviewer

A set of tools to visualize and interact with sequences of 3D data.
MIT License
497 stars 46 forks source link

Make the AMASS z_up a Node rotation to fix the joint orientation bug. #8

Closed MarilynKeller closed 1 year ago

MarilynKeller commented 1 year ago

In AMASS, the SMPL mesh needs to be rotated to be displayed head up in the viewer. This was discussed in #issue-1321548505.

I found that the way it is done introduces wrong joint orientations for AMASS.

Here is the joints orientation of SMPL given by load_template.py, they are correct: image

Here is the joints orientation of SMPL given by load_AMASS.py, they don't make sense: image

Trying to propagate the global rotation to the joints becomes heavy so I suggest just rotating the SMPL Node instead to have the z axis up.

In this commit, I modify the z_up flag to just set the Node.rotation to the proper rotation matrix, instead of rotating mesh vertices. I also added a z_up option to the PointClouds class to also rotate it in the Load_AMASS.py example.

This fixes AMASS joints orientation: image

Note: The alternative would be to apply the global transformation every time the global_joint_ori are computed, which would be done like this:

        rotated_joints_ori = np.zeros_like(self.joints_ori)
        for joint_idx in range(self.joints_ori.shape[1]):
            rotated_joints_ori[:,joint_idx,:,:] = np.matmul( self.rotation, self.joints_ori[:,joint_idx,:,:]) 
kaufManu commented 1 year ago

Thanks for this PR! Happy to accept it, it's an elegant solution. I assume doing it this way does not re-introduce the offset to the mocap markers that you had originally reported?

MarilynKeller commented 1 year ago

I confirm that doing it this way gives a visualization that superimposes with AMASS mocap data: image