Inference - some output angles seem wrong

[GuyTevet]

Visualizing the output .npy files of evaluator.py according to README with the provided checkpoints seems like some of the angles (e.g. shoulders) are wrong, maybe flipped. In this clip, green is the original sample from AIST++ and red is MINT inference.

To visualize, I implement the opposite operation described here, then used Blender's SMPLX addon to visualize. Here's my code:

    rotations = mint_data[:seq_len, 9:] # trim first 9 entries according to https://github.com/google-research/mint - mint/tools/preprocessing.py +161
    rotations = rotations.reshape([-1, 3, 3])
    rotations = R.from_matrix(rotations).as_rotvec().reshape([seq_len, (joint_dim-1)//9, 3])
    body_pose = rotations[:, :NUM_SMPLX_BODYJOINTS]  # FIXME - not sure about that (trimming last 3 joints from smpl 24, to smplx 21)

[GuyTevet]

I had a bug in my code, the following is the right way to do it. Closing.

    seq_len, joint_dim = mint_data.shape
    seq_len = SEQ_LEN_LIMIT  # FIXME lose this SEQ_LEN_LIMIT
    rotations = mint_data[:seq_len, 2*9:-2*9]
    # eliminate first 9 entries according to https://github.com/google-research/mint - mint/tools/preprocessing.py +161
    # eliminate the following 9 entries, belong to the pelvis, which isn't included in the 21 body joint
    # eliminate the last two joints of SMPL (L_Hand, R_Hand), which aren't exist in SMPLX (instead, there is a full hand model)
    rotations = rotations.reshape([-1, 3, 3])
    rotations = R.from_matrix(rotations).as_rotvec().reshape([seq_len, (joint_dim//9)-4, 3])
    assert rotations.shape[1] == NUM_SMPLX_BODYJOINTS

[birdflies]

@GuyTevet Hi, would you like to share the total code? Thanks~

[birdflies]

@GuyTevet
smpl_poses = R.from_rotvec( smpl_poses.reshape(-1, 3)).as_matrix().reshape(smpl_poses.shape[0], -1) smpl_motion = np.concatenate([smpl_trans, smpl_poses], axis=-1) you mean mint_data = smpl_motion？

[birdflies]

I know what you mean. Thanks.

NUM_SMPLX_BODYJOINTS = 21 smlp_path = "./outputs/gJS_sBM_c01_d03_mJS3_ch02_mJS3.npy" mint_data = np.load(smlp_path)

[h310558606]

Could you tell me after getting the rotations, how to visualize it in the Blender? Thanks a lot!