search

sha2nkt / moyo_toolkit

This is a repository for download, preprocessing, visualizing, running evaluations on the MOYO dataset.
Other
78 stars 5 forks source link

The twist of smplx arm looks weird. #2

Closed tszhang97 closed 1 year ago

tszhang97 commented 1 year ago

I use the provdied smpl param and visualize with mesh, the twist of arms looks werid

https://github.com/sha2nkt/moyo_toolkit/assets/39046939/a8ec5147-62d0-48cd-8487-3174edb12981

sha2nkt commented 1 year ago

I am not sure how you're obtaining these meshes. Could you try this function and let me know if this problem still exists?

DavidBoja commented 1 year ago

Hi, thanks for sharing your work!

I noticed the same issue of weird hand twists. I downloaded the smpl parameters https://download.is.tue.mpg.de/download.php?domain=moyo&resume=1&sfile=MOYO/20220923_20220926_with_hands/mosh_smpl.zip and tried visualizing the sequence. I was looking at the 220923_yogi_body_hands_03596_Boat_Pose_or_Paripurna_Navasana_-a_stageii_smpl.pkl parameters and noticed the weird hand twists -- I think that joints 22 and 23 of the hands are too twisted 😅.

Here's how the hands look for pose 160 in the sequence (hopefully you can see it):

Screenshot 2023-11-14 at 20 35 15

It can be fixed by setting the parameters for joints 22 and 23 to zero.

I'm using the smplx package, same as in the function smplx_to_mesh you mentioned. This is the abbreviated snippet:

import pickle
with open("220923_yogi_body_hands_03596_Boat_Pose_or_Paripurna_Navasana_-a_stageii_smpl.pkl","rb") as f:
     data = pickle.load(f)

body_pose = torch.from_numpy(data["body_pose"]).float()
global_orient = torch.from_numpy(data["global_orient"]).float()
shape = torch.zeros((1,3))

bm = smplx.create(smpl_path,
                          model_type="SMPL",
                          gender="MALE",
                          num_betas=10,
                          ext='pkl')

out = bm(body_pose=body_pose[160,:].reshape(1,-1),
                    betas=shape, 
                    global_orient=global_orient[160,:].reshape(1,-1))
verts = out.vertices

I haven't played with the create_left_hand_pose, create_right_hand_pose parameters - maybe it has something to do with that?

sha2nkt commented 1 year ago

That's correct. Could you please try initializing the smplx model like in the following function and let me know if you still see this issue? https://github.com/sha2nkt/moyo_toolkit/blob/master/moyo/utils/pressure_map_evaluation.py#L177