search

sha2nkt / moyo_toolkit

This is a repository for download, preprocessing, visualizing, running evaluations on the MOYO dataset.
Other
76 stars 5 forks source link

The twist of smplx arm looks weird. #2

Closed tszhang97 closed 1 year ago

tszhang97 commented 1 year ago

I use the provdied smpl param and visualize with mesh, the twist of arms looks werid

https://github.com/sha2nkt/moyo_toolkit/assets/39046939/a8ec5147-62d0-48cd-8487-3174edb12981

sha2nkt commented 1 year ago

I am not sure how you're obtaining these meshes. Could you try this function and let me know if this problem still exists?

DavidBoja commented 12 months ago

Hi, thanks for sharing your work!

I noticed the same issue of weird hand twists. I downloaded the smpl parameters https://download.is.tue.mpg.de/download.php?domain=moyo&resume=1&sfile=MOYO/20220923_20220926_with_hands/mosh_smpl.zip and tried visualizing the sequence. I was looking at the 220923_yogi_body_hands_03596_Boat_Pose_or_Paripurna_Navasana_-a_stageii_smpl.pkl parameters and noticed the weird hand twists -- I think that joints 22 and 23 of the hands are too twisted 😅.

Here's how the hands look for pose 160 in the sequence (hopefully you can see it):

Screenshot 2023-11-14 at 20 35 15

It can be fixed by setting the parameters for joints 22 and 23 to zero.

I'm using the smplx package, same as in the function smplx_to_mesh you mentioned. This is the abbreviated snippet:

import pickle
with open("220923_yogi_body_hands_03596_Boat_Pose_or_Paripurna_Navasana_-a_stageii_smpl.pkl","rb") as f:
     data = pickle.load(f)

body_pose = torch.from_numpy(data["body_pose"]).float()
global_orient = torch.from_numpy(data["global_orient"]).float()
shape = torch.zeros((1,3))

bm = smplx.create(smpl_path,
                          model_type="SMPL",
                          gender="MALE",
                          num_betas=10,
                          ext='pkl')

out = bm(body_pose=body_pose[160,:].reshape(1,-1),
                    betas=shape, 
                    global_orient=global_orient[160,:].reshape(1,-1))
verts = out.vertices

I haven't played with the create_left_hand_pose, create_right_hand_pose parameters - maybe it has something to do with that?

sha2nkt commented 12 months ago

That's correct. Could you please try initializing the smplx model like in the following function and let me know if you still see this issue? https://github.com/sha2nkt/moyo_toolkit/blob/master/moyo/utils/pressure_map_evaluation.py#L177