caizhongang / SMPLer-X

Official Code for "SMPLer-X: Scaling Up Expressive Human Pose and Shape Estimation"
https://caizhongang.github.io/projects/SMPLer-X/
Other
1.01k stars 73 forks source link

Hand Rotation problem #52

Open liudongdong1 opened 8 months ago

liudongdong1 commented 8 months ago

Hi, thanks for your great work again. I am utilizing the SMPL-X-H32 model to process an online video featuring extensive hand movements. When attempting to transform the model's hand rotation output into BVH motion files, I've observed that the resulting hand rotations are less than ideal. Despite adjusting the orientations, which improved some frames, inconsistencies persist in the hand orientations across others.

question 1. The degree of finger bending is too slight, and for some frame, the direction of finger bending seems wrong; image image question 2. i modify the direction of the above frame, it seems ok, but other frames seem wrong; image image question3. The facial parameters seem not taken effect. `class FaceRegressor(nn.Module): def init(self, feat_dim=768): super(FaceRegressor, self).init() self.expr_out = make_linear_layers([feat_dim, smpl_x.expr_code_dim], relu_final=False) self.jaw_pose_out = make_linear_layers([feat_dim, 6], relu_final=False)

def forward(self, expr_token, jaw_pose_token):
    expr_param = self.expr_out(expr_token)  # expression parameter
    jaw_pose = self.jaw_pose_out(jaw_pose_token)  # jaw pose parameter
    return expr_param, jaw_pose`
zxx123518 commented 8 months ago

Hello, I am also studying hand Rotation problem at present, could you share the code related to your movement output? Thank you very much!

WingkitChou commented 7 months ago

model hand's initial pose is different, use relaxing hand instead.

Wei-Chen-hub commented 7 months ago

Hi, the problem seems related to flat_hand_mean, smplerx assumes 'flat_hand_mean'=False. Please check this kwarg when creating the smplx model.

TDIT-haha commented 5 months ago

I want to know if this problem was solved later? What is the reason?

TDIT-haha commented 5 months ago

我想知道这个问题后来解决了吗?是什么原因?

后来我解决了问题,可以参考https://github.com/IDEA-Research/OSX/issues/84的问题解决方案,简单的解决方案是我在推理输出中将smplx的full_pose提取出来,这是有经过偏移处理的,再转fbx文件后手就没有问题了

iamthephd commented 4 months ago

我想知道这个问题后来解决了吗?是什么原因?

后来我解决了问题,可以参考https://github.com/IDEA-Research/OSX/issues/84的问题解决方案,简单的解决方案是我在推理输出中将smplx的full_pose提取出来,这是有经过偏移处理的,再转fbx文件后手就没有问题了

Link not working!

sonukiller commented 4 months ago

I want to know if this problem was solved later? What is the reason?

Yes, I was able to solve the problem using flat_hand_mean=False in kwarg while creating SMPLX model.