train Freihand problem

[yongyongdown]

Hi, Thank you for your excellent research.

I was trying to train on freihand data. The config is as follows.

trainset_3d = ['FreiHAND'] trainset_2d = [] testset = 'FreiHAND'

And this error appears.

loss.py", line 13, in forward loss = torch.abs(coord_out - coord_gt) * valid RuntimeError: The size of tensor a (16) must match the size of tensor b (21) at non-singleton dimension 1

Since freihand has 21 keypoints and mano has 16 keypoints, it seems natural that an error would occur.

I tried changing orig_joint_num to 21 in human models. Then this error appears

body_models.py", line 1614, in forward full_pose += self.pose_mean RuntimeError: The size of tensor a (63) must match the size of tensor b (48) at non-singleton dimension 1

How do you solve this problem?

Let me add something a little strange...

In the MANO class of body models.py I printed a few variables, and at first it appeared as shown below

global_orient : torch.Size([1, 3]) hand_pose : torch.Size([1, 45]) full_pose : torch.Size([1, 48]) pose_mean : torch.Size([48])

but then suddenly it appeared like this

mano_hand_pose : torch.Size([8, 60]) global_orient : torch.Size([8, 3]) hand_pose : torch.Size([8, 60]) full_pose : torch.Size([8, 63]) pose_mean : torch.Size([48])

and the error I described at "I tried changing orig_joint_num to 21 in human models." appeared.

It seems as if batch (8) was not applied at first and then suddenly works. Is this normal?

thanks.

[mks0601]

HI, did you use the Pose2Pose branch?

[yongyongdown]

yes. pose2pose branch

[mks0601]

joint_regressor (https://github.com/mks0601/Hand4Whole_RELEASE/blob/dd4ddf92264f79d29a646613bdfc50aaf63b6971/common/utils/human_models.py#L108) outputs 21 hand keypoints. This line uses the joint_regressor (https://github.com/mks0601/Hand4Whole_RELEASE/blob/dd4ddf92264f79d29a646613bdfc50aaf63b6971/main/model.py#L90). DId you change any of them?

[yongyongdown]

I touched a lot of code here and there.. but I didn't touch those lines.

[mks0601]

please check dimension of your joint_cam. If it has only 16 keypints, please check the joint_regressor.

[yongyongdown]

Yes joint_cam has 16 keypoints. But I don't think I touched the regressor. Come to think of it, I think joint_cam was always 16 during my experiments. Shouldn't it be 16?

[mks0601]

please check the dimension of the joint_regressor. If the regressor has 21 keypoints and if it is used for the joint_cam, there is no reason for joint_cam to have only 16 keypoints.

[yongyongdown]

Found it!

For some reason I

joint_cam = torch.bmm(torch.from_numpy(mano.joint_regressor).cuda()[None,:,:].repeat(batch_size,1,1), mesh_cam)

I changed this part.

I hadn't even thought about it, but thanks to you I found it. Thank you so much. Training seems to be going well. Thank you for your help.