Why DEX_YCB .json annotations are different from the original dataset's .npz file

[karta2155802]

I'm working on the two datasets. I found that mano_param : pose parameters in json file are different from the original dataset's pose parameters. So I wonder what makes them different? And how did you generate the pose parameters in the annotation json file?

[namepllet]

The pose_m parameter in original dataset's annotation file is PCA coefficients.

I changed it to axis angle representation.

[karta2155802]

Could you provide the script for changing PCA coefficiets to axis angle representation? Thank you verty mush.

[namepllet]

Here are core lines we used.

from manopth.manolayer import ManoLayer
manolayer_left = ManoLayer(mano_root=osp.join(cfg.mano_path, 'mano', 'models'), flat_hand_mean=False, use_pca=True, side='left', ncomps=45)
manolayer_right = ManoLayer(mano_root=osp.join(cfg.mano_path, 'mano', 'models'), flat_hand_mean=False, use_pca=True, side='right', ncomps=45)

mano_pose = label['pose_m'][0,:48] # PCA
th_mano_pose = torch.FloatTensor(mano_pose).view(1,-1)
selected_comps = manolayer_left.th_selected_comps if hand_type == 'left' else manolayer_right.th_selected_comps
th_mano_pose[:,3:] = th_mano_pose[:,3:].mm(selected_comps)
mano_pose = th_mano_pose.numpy()[0] # Axis-angle

[karta2155802]

Hi, I have the same question on HO3D dataset. The original dataset said their hand pose annotations are already axis-angle representation and can be directly fed into mano model, but your annotation in json file are still different from theirs. But when I tried to generate hand mesh using manolayer. Results that used your hand pose annotation are more close to the real image than the original dataset's annotation. I don't get it.

[namepllet]

For the original dataset, set flat_hand_mean=True in here https://github.com/namepllet/HandOccNet/blob/65c00af06ad81f568a6ad0ff4919f7f1d6c65e44/common/utils/mano.py#L37

[karta2155802]

Thanks for the quick reply. So what's the reson that you chosed to use average hand coefficients instead of flat hand? Does it make the performance better? Cause I read some other papers still use flat hand coefficients as groundtruth. And how to generate the average hand coefficients from the original dataset?

[namepllet]

Just for convenience. I think the performance will not much different since the average hand pose is constant offset.

The average hand pose is not for original dataset(HO3D). It is saved in MANO_RIGHT.pkl file.

[heartStrive]

In the HO3D v3 dataset, I simply write the pose parameter in the pickle to a json file, and the 21 joints are stored in the json file in the way you mentioned. However, the json file generated in this way is poorly used to train the model, could you please share the code for handling the pose parameter? Thank you very much!

[luckyday2022]

注释已经是轴角表示，可以直接输入到 mano 模型中，但你在 json 文件中的注释仍然与他们不同。 但是当我尝试使用 manolayer 生成手部网格时。使用手部姿势注释的结果比原始数

Do you know how to generate HO3D.json files？