search

leolyliu / TACO-Instructions

Official repository of "TACO: Benchmarking Generalizable Bimanual Tool-ACtion-Object Understanding".
46 stars 1 forks source link

Release of full hand poses #4

Closed enesduran closed 3 months ago

enesduran commented 4 months ago

Hello, thank you for your work!

AFAIK you only have released a subset of the hand poses under the folder TACO_Dataset_Examples. Am I wrong? Are you intending to release whole hand pose data?

Best

leolyliu commented 4 months ago

Hi enesduran,

So far, we have only released a subset of the hand poses under TACO_Dataset_Examples. We expect to release the full dataset in August 2024. Now we are fixing a few errors in object poses, if you need the hand-object poses of the whole dataset urgently, we can provide you with the current version.

Best, Yun Liu

enesduran commented 3 months ago

Hey, thank you for the fast response! I think it is not an urgent matter if full version of the dataset is going to be shared in August. Another question is do you intend to improve the hand annotations in the dataset? I have seen prevalent cases where: 1) The hand shapes(MANO betas) are visibly different 2) The annotations are jittery does not align well with the video. 1) example_20231005_032

2)

https://github.com/user-attachments/assets/d1ea32cb-ca17-4fe2-976f-36b5d97fcb8b

leolyliu commented 3 months ago

Hi enesduran,

(1) For the hand shapes, each hand of each person will share the same hand shape among the whole dataset. The shapes of the left hand and the right hand could be different. We will improve some persons' hand shapes in the next dataset version.

(2) The misalignment is hard to fix due to heavy occlusions. Currently, we have tried our best to reconstruct hands from the videos. If better annotations are acquired, we will update our dataset.

Best

enesduran commented 3 months ago

Hey, so AFAIK the hand shape parameters for each unique hand will be the same across all timesteps. However, the shape parameters do not have to be same for right and left hand. Thanks for your efforts! Looking forward to seeing new version!

leolyliu commented 3 months ago

Hey, so AFAIK the hand shape parameters for each unique hand will be the same across all timesteps. However, the shape parameters do not have to be same for right and left hand. Thanks for your efforts! Looking forward to seeing new version!

Yes, exactly.