about dataset labels

[zachluo]

Hi,

Thanks for your great job! I have two questions about the labels provided in this dataset.

i) What do the values in hand poses mean? For example, -0.03166291117668152 and 0.11372638493776321. ii) I see that the hand pose labels are available for the testing set. Do I misunderstand it?

For the challenge host in ECCV2022, can I use other pre-trained models like k400 or epic-kitchen？ Also, I can only use the training set rather than training+validation. Am I right?

Best

[taeinkwon]

Hi @zachluo ,

Thank you for your interest in our dataset!

1) The values in hand pose in meters. The order follows below:

1 (whether annotate or not, 0: not annotate 1: annotate) + 21 3 (x, y, z in order) + 1 + 21 3 (right hand) The first 64 numbers belong to the left hand. The next 64 numbers belong to the right hand

2) Yes you can see the test set as well. This is for object & hand pose-based action recognition. Please do not rely on the test set and validate/report your number here (https://codalab.lisn.upsaclay.fr/competitions/4822).

Best, Taein Kwon

[taeinkwon]

For the challenge, we don't allow to use of other pre-trained models. Yes, you can only use the training set for training purposes. I hope you can get good results. Thanks!

[zachluo]

1) Do you mean that I can use the ground-truth of hand and object poses for action recognition? However, I see that you infer the poses by yolov2 for action recognition in the paper. Which one would be correct? 2) Can I use the depth maps and static camera images for egocentric hand pose estimation in the challenge？

Thanks.

---Original--- From: @.> Date: Fri, Sep 9, 2022 05:06 AM To: @.>; Cc: "Weixin Luo @.**@.>; Subject: Re: [taeinkwon/h2odataset] about dataset labels (Issue #13)

Hi @zachluo ,

Thank you for your interest in our dataset!

The values in hand pose in meters. The order follows below:

1 (whether annotate or not, 0: not annotate 1: annotate) + 21 3 (x, y, z in order) + 1 + 21 3 (right hand) The first 64 numbers belong to the left hand. The next 64 numbers belong to the right hand

Yes you can see the test set as well. This is for object & hand pose-based action recognition. Please do not rely on the test set and validate/report your number here (https://codalab.lisn.upsaclay.fr/competitions/4822).

Best, Taein Kwon

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: @.***>

[taeinkwon]

1) Yes you can use the ground truth of hand and object poses for action recognition. It's up to the challengers which modalities they will use for the action recognition challenge as hand & object based action recognition is a new challenge. What you need to do is write the modalities that you used when you submit. 2) In terms of the hand pose estimation challenge, it is a bit more limited. you can only use images as depth makes a big difference when we estimate hand pose.

Best, Taein Kwon

[zachluo]

@taeinkwon great! One more question. That means I can use multi-camera images for pose estimation. Is it right?

[taeinkwon]

To explain it more, the challenge is targeting hand pose estimation in an egocentric view. Therefore, the challenge only allows RGBs from the egocentric view.

Besides the challenge, it would be interesting to see the result from other modalities such as depth or multi-view cameras.

[zachluo]

@taeinkwon great! Thanks!