search

EGO4D / social-interactions

MIT License
45 stars 8 forks source link

A few questions about Task TTM #4

Closed lemon-prog123 closed 2 years ago

lemon-prog123 commented 2 years ago

Hi! I've downloaded your baseline model designed for Task called Talk to me and I also read the paper on Arxiv.

  1. I've read your guideline for annotations and I've tried to open 'av_train.json' on my desktop. I find that the array named 'social_segments_talking' has its attributions 'person','target' ,'is_at_me'. But I'm confused that it seems that 'person' can be the value of zero in many situations,which means that the segment belongs to the camera wearer.However, according to the defination of the TTM, I think a segment belongs to camera wearer will not be taken into consideration. And I've read your code of baseline model, it seems that you also use this part of annotation indiscriminately.

  2. And I also have some questions about 'target'. In your annotation,'target' can be None or one or zero. And your baseline model use label zero to stand for 'target' None and label one to standsfor the rest targets. Whether the target zero stands for 'only talk to me' and target one stands for 'not only talk to me but also others'？

4.The last question. I have noticed that parts of 'person_id' are negative one.Is that standing for person who stand out of the camera view?

Thank you!

zcxu-eric commented 2 years ago

Hi, please find the answers:

  1. Yes, the camera wearer will not be considered in the TTM task. The codebase is out of date and has been updated now.
  2. Currently, our TTM task only has two classes: talking-to-me or not-talking-to-me. Talking-to-all is also considered as talking-to-me, it would be considered a new class in future work.
  3. Negative person id stands for participants that cannot be identified by annotators, these segments will be ignored. Hope this clarifies.
lemon-prog123 commented 2 years ago

Ok thank you for your hot fix ! I've checked it out.