celebv-text / CelebV-Text

(CVPR 2023) CelebV-Text: A Large-Scale Facial Text-Video Dataset
https://celebv-text.github.io/
388 stars 33 forks source link

Confusion regarding the annotation file #16

Open Changezi001 opened 9 months ago

Changezi001 commented 9 months ago

Hi,

Thank you for sharing the data. But there is a confusion regarding the annotations.pk file. This dictionary has just two key values, i.e., act and app, corresponding to actions and appearances. There is no emotion information according to my understanding, or may be I am interpreting it incorrectly. It would be great if you could explain the structure of annotations.pkl file to clarify this confusion.

For instance a single entry from the action part looks like this: Key: zYeLz4c4qYc_15_0, Value: [['talk', ['0:03:37', '0:03:43', 6, 0]], ['head_wagging', ['0:03:41', '0:03:43', 2, 1]]]

It would be beneficial to the research community if you could explain what 6 and 0 mean in the 'talk' bracket and what 2 and 1 mean in the 'head_wagging' bracket. Do they correspond to emotion labels?

Thank you,

celebv-text commented 9 months ago

Hi @Changezi001, the number "6" indicates the time duration in sconds, and the "0" indicates the order of the action happening. For some cases you might see the following: [['talk', ['0:09:07', '0:09:18', 11, 0]], ['gaze', ['0:09:07', '0:09:18', 11, 0]], ['frown', ['0:09:09', '0:09:10', 1, 1]], ['blink', ['0:09:11', '0:09:18', 7, 2]]] where both 'talk' and 'gaze' happen at the same time. In this case, their orders should be the same, which are both '0'.

Changezi001 commented 9 months ago

@celebv-text Thank you for the prompt response. Are you sure the number "6" indicates the time duration in seconds? Because when I go to the video zYeLz4c4qYc_15_0.mp4, the total length of the video is 5 seconds. However, the entry from the action part given below indicates that the video has a duration of more than 3 minutes : Key: zYeLz4c4qYc_15_0, Value: [['talk', ['0:03:37', '0:03:43', 6, 0]], ['head_wagging', ['0:03:41', '0:03:43', 2, 1]]]

Note that I have downloaded the videos from the link given here

video download link: https://onedrive.live.com/?authkey=%21AAeock8a7zyMZ1E&id=B4BF941CAD0326B7%21196&cid=B4BF941CAD0326B7

Thank you

9B8DY6 commented 8 months ago

Does meta information pkl file only contain info. about action? @celebv-text Could you tell me how to aggregate emotion txt with action txt?