Source of Answer Annotations for Ego Reasoning Task

[Becomebright]

Thanks for your amazing work.

Your paper states, "Both videos and QA annotations are collected from the NLQ task of Ego4D". However, NLQ does not provide answer annotations. Could you clarify how the answers were obtained for the Ego Reasoning task?

Thanks!

[JUNJIE99]

Thank you for your interest!

In the NLQ annotations of Ego4D, some questions are accompanied by answers, but it's not mandatory for all questions to have them. Below is an example of a data entry copied from the original annotations, which includes an answer:

{
    "language_queries": [
        {
            "clip_start_sec": 240.50272,
            "clip_end_sec": 254.69288,
            "video_start_sec": 2580.50272,
            "video_end_sec": 2594.69288,
            "video_start_frame": 77415,
            "video_end_frame": 77841,
            "template": "Objects: What X is Y?",
            "query": "Query Text:What color t-shirt was the man wearing holding the air-blower machine?",
            "slot_x": "X Text (Optional):color t-shirt",
            "verb_x": "X Verb (Optional):[verb_not_applicable]",
            "slot_y": "Y Text (Optional):the man wearing holding the air-blower machine",
            "verb_y": "Y Verb (Optional):wear",
            "answer": "Answer (Optional):Grey",
            "raw_tags": [
                "Objects: What X is Y?",
                "Query Text:What color t-shirt was the man wearing holding the air-blower machine?",
                "X Text (Optional):color t-shirt",
                "X Verb (Optional):[verb_not_applicable]",
                "Y Text (Optional):the man wearing holding the air-blower machine",
                "Y Verb (Optional):wear",
                "Answer (Optional):Grey"
            ]
        }
    ],
    "annotation_uid": "1ec95b3c-8291-476e-b658-902919debf92",
    "request_uid": "950e1cf5-3bed-44ab-bf2f-fac7d62ccbc9"
}

As shown in the annotation above, the answer is marked as "optional", meaning that while not every question has one, there are indeed some questions with answer annotations. Our MLVU has utilized these questions with answer annotations, and has annotated incorrect answers for these questions to form them into multiple-choice questions.

If you have any other questions or concerns, please feel free to discuss them with us.

[JUNJIE99]

This issue is being closed due to a prolonged period of inactivity. Feel free to reopen it if you have any further questions.