search

OpenGVLab / Ask-Anything

[CVPR2024 Highlight][VideoChatGPT] ChatGPT with video understanding! And many more supported LMs such as miniGPT4, StableLM, and MOSS.
https://vchat.opengvlab.com/
MIT License
2.85k stars 230 forks source link

Bug Report: Identical Start and End Timestamps in object_interaction.json of MVBench #191

Closed patrick-tssn closed 2 weeks ago

patrick-tssn commented 2 weeks ago

Thank you for your work. I've discovered an issue within the MVBench, specifically in the object_interaction.json file. It appears that there is a case where the start time is identical to the end timestamp.

{
        "video": "5W1Z7.mp4",
        "question": "Which object was lied on by the person?",
        "candidates": [
            "The bed.",
            "The bag.",
            "The sofa/couch.",
            "The floor."
        ],
        "answer": "The bed.",
        "start": 7.5,
        "end": 7.5,
        "accurate_start": 0.0,
        "accurate_end": 8.0
    }
Andy1621 commented 2 weeks ago

Good!!! I have checked the data and it's the only one bug data with start >= end. I will fixed it later. Considering it only affect the results by 1/200=0.005, you can still use the original version.

Andy1621 commented 2 weeks ago

Thanks for your report again. I have updated the JSON files.

I found the BUG is caused since the video duration is actually 6s, while the accurate_start=0, accurate_end=8. To fix it, I simply use start=0, end=8. In my code, it will change the start and end when exceed the boundary.