Closed TXH-mercury closed 7 months ago
Hi, following previous work (e.g., https://github.com/jayleicn/recurrent-transformer), we can use val1 as a val set, and val2 as a test set.
To facilitate evaluation, we provide the processed coco-format annotations in https://huggingface.co/datasets/ShuhuaiRen/TimeIT/tree/main/data/dense_video_captioning/anet. The script for evaluating anet dense captioning can be found in https://github.com/RenShuhuai-Andy/TimeChat/blob/master/eval.sh#L15-L21.
If you want to process other custom evaluation data, please refer to https://github.com/RenShuhuai-Andy/TimeChat/blob/master/docs/DATA.md#process-custom-evaluation-data-into-coco-format.
Hi, the annotation files and processing code that I uploaded yesterday have something wrong. I have updated them just now. Please re-download them, sorry about that.
We recommend to run our processing code by yourself to ensure that all the broken local videos are filtered.
For your information, we evaluate the performance of TimeChat-7B on anet dvc test set (val2), the results are:
Thanks for the good job! I want to evaluate dvc on activitynet captions, there are val1 and val2 two annotations, how to modify the code?