Closed NIneeeeeem closed 3 months ago
Hi, thanks for your interest.
We conduct zero-shot evaluation on the QA benchmarks using the original TimeChat weights.
@RenShuhuai-Andy Thanks very mach for your reply, I just tested the performance of zeroshot and it does have good performance on open-set VQA.
However, there is no output that matches the form on benchmarks with multiple choice questions such as EgoScheme. This type of problem also occurs in many models that have not been fine-tuned for multiple choice questions, and I wonder if you have a good solution for this!
Thank you for sharing your work! No problem now.
Hi, there. Thanks for sharing your great work.
I wonder if the performance on the recently released QA benchmarks is zeroshot performance. Or rather, whether the original TimeChat weights have been imported or if there are new fine-tuned weights?