Closed ShramanPramanick closed 6 months ago
Hi, thanks for your interest.
The results reported in the paper were obtained using the TimeIT + Valley dataset (we will note this more clearly in our paper update).
TimeIT is a timestamp-related instruction-tuning dataset, while Valley is a general video instruction-tuning dataset. The joint training with both datasets ensures that our model possesses a dual proficiency in both time-aware and general video comprehension.
Hi @RenShuhuai-Andy, Thanks for the clarification. Please update the paper, as it does not mention that Valley is used for training. Thanks.
Hi Authors,
I congratulate you for the great work and CVPR 2024 acceptance. Could you clarify the number of samples used for fine-tuning?
It is mentioned in the paper that TimeChat is fine-tuned on TimeIT for three epochs. However, in the training config file
train_configs/stage2_finetune_time104k_valley72k.yaml
, both TimeIT and Valley datasets are listed for fine-tuning, which together results in a total of ~176K samples. Please let me know the reason behind using the Valley dataset.Thanks, Shraman