search

cvdfoundation / kinetics-dataset

758 stars 96 forks source link

Question Regarding kinetics-400 Dataset: What are test videos? #49

Open dkmv0623 opened 8 months ago

dkmv0623 commented 8 months ago

Hello, I'm new to the field of Action recognition and have a question regarding the dataset split. Specifically for the kinetics-400 dataset, in the paper "Unmasked Teacher: Towards Training-Efficient Video Foundation Models," they provide the following summary for the number of training and validation data: unmasked_teacher

In the Video Swin Transformer paper, they also describe the kinetics-400 dataset as follows: video_swin

Both papers commonly state that kinetics-400 consists of approximately 240k training videos and 20k validation videos. However, the CSV file provided in this GitHub repository contains around 40k test videos that are not mentioned in the papers. Could you please clarify what are these test videos?

Additionally, the link to https://deepmind.com/research/open-source/kinetics is not working correctly. Has the official project page been removed?

I would appreciate insights from those who have continued their research in the field of Action Recognition, and familiar with the kinetics dataset.

joaoluiscarreira commented 8 months ago

Hi, the dataset has a validation set and a test set. Both are for testing, but validation was designed for people to iterate on when publishing (it was public so people could compute performance). The test set was held out, so only the kinetics dataset creators could compute performance of a method -- this was useful for building competitions.

The website at deepmind.com was removed when the website was updated (i should re-create it somewhere when i have time).

matzolla commented 7 months ago

Hello @joaoluiscarreira thanks for this clarification.