search

TengdaHan / MemDPC

[ECCV'20 Spotlight] Memory-augmented Dense Predictive Coding for Video Representation Learning. Tengda Han, Weidi Xie, Andrew Zisserman.
Apache License 2.0
164 stars 20 forks source link

Question about pre-processing of Kinetics-400 #2

Closed liuhualin333 closed 4 years ago

liuhualin333 commented 4 years ago

Hi Tengda,

Thanks for sharing your work! I have some questions regarding the Kinetics-400.

  1. Understanding that one must download the Kinetics videos himself, may I ask how many videos have you crawled? Is there any possibilities that you could share your downloaded videos for better replication?
  2. Which ffmpeg command do you use for resizing Kinetics videos to short side of 256?

Looking forward to hearing from you!

Best Regards, Hualin

TengdaHan commented 4 years ago

Hi, thank you for your interest.

  1. For Kinetics400, our train set has 232422 videos, val set has 19141 videos. It's also not a complete version. As long as your number of videos is not far from this, I think it won't affect much. BTW, we only use the train set for self-supervised training.

  2. I just added an example in process_data/src/ for resizing videos to short size == 256.

liuhualin333 commented 4 years ago

Hi Tengda, thanks for your prompt reply!

  1. I am currently downloading Kinetics-400 myself and the expected video number for training set is around 210000 only (considering the dataset is out for several years). I think it could have a potential huge impact on performance. It would be highly appreciated if you could kindly share a dropbox copy of your dataset by email for academic purposes🙏.

  2. Thx for providing the cmd!

  3. I have run your pre-training code and noticed that with multi-gpu setting your DataParallel nn module would gather multiple copies of loss and accuracies which would cause error in the main training procedure. The exact command to replicate is python main.py --gpu 0,1 --net resnet18 --dataset ucf101 --batch_size 16 --img_dim 128 --epochs 500

The line which throws the error is in main.py is listed here: 200 accuracy[i][0].update(top1.item(), B)

TengdaHan commented 4 years ago

Hi,

  1. For Kinetics dataset: Sorry, I don't think I can re-distribute the videos due to potential copyright issues, also Kinetics400 is too large to save in online storage. From my experience, 10% fewer videos will not have a huge impact on performance. If you really want to match the number of videos, one possible solution is to download some videos (from the same class) from Kinetics600 or Kinetics700. e.g. 300+ action classes from K400 are included in K700.

  2. For the error: Thanks for pointing out. I corrected it just now. Hope it works well.

liuhualin333 commented 4 years ago

Thanks for the advice. Close the issue now.