search

OpenGVLab / efficient-video-recognition

166 stars 16 forks source link

ssv2 data sampling #4

Open Hanqer opened 2 years ago

Hanqer commented 2 years ago

Thanks for your work. But when I experimented on SSv2 dataset, as described in the paper that using the TSN style to sample frames, the sampling rate is set to -1, which will perform wrong sampling result in '_generate_temporal_crops'.

Could you kindly provide the ssv2 runable scripts?

Hanqer commented 2 years ago

I reimplement the ssv2 sampling and adjust the training setting accroding to the paper: change the iterations to 30,000 with batch-size 256, the sampling manner is change to TSN style and using 3 spatial crops for testing. The remaining settings are same with K400 as the provided scripts. However, the VIT-B/16 model only archieves 58.9 top-1 acc on SSv2 (compared with 61.7 acc in the paper ). Do I missing same changes for SSv2 dataset?

Hanqer commented 2 years ago

I reimplement the ssv2 sampling and adjust the training setting accroding to the paper: change the iterations to 30,000 with batch-size 256, the sampling manner is change to TSN style and using 3 spatial crops for testing. The remaining settings are same with K400 as the provided scripts. However, the VIT-B/16 model only archieves 58.9 top-1 acc on SSv2 (compared with 61.7 acc in the paper ). Do I missing same changes for SSv2 dataset?

linziyi96 commented 2 years ago

Would you mind letting me know how many input frames and how many decoder layers you are using? 8 frames + 4 decoder layers should obtain similar performance as you reported. We use 12 decoder layers for SSv2 as we found they provide clear improvements (to about 61% accuracy).

Hanqer commented 2 years ago

I used 16 frames input and 4 layers decoder.

Hanqer commented 2 years ago

Thanks for your help @linziyi96 . By changing the input frames to 8, and decoder layers to 12. I got 61.1% top-1 acc on SSv2 dataset.

I run this experiment with bs128, accrodingly changing the lr to 2e-4 and iterations to 60k steps.

CZC-026 commented 1 year ago

Thanks for your help @linziyi96 . By changing the input frames to 8, and decoder layers to 12. I got 61.1% top-1 acc on SSv2 dataset.

I run this experiment with bs128, accrodingly changing the lr to 2e-4 and iterations to 60k steps.

Hello , did you reproduce the experimental results on the k400? The accuracy I got is a bit lower than in the paper(about 3%)