search

mit-han-lab / temporal-shift-module

[ICCV 2019] TSM: Temporal Shift Module for Efficient Video Understanding
https://arxiv.org/abs/1811.08383
MIT License
2.05k stars 418 forks source link

the ssv1 acc of using test scripts is the results on val dataset. how to get the results on the official test dataset ? #172

Open gitxdf opened 3 years ago

gitxdf commented 3 years ago

The TSM test scripts on Something-Something v1 dataset is as follow: efficient setting: center crop and 1 clip

python test_models.py something \
    --weights=pretrained/TSM_something_RGB_resnet50_shift8_blockres_avg_segment8_e45.pth \
    --test_segments=8 --batch_size=72 -j 24 --test_crops=1

accurate setting: full resolution and 2 clips (--twice sample)

python test_models.py something \
    --weights=pretrained/TSM_something_RGB_resnet50_shift8_blockres_avg_segment8_e45.pth \
    --test_segments=8 --batch_size=72 -j 24 --test_crops=3  --twice_sample

But, there is no argument about test_list. We can know the acc is on the val dataset from the code in line 177 of test_models.py : test_file if test_file is not None else val_list

The test data is offered on the official site: https://20bn.com/datasets/something-something/v1 So how to get the results on the test dataset? Or, the acc on the paper is based on the val dataset ? Thanks.

tjinjin95 commented 3 years ago

hi, I have the same question. Could you add a "true test"(test_models.py row 38) method to the readme ? @gitxdf @willprice @songhan @zhijian-liu @JoshNoel

Qe-L commented 2 years ago

I have same question. There is one thing clear that the val set of ssv2 is not used in training process(can be seen from main.py). So, It is means that this paper and other papers only report acc on val set?