About precision on ucf101

[zhishihaizhi]

Hi， Thank you for your wonderful job! I use the UCF101 pretrained model downloaded by the script to test the split1 of UCF101, but i only got 88.9% (lite model) and 89.74% (full model) compare to the 91.6% and 92.8% in the paper. The segment is set to 16 and no parameter is changed.

Thx.

[mzolfaghari]

Hi @zhishihaizhi

Others who used our models reported the same results as reported in the paper. Please check your sampling or cropping. In the test time, you must crop from the center and sample 16 frames uniformly from the entire video. If this didn't help, share your evaluation script and I'll have a look to see what's the problem.

Best,

[phexic]

Hi, @mzolfaghari , i finetuned ucf101 on pytorch with command "sh run_demo_ECO_Lite.sh local ", however I only got top1 76.48%. Is there something wrong?

[mzolfaghari]

@phexic Did you download pre-trained models on Kinetics dataset and initialize the network?

[phexic]

@mzolfaghari Yean, I have downloaded pre-trained models "ECO_Lite_rgb_model_Kinetics.pth.tar". I put ECO_Lite_rgb_model_Kinetics.pth.tar in the models file and modify the nECO-model path("./models/ECO_Lite_rgb_model_Kinetics.pth.tar") in run_demo_ECO_Lite.sh script. Then run the "sh run_demo_ECO_Lite.sh local" script. top1 on train_split is ~93%, top1 on val_split can only reach ~76%. Thanks,

[accountcwd]

@mzolfaghari Yean, I have downloaded pre-trained models "ECO_Lite_rgb_model_Kinetics.pth.tar". I put ECO_Lite_rgb_model_Kinetics.pth.tar in the models file and modify the nECO-model path("./models/ECO_Lite_rgb_model_Kinetics.pth.tar") in run_demo_ECO_Lite.sh script. Then run the "sh run_demo_ECO_Lite.sh local" script. top1 on train_split is ~93%, top1 on val_split can only reach ~76%. Thanks,

hello , i got the same val acc(about76%) too ,have you solved this problem？

[mzolfaghari]

@zhishihaizhi @phexic @accountcwd To answer this issue. I ran the code again and got 90% accuracy after 3 epochs. I did the following steps: 1- Download code from: https://github.com/mzolfaghari/ECO-pytorch 2- Download ECO lite model from: https://drive.google.com/open?id=1XNIq7byciKgrn011jLBggd2g79jKX4uD 3- Set UCF101 split 1 list path in "run_demo_ECO_Lite.sh" 4- Set the "ECO_Lite_rgb_model_Kinetics.pth.tar" path in the "run_demo_ECO_Lite.sh" for the "nECO_model" 5- In the "run_demo_ECO_Lite.sh" set dropout to 0 or a small number like 0.3. Then, Run the network.

If finetuning is correct, you should get the following in un_init_dict_keys: un_init_dict_keys: ['module.new_fc.weight', 'module.new_fc.bias'] If after these steps you couldn't get the same results, then send your log file so that I can check and see what's the problem.

[chenyihang1993]

@mzolfaghari Hi, I follow the steps, but I got top-1 89.9. The head of the log file is:

Environment Versions:
- Python: 3.6.9 |Anaconda, Inc.| (default, Jul 30 2019, 19:07:31) 
[GCC 7.3.0]
- PyTorch: 1.2.0
- TorchVison: 0.4.0a0+9232c4a
------------------------------------
ECO Configurations:
- dataset: ucf101
- modality: RGB
- train_list: list/ucf101_train_split1.txt
- val_list: list/ucf101_val_split1.txt
- net_model: None
- net_model2D: nll
- net_modelECO: ECO_Lite_rgb_model_Kinetics.pth.tar
- net_model3D: nll
- arch: ECO
- num_segments: 16
- consensus_type: identity
- pretrained_parts: finetune
- k: 3
- dropout: 0.0
- loss_type: nll
- epochs: 40
- batch_size: 16
- iter_size: 4
- lr: 0.001
- lr_steps: [20, 40]
- momentum: 0.9
- weight_decay: 0.0005
- clip_gradient: 50.0
- no_partialbn: True
- nesterov: True
- num_saturate: 5
- print_freq: 20
- eval_freq: 1
- workers: 2
- resume: 
- evaluate: False
- snapshot_pref: net_runs/ECO_lite_finetune_UCF101_run1/eco_lite_finetune_UCF101
- start_epoch: 0
- gpus: None
- flow_prefix: 
- rgb_prefix: img_
------------------------------------

Initializing TSN with base model: ECO.
TSN Configurations:
    input_modality:     RGB
    num_segments:       16
    new_length:         1
    consensus_module:   identity
    dropout_ratio:      0.0

pretrained_parts:  finetune
ECO_Lite_rgb_model_Kinetics.pth.tar
88888888888888888888888888888888888888888888888888888888888888888888888888888888
=> loading model-finetune: 'ECO_Lite_rgb_model_Kinetics.pth.tar'
**************************************************
Start finetuning ..
un_init_dict_keys:  ['module.base_model.fc_final.weight', 'module.base_model.fc_final.bias']

------------------------------------
module.base_model.fc_final.weight init as: xavier
module.base_model.fc_final.bias init as: 0
------------------------------------
group: first_3d_conv_weight has 1 params, lr_mult: 1, decay_mult: 1
group: first_3d_conv_bias has 1 params, lr_mult: 2, decay_mult: 0
group: normal_weight has 32 params, lr_mult: 1, decay_mult: 1
group: normal_bias has 32 params, lr_mult: 2, decay_mult: 0
group: BN scale/shift has 60 params, lr_mult: 1, decay_mult: 0
No BN layer Freezing.
Epoch: [0][3/597], lr: 0.0010000        Time 0.36 (17.59)       UTime 2019/09/26-10:18:24       Data 0.00 (0.38)        Loss 5.258 (3.220)      Prec@1 0.00 (0.00)      Prec@5 0.00 (3.12)