r(2+1) d -18 pretrained model not fully reproducible

[fmthoker]

Hi, I finetuned the given pre-trained r(2+1)d model on ucf-101 using the given finetuning code. It only achieves (76 -77%) accuracy. Can you confirm if the given model is the correct one. I use the same setup as mentioned in the readme.

[PeihaoChen]

Hi, we rerun the code and get 81.84 top-1 accuracy on UCF-101. Maybe you can provide the config.json saved at the experiment directory for further analysis.

Here is the config.json used for experiments. We run this experiment on 4 GPU with batch size 8 for each GPU (total batch size is 4*8=32 )

{ "batch_size": 8, "dataset": { "annotation_path": "data/UCF101/ucfTrainTestlist", "fold": 1, "mean": [ 0.485, 0.456, 0.406 ], "name": "ucf101", "num_classes": 101, "root": "data/UCF101/UCF-101", "std": [ 0.229, 0.224, 0.225 ] }, "final_validate": { "batch_size": 8 }, "log_interval": 10, "method": "from-scratch", "model": { "arch": "r2plus1d-vcop" }, "model_type": "multitask", "num_epochs": 30, "num_workers": 8, "optimizer": { "dampening": 0, "lr": 0.1, "milestones": [ 50, 100, 150 ], "momentum": 0.9, "nesterov": false, "patience": 10, "schedule": "cosine", "weight_decay": 0.0001 }, "spatial_transforms": { "color_jitter": { "brightness": 0, "contrast": 0, "hue": 0, "saturation": 0 }, "crop_area": { "max": 1, "min": 0.25 }, "gray_scale": 0, "size": 112 }, "temporal_transforms": { "frame_rate": null, "size": 16, "strides": [ { "stride": 1, "weight": 1 } ], "validate": { "final_n_crop": 10, "n_crop": 1, "stride": 1 } }, "validate": { "batch_size": 64 } }

[fmthoker]

Here is the config file. I also train with 4 GPUS.

{ "batch_size": 8, "dataset": { "annotation_path": "/ssdstore/fmthoker/UCF101/ucfTrainTestlist", "fold": 1, "mean": [ 0.485, 0.456, 0.406 ], "name": "ucf101", "num_classes": 101, "root": "/ssdstore/fmthoker/UCF101/UCF-101", "std": [ 0.229, 0.224, 0.225 ] }, "final_validate": { "batch_size": 8 }, "log_interval": 10, "method": "from-scratch", "model": { "arch": "r2plus1d-vcop" }, "model_type": "multitask", "num_epochs": 30, "num_workers": 8, "optimizer": { "dampening": 0, "lr": 0.1, "milestones": [ 50, 100, 150 ], "momentum": 0.9, "nesterov": false, "patience": 10, "schedule": "cosine", "weight_decay": 0.0001 }, "spatial_transforms": { "color_jitter": { "brightness": 0, "contrast": 0, "hue": 0, "saturation": 0 }, "crop_area": { "max": 1, "min": 0.25 }, "gray_scale": 0, "size": 112 }, "temporal_transforms": { "frame_rate": null, "size": 16, "strides": [ { "stride": 1, "weight": 1 } ], "validate": { "final_n_crop": 10, "n_crop": 1, "stride": 1 } }, "validate": { "batch_size": 64 } }

Final epoch and validate Logs

2021-07-08 02:04:46,333|INFO |Saving checkpoint to "exps/ucf101-r2plus1d/checkpoint.pth.tar" 2021-07-08 02:04:46,804|INFO |Checkpoint saved 2021-07-08 02:04:46,805|INFO |Current LR:[0.0003736313263547437] 2021-07-08 02:04:46,988|INFO |Train epoch begin. 2021-07-08 02:04:50,179|INFO |Train [29/30][9/298] Loss 0.164545 (0.105715) Acc@1 100.00 ( 97.50) Acc@5 100.00 (100.00) 2021-07-08 02:04:53,112|INFO |Train [29/30][19/298] Loss 0.404558 (0.105021) Acc@1 87.50 ( 97.50) Acc@5 100.00 (100.00) 2021-07-08 02:04:56,055|INFO |Train [29/30][29/298] Loss 0.007631 (0.098247) Acc@1 100.00 ( 97.92) Acc@5 100.00 ( 99.58) 2021-07-08 02:04:58,979|INFO |Train [29/30][39/298] Loss 0.037493 (0.095261) Acc@1 100.00 ( 98.12) Acc@5 100.00 ( 99.38) 2021-07-08 02:05:01,908|INFO |Train [29/30][49/298] Loss 0.000561 (0.087462) Acc@1 100.00 ( 98.25) Acc@5 100.00 ( 99.50) 2021-07-08 02:05:04,826|INFO |Train [29/30][59/298] Loss 0.093303 (0.082671) Acc@1 100.00 ( 98.54) Acc@5 100.00 ( 99.58) 2021-07-08 02:05:07,753|INFO |Train [29/30][69/298] Loss 0.006309 (0.080853) Acc@1 100.00 ( 98.57) Acc@5 100.00 ( 99.64) 2021-07-08 02:05:10,677|INFO |Train [29/30][79/298] Loss 0.392491 (0.079365) Acc@1 87.50 ( 98.59) Acc@5 100.00 ( 99.69) 2021-07-08 02:05:13,597|INFO |Train [29/30][89/298] Loss 0.010696 (0.077485) Acc@1 100.00 ( 98.61) Acc@5 100.00 ( 99.58) 2021-07-08 02:05:16,568|INFO |Train [29/30][99/298] Loss 0.006068 (0.072188) Acc@1 100.00 ( 98.75) Acc@5 100.00 ( 99.62) 2021-07-08 02:05:19,491|INFO |Train [29/30][109/298] Loss 0.001256 (0.070225) Acc@1 100.00 ( 98.75) Acc@5 100.00 ( 99.66) 2021-07-08 02:05:22,501|INFO |Train [29/30][119/298] Loss 0.079567 (0.067737) Acc@1 100.00 ( 98.85) Acc@5 100.00 ( 99.69) 2021-07-08 02:05:25,466|INFO |Train [29/30][129/298] Loss 0.112815 (0.065780) Acc@1 87.50 ( 98.85) Acc@5 100.00 ( 99.71) 2021-07-08 02:05:28,457|INFO |Train [29/30][139/298] Loss 0.016850 (0.066030) Acc@1 100.00 ( 98.75) Acc@5 100.00 ( 99.73) 2021-07-08 02:05:31,410|INFO |Train [29/30][149/298] Loss 0.034186 (0.067202) Acc@1 100.00 ( 98.75) Acc@5 100.00 ( 99.75) 2021-07-08 02:05:34,386|INFO |Train [29/30][159/298] Loss 0.001834 (0.066180) Acc@1 100.00 ( 98.75) Acc@5 100.00 ( 99.77) 2021-07-08 02:05:37,303|INFO |Train [29/30][169/298] Loss 0.075972 (0.065335) Acc@1 100.00 ( 98.75) Acc@5 100.00 ( 99.78) 2021-07-08 02:05:40,330|INFO |Train [29/30][179/298] Loss 0.005939 (0.065777) Acc@1 100.00 ( 98.75) Acc@5 100.00 ( 99.79) 2021-07-08 02:05:43,280|INFO |Train [29/30][189/298] Loss 0.135287 (0.065799) Acc@1 87.50 ( 98.68) Acc@5 100.00 ( 99.80) 2021-07-08 02:05:46,219|INFO |Train [29/30][199/298] Loss 0.018424 (0.063578) Acc@1 100.00 ( 98.75) Acc@5 100.00 ( 99.81) 2021-07-08 02:05:49,171|INFO |Train [29/30][209/298] Loss 0.017624 (0.063104) Acc@1 100.00 ( 98.75) Acc@5 100.00 ( 99.82) 2021-07-08 02:05:52,140|INFO |Train [29/30][219/298] Loss 0.041978 (0.062044) Acc@1 100.00 ( 98.75) Acc@5 100.00 ( 99.83) 2021-07-08 02:05:55,171|INFO |Train [29/30][229/298] Loss 0.619265 (0.062907) Acc@1 87.50 ( 98.75) Acc@5 87.50 ( 99.78) 2021-07-08 02:05:58,110|INFO |Train [29/30][239/298] Loss 0.002491 (0.064087) Acc@1 100.00 ( 98.70) Acc@5 100.00 ( 99.79) 2021-07-08 02:06:01,148|INFO |Train [29/30][249/298] Loss 0.040459 (0.064124) Acc@1 100.00 ( 98.70) Acc@5 100.00 ( 99.80) 2021-07-08 02:06:04,088|INFO |Train [29/30][259/298] Loss 0.006378 (0.067419) Acc@1 100.00 ( 98.56) Acc@5 100.00 ( 99.76) 2021-07-08 02:06:07,037|INFO |Train [29/30][269/298] Loss 0.083955 (0.069783) Acc@1 100.00 ( 98.52) Acc@5 100.00 ( 99.72) 2021-07-08 02:06:09,981|INFO |Train [29/30][279/298] Loss 0.002853 (0.071010) Acc@1 100.00 ( 98.44) Acc@5 100.00 ( 99.73) 2021-07-08 02:06:12,913|INFO |Train [29/30][289/298] Loss 0.271745 (0.074736) Acc@1 87.50 ( 98.36) Acc@5 100.00 ( 99.70) 2021-07-08 02:06:15,228|INFO |Train epoch finished. Time: 88.24 sec. Loss 0.003587 (0.075304) Acc@1 100.00 ( 98.32) Acc@5 100.00 ( 99.71) 2021-07-08 02:06:15,231|INFO |Validate epoch begin. 2021-07-08 02:06:22,221|INFO |Validate [29/30][9/15] Loss 0.951899 (1.170478) Acc@1 81.25 ( 72.81) Acc@5 90.62 ( 91.41) 2021-07-08 02:06:24,307|INFO |Validate epoch finished. Time: 9.08 sec. Loss 0.948824 (1.081880) Acc@1 74.00 ( 74.52) Acc@5 90.00 ( 91.86) 2021-07-08 02:06:24,659|INFO |Validation finished. Loss = 1.070267 Acc@1 = 74.31% (2811/3783) Acc@5 = 92.02% (3481/3783)

2021-07-08 02:06:24,661|INFO |Saving checkpoint to "exps/ucf101-r2plus1d/checkpoint.pth.tar" 2021-07-08 02:06:25,056|INFO |Checkpoint saved 2021-07-08 02:06:25,057|INFO |Doing final validate. 2021-07-08 02:06:25,057|INFO |Using global get_model_class({'arch': 'r2plus1d-vcop'}) 2021-07-08 02:06:25,057|INFO |Using MultiTask Wrapper 2021-07-08 02:06:25,057|WARNING |<class 'moco.split_wrapper.MultiTaskWrapper'> using groups: 1 2021-07-08 02:06:25,179|INFO |Building Dataset: VID: False, Split=val 2021-07-08 02:06:25,179|INFO |Temporal transform type: clip 2021-07-08 02:06:25,284|INFO |You are using VideoDataset: <class 'datasets.classification.video.VideoDataset'> 2021-07-08 02:06:25,287|INFO |Engine: n_crop=10 2021-07-08 05:05:20,780|INFO |Loading checkpoint from exps/ucf101-r2plus1d/model_best.pth.tar 2021-07-08 05:05:20,785|INFO |Checkpoint loaded 2021-07-08 05:05:20,930|INFO |Validate epoch begin. 2021-07-08 05:05:34,021|INFO |Validate [25/30][9/119] Loss 0.745242 (0.796004) Acc@1 75.00 ( 78.75) Acc@5 100.00 ( 98.75) 2021-07-08 05:05:41,442|INFO |Validate [25/30][19/119] Loss 1.454821 (0.548433) Acc@1 50.00 ( 83.12) Acc@5 75.00 ( 98.12) 2021-07-08 05:05:48,891|INFO |Validate [25/30][29/119] Loss 0.231421 (0.739975) Acc@1 100.00 ( 80.42) Acc@5 100.00 ( 96.67) 2021-07-08 05:05:56,384|INFO |Validate [25/30][39/119] Loss 1.614963 (0.753557) Acc@1 50.00 ( 79.38) Acc@5 100.00 ( 96.25) 2021-07-08 05:06:03,885|INFO |Validate [25/30][49/119] Loss 0.000212 (0.873438) Acc@1 100.00 ( 77.00) Acc@5 100.00 ( 95.25) 2021-07-08 05:06:11,414|INFO |Validate [25/30][59/119] Loss 0.834699 (0.881560) Acc@1 62.50 ( 77.08) Acc@5 100.00 ( 95.42) 2021-07-08 05:06:18,941|INFO |Validate [25/30][69/119] Loss 1.638537 (0.994781) Acc@1 75.00 ( 75.71) Acc@5 87.50 ( 94.11) 2021-07-08 05:06:26,487|INFO |Validate [25/30][79/119] Loss 1.217719 (0.975878) Acc@1 75.00 ( 76.41) Acc@5 100.00 ( 93.75) 2021-07-08 05:06:34,048|INFO |Validate [25/30][89/119] Loss 0.950695 (0.953382) Acc@1 50.00 ( 76.39) Acc@5 100.00 ( 93.89) 2021-07-08 05:06:41,609|INFO |Validate [25/30][99/119] Loss 0.001005 (0.967156) Acc@1 100.00 ( 76.25) Acc@5 100.00 ( 93.62) 2021-07-08 05:06:49,120|INFO |Validate [25/30][109/119] Loss 0.036038 (0.937112) Acc@1 100.00 ( 76.70) Acc@5 100.00 ( 93.98) 2021-07-08 05:06:55,569|INFO |Validate epoch finished. Time: 94.64 sec. Loss 4.500062 (0.913053) Acc@1 0.00 ( 76.96) Acc@5 50.00 ( 94.29) 2021-07-08 05:06:55,570|INFO |Validation finished. Loss = 0.901132 Acc@1 = 77.87% (2946/3783) Acc@5 = 93.81% (3549/3783)

[brian7685]

Same here.