search

FingerRec / BE

[CVPR2021] The source code for our paper 《Removing the Background by Adding the Background: Towards Background Robust Self-supervised Video Representation Learning》.
151 stars 11 forks source link

About the setting #3

Closed gazelxu closed 3 years ago

gazelxu commented 3 years ago

Hi, the default setting in the code is different from the paper, lr for example. So which setting should I apply? I tried the default setting in the code. And I get the fine-tune-best_prec1 result around 0.456 on ucf101. I want to know what causes this result, something wrong I made or just the setting problem?

FingerRec commented 3 years ago

Hi gazelxu: Thanks for your interest! Could you please provide more detail description about your experiment? e.g., which script or the pt&ft param in options.py. BTW, which dataset you pt/ft used?

If you pretrain on UCF101, could you please provide the result with this script https://github.com/FingerRec/BE/blob/main/src/Contrastive/scripts/hmdb51/ft.sh. Notice modify the last line with your trained weight.

Yours, Awiny

gazelxu commented 3 years ago

https://github.com/FingerRec/BE/blob/main/src/Contrastive/scripts/ucf101/pt_and_ft.sh. I used this script on the ucf101. I changed the bs to 32. The other setting are kept unchanged. And the result is like this: Epoch: [43][100/299], lr: 0.00000 Time 16.404 (3.141) Data 15.401 (2.456) Loss 2.3394 (2.2308)
Finetune Training: Top1:43.53341584158416 Top3:66.46039603960396 Epoch: [43][200/299], lr: 0.00000 Time 0.483 (2.630) Data 0.000 (1.967) Loss 1.6707 (2.2017)
Finetune Training: Top1:44.542910447761194 Top3:67.41293532338308 Test: [0/119] Time 71.947 (71.947) Loss 1.7760 (1.7760)
Finetune Eval: Top1:53.125 Top3:75.0 Test: [100/119] Time 0.243 (2.603) Loss 1.3026 (2.0036)
Finetune Eval: Top1:45.66831683168317 Top3:68.13118811881188 Step2: fine tune best_prec1 is: 45.625165212289886 left time is : 0d:0h:34m:32s now is : 2021-03-30 20:47:24.661284 Epoch: [44][0/299], lr: 0.00000 Time 84.942 (84.942) Data 83.502 (83.502) Loss 2.5034 (2.5034)
Finetune Training: Top1:43.75 Top3:59.375 Epoch: [44][100/299], lr: 0.00000 Time 1.509 (3.045) Data 0.988 (2.334) Loss 2.3056 (2.1865)
Finetune Training: Top1:44.863861386138616 Top3:67.29579207920793 Epoch: [44][200/299], lr: 0.00000 Time 0.496 (2.594) Data 0.000 (1.893) Loss 2.1795 (2.1826)
Finetune Training: Top1:45.52238805970149 Top3:67.63059701492537 Test: [0/119] Time 71.963 (71.963) Loss 1.8546 (1.8546)
Finetune Eval: Top1:53.125 Top3:65.625 Test: [100/119] Time 0.410 (2.585) Loss 1.5060 (2.0612)
Finetune Eval: Top1:44.89480198019802 Top3:67.48143564356435 Step2: fine tune best_prec1 is: 45.625165212289886 left time is : 0d:0h:17m:14s now is : 2021-03-30 21:03:20.304610 finished finetune with weight from: ../experiments/ucf101_to_ucf101/03-29-1929/pt/models/current.pth

FingerRec commented 3 years ago

The scripts now are with small batch size and few epochs, which aim to provide inference enivorment that test the model's efficiency for everyone even with laptop's GPU.

Notice when use larger batch size, the iteration is less which means the backward update is less. The network has not been convergence at all from your result. i.e., the test acc higher than train acc.

The normal train/test output loss should be close to zero and the train acc should close to 100, as follow: Epoch: [299][100/193], lr: 0.00000 Time 1.697 (0.890) Data 1.388 (0.387) Loss 0.0093 (0.0658) Finetune Training: Top1:98.90761548064918 Top3:99.812734082397 Test: [0/183] Time 0.251 (0.467) Loss 0.1671 (0.2473) Finetune Eval: Top1:83.73267326732673 Top3:93.29702970297029 Test: [100/183] Time 0.564 (0.460) Loss 0.2346 (0.2669) Finetune Eval: Top1:83.61691542288557 Top3:94.02985074626865 Step2: fine tune best_prec1 is: 83.06535947712418 left time is : 0d:0h:01m:18s now is : 2020-04-03 16:46:28.145831

Use large epochs and learning rate for large memory GPU (bsz).

In addition, the hyper parameter is depend on dataset and offer in supple. I will update these scripts and upload trained models after the conferences.