Closed hongluzhou closed 2 years ago
@hongluzhou Hello, honglu. I have some justification for doing this. The earliest effort for doing this can be traced back to PCTDM: https://github.com/ruiyan1995/Group-Activity-Recognition/blob/1018d00e142f2560561551de38b7d02f431aad50/Configs/Dataset_Configs.py#L22-L27
Then the following work ARG: https://github.com/wjchaoGit/Group-Activity-Recognition/blob/c8c1dff953d694db93b2943ba19888be4bd6c24d/config.py#L28-L29
My prior work on AAAI2021 also adopted this setting. Some of previous works are not publicly available, so I can not ensure that they all adopt this setting. However, according to their performances, I assume that they have done the same.
@JacobYuan7 I see! Thank you very much for the reply!
Could you share the checkpoints? I'm hoping to use your code to obtain ROI Aligned person features. I tried training the stage 1 base model on volleyball on my own (I did not change any settings), but the best accuracy I got is "Best group activity accuracy: 88.56% at epoch #36." I found that you mentioned 86.39% accuracy is bit low in this issue, so I'm not sure whether the accuracy that I got is high enough. Hence hoping you could share the checkpoints!
@hongluzhou
Sorry about the late reply. I am quite occupied recently. But feel free to ask questions.
I checked the codes again and found the reason that may cause the problem of low performances. I accidentally set the _numframes to be 20. Thus, I revised the code and you should produce similar performances. If you fail to reproduce it, just let me know. I will share a link for the checkpoints. BTW, VGG-16 produces more stable results, thus I recommend using it.
@hongluzhou Sorry about the late reply. I am quite occupied recently. But feel free to ask questions. I checked the codes again and found the reason that may cause the problem of low performances. I accidentally set the _numframes to be 20. Thus, I revised the code and you should produce similar performances. If you fail to reproduce it, just let me know. I will share a link for the checkpoints. BTW, VGG-16 produces more stable results, thus I recommend using it.
Thanks for sharing the code!
Could you share the checkpoints of stage1 and stage2?
@P-DX The checkpoints are available at this link
@P-DX The checkpoints are available at this link
Thanks a lot!
Does "V_VGG16_base_92.45%.pth" mean we can get the 92.45 at stage1? Could you tell us how to get this result?
As a comparision: I trained with vgg16
as stated in https://github.com/JacobYuan7/DIN_GAR/issues/2#issuecomment-932229980 and get the following results.
./[Volleyball_stage1_stage1]<2021-11-02_15-09-00>/stage1_epoch192_87.73%.pth
./[SACRF Volleyball_stage2_stage2]<2021-10-01_14-57-51>/stage2_epoch24_92.00%.pth
./[Dynamic Volleyball_stage2_res18_litedim128_reproduce_1_stage2]<2021-11-05_17-00-56>/stage2_epoch16_92.30%.pth
./[ARG Volleyball_stage2_stage2]<2021-10-01_15-16-02>/stage2_epoch23_91.70%.pth
./[AT Volleyball_stage2_stage2]<2021-10-01_13-05-05>/stage2_epoch29_90.43%.pth
./[HiGCIN Volleyball_stage2_stage2]<2021-10-01_15-00-47>/stage2_epoch28_83.62%.pth
./[PCTDM Volleyball_stage2_stage2]<2021-11-05_17-08-22>/stage2_epoch28_91.10%.pth
Script | Accuracy |
---|---|
Stage 1: VGG16 | 87.73% |
Stage 2: SACRF | 92.00% |
Stage 2: Dynamic | 92.30% |
Stage 2: ARG | 91.70% |
Stage 2: AT | 90.43% |
Stage 2: HiGCIN | 83.62% |
Stage 2: PCTDM | 91.10% |
@P-DX The checkpoints are available at this link
Thanks a lot!
Does "V_VGG16_base_92.45%.pth" mean we can get the 92.45 at stage1? Could you tell us how to get this result?
Simply by running the codes. Would you please tell me the result you get and more details about how you reproduce it? Thus, I can analyze the reasons accordingly.
As a comparision: I trained with
vgg16
as stated in #2 (comment) and get the following results../[Volleyball_stage1_stage1]<2021-11-02_15-09-00>/stage1_epoch192_87.73%.pth ./[SACRF Volleyball_stage2_stage2]<2021-10-01_14-57-51>/stage2_epoch24_92.00%.pth ./[Dynamic Volleyball_stage2_res18_litedim128_reproduce_1_stage2]<2021-11-05_17-00-56>/stage2_epoch16_92.30%.pth ./[ARG Volleyball_stage2_stage2]<2021-10-01_15-16-02>/stage2_epoch23_91.70%.pth ./[AT Volleyball_stage2_stage2]<2021-10-01_13-05-05>/stage2_epoch29_90.43%.pth ./[HiGCIN Volleyball_stage2_stage2]<2021-10-01_15-00-47>/stage2_epoch28_83.62%.pth ./[PCTDM Volleyball_stage2_stage2]<2021-11-05_17-08-22>/stage2_epoch28_91.10%.pth
Script Accuracy Stage 1: VGG16 87.73% Stage 2: SACRF 92.00% Stage 2: Dynamic 92.30% Stage 2: ARG 91.70% Stage 2: AT 90.43% Stage 2: HiGCIN 83.62% Stage 2: PCTDM 91.10%
Can you give me more information about the script that VGG base model trained with? I never get such low base model performance.
Can you give me more information about the script that VGG base model trained with? I never get such low base model performance.
I started a Docker container and executed python scripts/train_volleyball_stage1.py
- without any modifications, as stated in the README.
Ubuntu 16.04.6 LTS
Torch 1.2.0
CUDA 10.0.130
cuDNN 7.6.03
Python 3.6.9
scikit-image 0.17.2
Pillow 6.2.0
thop 0.0.31.post2005241907
fvcore 0.1.2.post20210128
opencv-python 4.5.3.56
To copy the best model I use the following command:
cp $(find ./result -name "*stage1*.pth" -printf '%f %p\n' | sort -t_ -k3 -Vr | cut -f 2- -d ' ' | head -1) ./result/basemodel_VD_vgg16.pth --verbose
The raw saved model file names are:
./[Volleyball_stage1_stage1]<2021-11-02_15-09-00>/stage1_epoch192_87.73%.pth
./[Volleyball_stage1_stage1]<2021-11-02_15-09-00>/stage1_epoch191_87.36%.pth
./[Volleyball_stage1_stage1]<2021-11-02_15-09-00>/stage1_epoch154_87.21%.pth
./[Volleyball_stage1_stage1]<2021-11-02_15-09-00>/stage1_epoch151_87.06%.pth
./[Volleyball_stage1_stage1]<2021-11-02_15-09-00>/stage1_epoch142_86.91%.pth
./[Volleyball_stage1_stage1]<2021-11-02_15-09-00>/stage1_epoch119_86.84%.pth
./[Volleyball_stage1_stage1]<2021-11-02_15-09-00>/stage1_epoch101_86.84%.pth
./[Volleyball_stage1_stage1]<2021-11-02_15-09-00>/stage1_epoch86_86.76%.pth
./[Volleyball_stage1_stage1]<2021-11-02_15-09-00>/stage1_epoch78_86.76%.pth
./[Volleyball_stage1_stage1]<2021-09-23_12-46-16>/stage1_epoch192_86.39%.pth
./[Volleyball_stage1_stage1]<2021-11-02_15-09-00>/stage1_epoch55_86.24%.pth
./[Volleyball_stage1_stage1]<2021-09-23_12-46-16>/stage1_epoch110_86.24%.pth
./[Volleyball_stage1_stage1]<2021-09-23_12-46-16>/stage1_epoch49_86.16%.pth
./[Volleyball_stage1_stage1]<2021-11-02_15-09-00>/stage1_epoch34_86.01%.pth
./[Volleyball_stage1_stage1]<2021-11-02_15-09-00>/stage1_epoch31_85.79%.pth
./[Volleyball_stage1_stage1]<2021-11-02_15-09-00>/stage1_epoch27_85.49%.pth
./[Volleyball_stage1_stage1]<2021-11-02_15-09-00>/stage1_epoch23_85.49%.pth
./[Volleyball_stage1_stage1]<2021-09-23_12-46-16>/stage1_epoch38_85.42%.pth
./[Volleyball_stage1_stage1]<2021-09-15_19-39-19>/stage1_epoch20_85.34%.pth
./[Volleyball_stage1_stage1]<2021-11-02_15-09-00>/stage1_epoch21_85.19%.pth
./[Volleyball_stage1_stage1]<2021-09-23_12-46-16>/stage1_epoch20_85.19%.pth
./[Volleyball_stage1_stage1]<2021-11-02_15-09-00>/stage1_epoch12_85.12%.pth
./[Volleyball_stage1_stage1]<2021-09-15_19-39-19>/stage1_epoch16_84.82%.pth
./[Volleyball_stage1_stage1]<2021-09-23_12-46-16>/stage1_epoch16_83.99%.pth
./[Volleyball_stage1_stage1]<2021-11-02_15-09-00>/stage1_epoch11_83.77%.pth
./[Volleyball_stage1_stage1]<2021-09-15_19-39-19>/stage1_epoch12_83.62%.pth
./[Volleyball_stage1_stage1]<2021-11-02_15-09-00>/stage1_epoch8_82.95%.pth
./[Volleyball_stage1_stage1]<2021-09-23_12-46-16>/stage1_epoch12_82.72%.pth
./[Volleyball_stage1_stage1]<2021-09-23_12-46-16>/stage1_epoch8_81.53%.pth
./[Volleyball_stage1_stage1]<2021-09-15_19-39-19>/stage1_epoch9_81.53%.pth
./[Volleyball_stage1_stage1]<2021-11-02_15-09-00>/stage1_epoch5_81.45%.pth
./[Volleyball_stage1_stage1]<2021-09-15_19-39-19>/stage1_epoch7_81.23%.pth
./[Volleyball_stage1_stage1]<2021-09-23_12-46-16>/stage1_epoch7_80.70%.pth
./[Volleyball_stage1_stage1]<2021-09-23_12-46-16>/stage1_epoch5_80.55%.pth
./[Volleyball_stage1_stage1]<2021-09-15_19-39-19>/stage1_epoch5_80.55%.pth
./[Volleyball_stage1_stage1]<2021-11-02_15-09-00>/stage1_epoch4_79.51%.pth
./[Volleyball_stage1_stage1]<2021-11-02_15-09-00>/stage1_epoch3_79.43%.pth
./[Volleyball_stage1_stage1]<2021-09-23_12-46-16>/stage1_epoch4_77.26%.pth
./[Volleyball_stage1_stage1]<2021-09-23_12-46-16>/stage1_epoch3_76.66%.pth
./[Volleyball_stage1_stage1]<2021-09-15_19-39-19>/stage1_epoch4_76.66%.pth
./[Volleyball_stage1_stage1]<2021-09-15_19-39-19>/stage1_epoch3_76.44%.pth
./[Volleyball_stage1_stage1]<2021-11-02_15-09-00>/stage1_epoch2_70.91%.pth
./[Volleyball_stage1_stage1]<2021-09-23_12-46-16>/stage1_epoch2_69.33%.pth
./[Volleyball_stage1_stage1]<2021-09-15_19-39-19>/stage1_epoch2_69.18%.pth
./[Volleyball_stage1_stage1]<2021-11-02_15-09-00>/stage1_epoch1_66.57%.pth
./[Volleyball_stage1_stage1]<2021-09-15_19-39-19>/stage1_epoch1_59.76%.pth
./[Volleyball_stage1_stage1]<2021-09-23_12-46-16>/stage1_epoch1_58.79%.pth
Can you give me more information about the script that VGG base model trained with? I never get such low base model performance.
I started a Docker container and executed
python scripts/train_volleyball_stage1.py
- without any modifications, as stated in the README.Ubuntu 16.04.6 LTS Torch 1.2.0 CUDA 10.0.130 cuDNN 7.6.03 Python 3.6.9 scikit-image 0.17.2 Pillow 6.2.0 thop 0.0.31.post2005241907 fvcore 0.1.2.post20210128 opencv-python 4.5.3.56
To copy the best model I use the following command:
cp $(find ./result -name "*stage1*.pth" -printf '%f %p\n' | sort -t_ -k3 -Vr | cut -f 2- -d ' ' | head -1) ./result/basemodel_VD_vgg16.pth --verbose
The raw saved model file names are:
./[Volleyball_stage1_stage1]<2021-11-02_15-09-00>/stage1_epoch192_87.73%.pth ./[Volleyball_stage1_stage1]<2021-11-02_15-09-00>/stage1_epoch191_87.36%.pth ./[Volleyball_stage1_stage1]<2021-11-02_15-09-00>/stage1_epoch154_87.21%.pth ./[Volleyball_stage1_stage1]<2021-11-02_15-09-00>/stage1_epoch151_87.06%.pth ./[Volleyball_stage1_stage1]<2021-11-02_15-09-00>/stage1_epoch142_86.91%.pth ./[Volleyball_stage1_stage1]<2021-11-02_15-09-00>/stage1_epoch119_86.84%.pth ./[Volleyball_stage1_stage1]<2021-11-02_15-09-00>/stage1_epoch101_86.84%.pth ./[Volleyball_stage1_stage1]<2021-11-02_15-09-00>/stage1_epoch86_86.76%.pth ./[Volleyball_stage1_stage1]<2021-11-02_15-09-00>/stage1_epoch78_86.76%.pth ./[Volleyball_stage1_stage1]<2021-09-23_12-46-16>/stage1_epoch192_86.39%.pth ./[Volleyball_stage1_stage1]<2021-11-02_15-09-00>/stage1_epoch55_86.24%.pth ./[Volleyball_stage1_stage1]<2021-09-23_12-46-16>/stage1_epoch110_86.24%.pth ./[Volleyball_stage1_stage1]<2021-09-23_12-46-16>/stage1_epoch49_86.16%.pth ./[Volleyball_stage1_stage1]<2021-11-02_15-09-00>/stage1_epoch34_86.01%.pth ./[Volleyball_stage1_stage1]<2021-11-02_15-09-00>/stage1_epoch31_85.79%.pth ./[Volleyball_stage1_stage1]<2021-11-02_15-09-00>/stage1_epoch27_85.49%.pth ./[Volleyball_stage1_stage1]<2021-11-02_15-09-00>/stage1_epoch23_85.49%.pth ./[Volleyball_stage1_stage1]<2021-09-23_12-46-16>/stage1_epoch38_85.42%.pth ./[Volleyball_stage1_stage1]<2021-09-15_19-39-19>/stage1_epoch20_85.34%.pth ./[Volleyball_stage1_stage1]<2021-11-02_15-09-00>/stage1_epoch21_85.19%.pth ./[Volleyball_stage1_stage1]<2021-09-23_12-46-16>/stage1_epoch20_85.19%.pth ./[Volleyball_stage1_stage1]<2021-11-02_15-09-00>/stage1_epoch12_85.12%.pth ./[Volleyball_stage1_stage1]<2021-09-15_19-39-19>/stage1_epoch16_84.82%.pth ./[Volleyball_stage1_stage1]<2021-09-23_12-46-16>/stage1_epoch16_83.99%.pth ./[Volleyball_stage1_stage1]<2021-11-02_15-09-00>/stage1_epoch11_83.77%.pth ./[Volleyball_stage1_stage1]<2021-09-15_19-39-19>/stage1_epoch12_83.62%.pth ./[Volleyball_stage1_stage1]<2021-11-02_15-09-00>/stage1_epoch8_82.95%.pth ./[Volleyball_stage1_stage1]<2021-09-23_12-46-16>/stage1_epoch12_82.72%.pth ./[Volleyball_stage1_stage1]<2021-09-23_12-46-16>/stage1_epoch8_81.53%.pth ./[Volleyball_stage1_stage1]<2021-09-15_19-39-19>/stage1_epoch9_81.53%.pth ./[Volleyball_stage1_stage1]<2021-11-02_15-09-00>/stage1_epoch5_81.45%.pth ./[Volleyball_stage1_stage1]<2021-09-15_19-39-19>/stage1_epoch7_81.23%.pth ./[Volleyball_stage1_stage1]<2021-09-23_12-46-16>/stage1_epoch7_80.70%.pth ./[Volleyball_stage1_stage1]<2021-09-23_12-46-16>/stage1_epoch5_80.55%.pth ./[Volleyball_stage1_stage1]<2021-09-15_19-39-19>/stage1_epoch5_80.55%.pth ./[Volleyball_stage1_stage1]<2021-11-02_15-09-00>/stage1_epoch4_79.51%.pth ./[Volleyball_stage1_stage1]<2021-11-02_15-09-00>/stage1_epoch3_79.43%.pth ./[Volleyball_stage1_stage1]<2021-09-23_12-46-16>/stage1_epoch4_77.26%.pth ./[Volleyball_stage1_stage1]<2021-09-23_12-46-16>/stage1_epoch3_76.66%.pth ./[Volleyball_stage1_stage1]<2021-09-15_19-39-19>/stage1_epoch4_76.66%.pth ./[Volleyball_stage1_stage1]<2021-09-15_19-39-19>/stage1_epoch3_76.44%.pth ./[Volleyball_stage1_stage1]<2021-11-02_15-09-00>/stage1_epoch2_70.91%.pth ./[Volleyball_stage1_stage1]<2021-09-23_12-46-16>/stage1_epoch2_69.33%.pth ./[Volleyball_stage1_stage1]<2021-09-15_19-39-19>/stage1_epoch2_69.18%.pth ./[Volleyball_stage1_stage1]<2021-11-02_15-09-00>/stage1_epoch1_66.57%.pth ./[Volleyball_stage1_stage1]<2021-09-15_19-39-19>/stage1_epoch1_59.76%.pth ./[Volleyball_stage1_stage1]<2021-09-23_12-46-16>/stage1_epoch1_58.79%.pth
What about the training scripts? (including hyper-parameters)
What about the training scripts? (including hyper-parameters)
What did you mean by that? I cloned this repo and run the scripts without any modifications.
The output of python scripts/train_volleyball_stage1.py
that created ./[Volleyball_stage1_stage1]<2021-11-02_15-09-00>/stage1_epoch192_87.73%.pth
was:
Nov 02 15:09:08 =====================Config=====================
Nov 02 15:09:08 image_size : (720, 1280)
Nov 02 15:09:08 batch_size : 8
Nov 02 15:09:08 test_batch_size : 1
Nov 02 15:09:08 num_boxes : 12
Nov 02 15:09:08 use_gpu : True
Nov 02 15:09:08 use_multi_gpu : False
Nov 02 15:09:08 device_list : 0
Nov 02 15:09:08 dataset_name : volleyball
Nov 02 15:09:08 data_path : data/volleyball/videos
Nov 02 15:09:08 train_seqs : [1, 3, 6, 7, 10, 13, 15, 16, 18, 22, 23, 31, 32, 36, 38, 39, 40, 41, 42, 48, 50, 52, 53, 54, 0, 2, 8, 12, 17, 19, 24, 26, 27, 28, 30, 33, 46, 49, 51]
Nov 02 15:09:08 test_seqs : [4, 5, 9, 11, 14, 20, 21, 25, 29, 34, 35, 37, 43, 44, 45, 47]
Nov 02 15:09:08 backbone : vgg16
Nov 02 15:09:08 crop_size : (5, 5)
Nov 02 15:09:08 train_backbone : True
Nov 02 15:09:08 out_size : (22, 40)
Nov 02 15:09:08 emb_features : 512
Nov 02 15:09:08 num_actions : 9
Nov 02 15:09:08 num_activities : 8
Nov 02 15:09:08 actions_loss_weight : 1.0
Nov 02 15:09:08 actions_weights : [[1.0, 1.0, 2.0, 3.0, 1.0, 2.0, 2.0, 0.2, 1.0]]
Nov 02 15:09:08 num_frames : 1
Nov 02 15:09:08 num_before : 5
Nov 02 15:09:08 num_after : 4
Nov 02 15:09:08 num_features_boxes : 1024
Nov 02 15:09:08 num_features_relation : 256
Nov 02 15:09:08 num_graph : 16
Nov 02 15:09:08 num_features_gcn : 1024
Nov 02 15:09:08 gcn_layers : 1
Nov 02 15:09:08 tau_sqrt : False
Nov 02 15:09:08 pos_threshold : 0.2
Nov 02 15:09:08 train_random_seed : 0
Nov 02 15:09:08 train_learning_rate : 1e-05
Nov 02 15:09:08 lr_plan : {}
Nov 02 15:09:08 train_dropout_prob : 0.3
Nov 02 15:09:08 weight_decay : 0
Nov 02 15:09:08 max_epoch : 200
Nov 02 15:09:08 test_interval_epoch : 1
Nov 02 15:09:08 training_stage : 1
Nov 02 15:09:08 stage1_model_path :
Nov 02 15:09:08 test_before_train : True
Nov 02 15:09:08 exp_note : Volleyball_stage1
Nov 02 15:09:08 exp_name : [Volleyball_stage1_stage1]<2021-11-02_15-09-00>
Nov 02 15:09:08 set_bn_eval : False
Nov 02 15:09:08 inference_module_name : dynamic_volleyball
Nov 02 15:09:08 stride : 1
Nov 02 15:09:08 ST_kernel_size : 3
Nov 02 15:09:08 dynamic_sampling : True
Nov 02 15:09:08 sampling_ratio : [1, 3]
Nov 02 15:09:08 group : 1
Nov 02 15:09:08 scale_factor : True
Nov 02 15:09:08 beta_factor : True
Nov 02 15:09:08 load_backbone_stage2 : False
Nov 02 15:09:08 parallel_inference : False
Nov 02 15:09:08 hierarchical_inference : False
Nov 02 15:09:08 lite_dim : None
Nov 02 15:09:08 num_DIM : 1
Nov 02 15:09:08 load_stage2model : False
Nov 02 15:09:08 stage2model : None
Nov 02 15:09:08 temporal_pooled_first : False
Nov 02 15:09:08 halting_penalty : 0.0001
Nov 02 15:09:08 result_path : result/[Volleyball_stage1_stage1]<2021-11-02_15-09-00>
Nov 02 15:09:08 log_path : result/[Volleyball_stage1_stage1]<2021-11-02_15-09-00>/log.txt
Nov 02 15:09:08 ======================End=======================
Nov 02 15:09:08 Reading dataset finished...
Nov 02 15:09:08 3493 train samples
Nov 02 15:09:08 1337 test samples
Nov 02 15:19:15 {'time': 606.4387230873108, 'epoch': 0, 'loss': 7.11802880676629, 'activities_acc': 12.789827973074047, 'actions_acc': 5.204437796060833}
Nov 02 15:27:05
Nov 02 15:27:05 Train at epoch #1
Nov 02 15:27:05 Group Activity Accuracy: 38.56%, Loss: 3.47269, Using 470.8 seconds
...
Nov 05 02:07:34 Train at epoch #200
Nov 05 02:07:34 Group Activity Accuracy: 99.74%, Loss: 0.04666, Using 469.6 seconds
Nov 05 02:07:34
Nov 05 02:07:34
Nov 05 02:17:23
Nov 05 02:17:23 ====> Test at epoch #200
Nov 05 02:17:23 Group Activity Accuracy: 84.14%, Loss: 3.01014, Using 588.4 seconds
Nov 05 02:17:23
Nov 05 02:17:23
Nov 05 02:17:23 Best group activity accuracy: 87.73% at epoch #192.
link
@JacobYuan7 Thank you so much for replying and sharing the checkpoints! Could you share the link again? The link seems to be expired and I can't download the checkpoints anymore. (Also sorry about my late reply! )
Best, Honglu
@hongluzhou Hello Honglu, it's available now.
@daniel-richter I updated the training script. It should produce better results for the base model.
@hongluzhou Hello Honglu, it's available now.
Thank you so much! I'm gonna close this issue. Thanks again!!
Best, Honglu
@daniel-richter I updated the training script. It should produce better results for the base model.
Thanks. Now it is Best group activity accuracy: 91.92% at epoch #111.
Thanks for sharing the code! However, according to the code, it seems that the validation set of the volleyball dataset was used as part of the training set.
https://github.com/cvlab-epfl/social-scene-understanding/blob/b26080785f6ef1c1e0d08ab496744e5e8c1c8d4c/volleyball.py#L4 V.S. https://github.com/JacobYuan7/DIN_GAR/blob/7c5a8f54cc3b713ead438cc47dd54c491c837fbf/config.py#L28
May I ask what is the reason or justification for this? Did any prior effort do the same thing?