Accuracy difference when reproducing GaitBase model

[enemy1205]

I trained exactly according to the profile configs/gaitbase/gaitbase_da_gait3d.yaml ,without any changes

[2023-05-08 10:33:31] [INFO]: {'enable_float16': True, 'with_test': None, 'fix_BN': False, 'log_iter': 100, 'restore_ckpt_strict': True, 'optimizer_reset': False, 'scheduler_reset': False, 'restore_hint': 0, 'save_iter': 20000, 'save_name': 'GaitBase_DA', 'sync_BN': True, 'total_iter': 60000, 'sampler': {'batch_shuffle': True, 'batch_size': [8, 4], 'frames_num_fixed': 30, 'frames_num_max': 50, 'frames_num_min': 10, 'sample_type': 'unfixed_unordered', 'type': 'TripletSampler'}, 'transform': [{'type': 'Compose', 'trf_cfg': [{'type': 'RandomPerspective', 'prob': 0.2}, {'type': 'BaseSilCuttingTransform'}, {'type': 'RandomHorizontalFlip', 'prob': 0.2}, {'type': 'RandomRotate', 'prob': 0.2}]}]}
[2023-05-08 10:33:31] [INFO]: {'model': 'Baseline', 'backbone_cfg': {'type': 'ResNet9', 'block': 'BasicBlock', 'channels': [64, 128, 256, 512], 'layers': [1, 1, 1, 1], 'strides': [1, 2, 2, 1], 'maxpool': False}, 'SeparateFCs': {'in_channels': 512, 'out_channels': 256, 'parts_num': 16}, 'SeparateBNNecks': {'class_num': 3000, 'in_channels': 256, 'parts_num': 16}, 'bin_num': [16]}
[2023-05-08 10:33:31] [INFO]: {'dataset_name': 'Gait3D', 'dataset_root': 'Gait3D-sils-64-64-pkl/', 'num_workers': 8, 'dataset_partition': './datasets/Gait3D/Gait3D.json', 'remove_no_gallery': False, 'cache': False, 'test_dataset_name': 'Gait3D'}
[2023-05-20 10:33:32] [INFO]: -------- Train Pid List --------
[2023-05-20 10:33:32] [INFO]: [1234, 1512, ..., 1128]
[2023-05-20 10:33:35] [INFO]: {'lr': 0.1, 'momentum': 0.9, 'solver': 'SGD', 'weight_decay': 0.0005}
[2023-05-20 10:33:35] [INFO]: {'gamma': 0.1, 'milestones': [20000, 40000, 50000], 'scheduler': 'MultiStepLR'}
[2023-05-20 10:33:35] [INFO]: Parameters Count: 19.29030M
[2023-05-20 10:33:35] [INFO]: Model Initialization Finished!

After 60000 Iteration , i use the script test.sh , and got the result :

[Warning] Fused window process have not been installed. Please refer to get_started.md for installation.
[2023-05-22 10:36:37] [INFO]: {'enable_float16': True, 'restore_ckpt_strict': True, 'restore_hint': 60000, 'save_name': 'GaitBase_DA', 'eval_func': 'evaluate_Gait3D', 'sampler': {'batch_size': 8, 'sample_type': 'all_ordered', 'type': 'InferenceSampler', 'batch_shuffle': False, 'frames_all_limit': 720}, 'transform': [{'type': 'BaseSilCuttingTransform'}], 'metric': 'euc', 'cross_view_gallery': False}
[2023-05-22 10:36:37] [INFO]: {'model': 'Baseline', 'backbone_cfg': {'type': 'ResNet9', 'block': 'BasicBlock', 'channels': [64, 128, 256, 512], 'layers': [1, 1, 1, 1], 'strides': [1, 2, 2, 1], 'maxpool': False}, 'SeparateFCs': {'in_channels': 512, 'out_channels': 256, 'parts_num': 16}, 'SeparateBNNecks': {'class_num': 3000, 'in_channels': 256, 'parts_num': 16}, 'bin_num': [16]}
[2023-05-22 10:36:38] [INFO]: {'dataset_name': 'Gait3D', 'dataset_root': 'Gait3D-sils-64-64-pkl/', 'num_workers': 8, 'dataset_partition': './datasets/Gait3D/Gait3D.json', 'remove_no_gallery': False, 'cache': False, 'test_dataset_name': 'Gait3D'}
[2023-05-22 10:36:38] [INFO]: -------- Test Pid List --------
[2023-05-22 10:36:38] [INFO]: [0002, 0007, ..., 3064]
[2023-05-22 10:36:41] [INFO]: Restore Parameters from output/Gait3D/Baseline/GaitBase_DA/checkpoints/GaitBase_DA-60000.pt !!!
[2023-05-22 10:36:41] [INFO]: Parameters Count: 19.29030M
[2023-05-22 10:36:41] [INFO]: Model Initialization Finished!
Transforming: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 6369/6369 [01:33<00:00, 68.11it/s]
[2023-05-22 10:38:14] [INFO]: The test metric you choose is euc.
[2023-05-22 10:38:23] [INFO]: {'scalar/test_accuracy/Rank-1': 44.80000138282776, 'scalar/test_accuracy/Rank-5': 64.89999890327454, 'scalar/test_accuracy/Rank-10': 72.00000286102295, 'scalar/test_accuracy/mAP': 34.86630246754606, 'scalar/test_accuracy/mINP': 18.649806598429514}

According to OpenGait's paper,GaitBase's Rank-1 accuracy on Gait3d should be over 60%, but my test results are quite different

[ChaoFan996]

The relatively small batch size may make this difference, can you enlarge it to [32, 4]?

[enemy1205]

According to your documentation

batch_size: [P,K] where P denotes the subjects in training batch while the K represents the sequences every subject owns. Example:
8
16

Why does adjusting P affect training results? The number of subjects per batch should only affect how long the training takes. Indeed, I also found this problem when training other models in the repository. The batch_size setting will affect the accuracy, and even the same batch_size will be different between single-GPU and multi-GPU training

[github-actions[bot]]

Stale issue message