search

rui-yan / SSL-FL

Self-supervised federated learning for medical imaging - IEEE TMI
63 stars 9 forks source link

unable to reproduce the exact result from pre train model #17

Closed chillum-codeX closed 1 year ago

chillum-codeX commented 1 year ago

unable to reproduce the exact result from pre train model , the result which rae in paper can you pls provide assistance i tried to eval using pretrained model that you shared "/covidfl_pretrain_mae_base_central_checkpoint-1599.pth"

workspace/nvidia/pradeep/SSL-FL/code/fed_mae# python run_class_finetune_FedAvg.py --finetune /workspace/nvidia/pradeep/SSL-FL/output/covidfl_pretrain_mae_base_central_checkpoint-1599.pth --model vit_base_patch16 --output_dir /workspace/nvidia/pradeep/SSL-FL/OUTPUT_PATH_FT_mae/ --save_ckpt_freq 50 --batch_size 96 --warmup_epochs 5 --num_workers 4 --E_epoch 1 --weight_decay 0.05 --drop_path 0.1 --reprob 0.25 --mixup 0.8 --cutmix 1.0 --log_dir /workspace/nvidia/pradeep/SSL-FL/log_dir/ --nb_classes 3 --eval --n_clients 5 --num_local_clients -1

++++++ Running Validation ++++++ Test: [ 0/42] eta: 0:14:10 loss: 1.0987 (1.0987) acc1: 20.8333 (20.8333) time: 20.2454 data: 18.3205 max mem: 1003 Test: [10/42] eta: 0:02:29 loss: 1.0987 (1.0987) acc1: 20.8333 (20.9280) time: 4.6828 data: 4.3571 max mem: 1003 Test: [20/42] eta: 0:01:45 loss: 1.0987 (1.0987) acc1: 21.8750 (21.5278) time: 4.0387 data: 3.8719 max mem: 1003 Test: [30/42] eta: 0:00:55 loss: 1.0987 (1.0987) acc1: 20.8333 (21.4046) time: 4.5888 data: 4.4218 max mem: 1003 Test: [40/42] eta: 0:00:08 loss: 1.0987 (1.0987) acc1: 20.8333 (21.2907) time: 3.7900 data: 3.6241 max mem: 1003 Test: [41/42] eta: 0:00:04 loss: 1.0987 (1.0987) acc1: 20.8333 (21.2884) time: 3.7859 data: 3.6241 max mem: 1003 Test: Total time: 0:02:56 (4.2135 s / it)

rui-yan commented 1 year ago

Thank you for bringing this issue to our attention.

  1. There is an issue in data augmentation for COVID-FL that caused the subpar results. This issue has been fixed, and we recommend that you update your code by pulling the latest changes from SSL-FL/code/util/datasets.py and data_utils.py.
  2. To fine-tune Fed-MAE for COVID-FL, please refer to the following bash script: covidfl_central_ssl.sh. This will provide you with the necessary information to proceed with your fine-tuning process. I noticed an error in your command. Specifically, you have specified --n_clients as 5, whereas for COVID-FL datasets, it should be 12 instead.
chillum-codeX commented 1 year ago

Thank you so much for updating.