about cifar100 acc

[imomoe233]

hi ! why the cifar100 benign test acc only abou 55% after 1800 rounds pretrain? when 600epochs, it only 30% benign acc. what's your setting about the pretrain of cifar100?

should i change the snorm from 1 to 0.2 ?

mine ↓ python main_training.py --params utils/cifar100_params.yaml --run_slurm 0 --GPU_id 0 --start_epoch 1 --attack_num 250 --gradmask_ratio 1.0 --edge_case 0

[jhcknzzm]

I guess it may be that the non-iid degree of our FL setting leads to a decrease in benign accuracy. We set dirichlet_alpha: 0.9 (https://github.com/jhcknzzm/Federated-Learning-Backdoor/blob/63ef33665792a6c7243e577cc23b0e0357d10b96/FL_Backdoor_CV/utils/cifar100_params.yaml#L29), the smaller this parameter, the more non-iid the distribution between users. Maybe you can set dirichlet_alpha to 100.0, so that the non-iid degree of user data distribution is small, and the benign accuracy rate may be higher.

When the snorm is small, the model convergence will be very slow. I don't think reducing snorm is a good idea. Usually, we need to use grid search to find the appropriate snorm, for example, try to take 0.1, 1.0, 2.0, 4.0.

In addition, maybe you can first use the central training method to verify how much accuracy the currently used neural model can obtain on Cifar100.

[imomoe233]

oh!i get it ,and i note that only 'defence = true' will use snorm,so i change the snorm when pretrain may not work i will change the dirichlet_alpha to try and how can i run the central training method quickly?maybe let number_of_total_participants=1?i guess

[jhcknzzm]

Yeah, you could set number_of_total_participants=1. But I actually recommend that you rewrite a script to perform Cifar100's central training, which should be easy, and the code for central training Cifar100 is not difficult to write. Because performing centralized training on distributed FL code may bring some bugs (just a less optimistic guess).

[imomoe233]

thanks for your reply