search

zhmzm / FLDetector_pytorch

14 stars 1 forks source link

fld IndexError #2

Open fiora6 opened 1 year ago

fiora6 commented 1 year ago

When set defence='fld', this line args.old_update = args.old_update_list[idx] has IndexError. Maybe it's the old_update_list is an empty list. And the defence 'tr_mean' you mention in options.py seems not occur in main_fed.py, could you show it?

zhmzm commented 1 year ago

Hi, can you provide your input and the complete error log? 'tr_mean' is deleted because it is not effective for backdoor attacks.

fiora6 commented 1 year ago

IID: 1 Dataset: mnist Model: an one hidden-layer model Model Init: None Aggregation Function: fld Attack method: labelflip Attack tau: 0.8 Fraction of malicious agents: 20.0% Poison Frac: 0.2 Backdoor From -1 to 0 Attack Begin: 0 Trigger Shape: square Trigger Position X: 27 Trigger Position Y: 27 Number of agents: 100 Fraction of agents each turn: 100(100%) Local batch size: 500 Local epoch: 1 Client_LR: 0.01 Client_Momentum: 0.9 Global Rounds: 500

Traceback (most recent call last): File "/home/fws/code/fltrust/main_fed.py", line 231, in mal_weight, loss, args.attack_layers = attacker(malicious_list, attack_number, args.attack, dataset_train, dataset_test, dict_users, net_glob, args, idx = m_idx) File "/home/fws/code/fltrust/models/Attacker.py", line 161, in attacker args.old_update = args.old_update_list[idx] IndexError: list index out of range

zhmzm commented 1 year ago

'Attack.py' is updated now. https://github.com/zhmzm/FLDetector_pytorch/blob/165ea843568134edec052bbfa316cbad4979f054/models/Attacker.py#L158-L179 Can you update this function in your repo and try to run it again?

fiora6 commented 1 year ago

Now it works. And I wonder why the MNIST accuracy is around 90%, it usually achives very high in papers.

zhmzm commented 1 year ago

FLDetector in this repo may not be able to achieve as high ACC as the original paper. I am afraid the code is not totally correct. You can find if is there any error in implement of FLDector. And please report it if possible. I am not sure what is your experiment settings. I cannot analyze how to improve the accuracy directly. But there are several general methods you can improve the accuracy.

  1. I notice the model architecture you used is different from the papers, you can try to use the same model?
  2. Run more epochs?
  3. Include fewer malicious clients.