Low accuracy: Even without poisoning, the accuracy is only about 10%

[Juli-Eyre]

Even with "is_poison"=false, the accuracy is only about 10%. When "is_poison"=true and batch_size=264, I get the results as follows: When there are adversaries, accuracy of backdoor is about 100%，accuracy without backdoor is increasing as epoch increases. But when there are not adversaries, accuracy is always about 10%

[ybdai7]

try to run it with “python training.py --params = utils/params.yaml" after changing the "is_poison" option to True in the .yaml file, the test accuracy on main task reaches 90% on my machine.

However, i am having another problem. The accuracy of backdoor task cant reach 100%, it always remain to be around 70%. Do you have any ideas how to fix this?

[pythonloveing]

Even with "is_poison"=false, the accuracy is only about 10%. When "is_poison"=true and batch_size=264, I get the results as follows: When there are adversaries, accuracy of backdoor is about 100%，accuracy without backdoor is increasing as epoch increases. But when there are not adversaries, accuracy is always about 10%

Have you solved this problem? I had a similar problem with cifar10 dataset not converging under resnet10 without poisoning

[C-andour]

我也是准确率只有10%，请问是什么问题呢？

[YH-star1]

我也是准确率只有10%，请问是什么问题呢？

Have you solved the problem? I found a very strange problem, in param.yaml where is_posion is false, the global model accuracy is always 10%, but its loss is increasing upward to stop convergence. This is a very strange phenomenon.

[YH-star1]

我也是准确率只有10%，请问是什么问题呢？

Have you solved the problem? I found a very strange problem, in param.yaml where is_posion is false, the global model accuracy is always 10%, but its loss is increasing upward to stop convergence. This is a very strange phenomenon.

[dou-xinyu]

try to run it with “python training.py --params = utils/params.yaml" after changing the "is_poison" option to True in the .yaml file, the test accuracy on main task reaches 90% on my machine.

However, i am having another problem. The accuracy of backdoor task cant reach 100%, it always remain to be around 70%. Do you have any ideas how to fix this?

请问大概多少epoch之后可以跑到90%的准确率呢

[ybdai7]

Hi. I can hardly remember all the details when i try to reproduce the repo as its like two years ago. Maybe you can resort to my repo https://github.com/ybdai7/Chameleon-durable-backdoor, which is the official implementation of an ICML paper and the code is organized based on this repo.

[dou-xinyu]

Thank you so much for your help.

---Original--- From: "Yanbo @.> Date: Sat, Apr 13, 2024 14:03 PM To: @.>; Cc: @.**@.>; Subject: Re: [ebagdasa/backdoor_federated_learning] Low accuracy: Even withoutpoisoning, the accuracy is only about 10% (Issue #15)

Hi. I can hardly remember all the details when i try to reproduce the repo as its like two years ago. Maybe you can resort to my repo https://github.com/ybdai7/Chameleon-durable-backdoor, which is the official implementation of an ICML paper and the code is organized based on this repo.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: @.***>

[liang5211314]

尝试在.yaml 文件中将“is_poison”选项改为 True 后，使用“python training.py --params = utils/params.yaml”运行它，在我的项目中，主任务的测试准确率达到了 90%。 但是，我遇到了另一个问题。后门任务的准确率无法达到 100%，始终保持在 70% 左右。您有什么办法可以规定吗？

请问大概多少epoch之后可以跑到90%准确率呢

我跑的大概在1800轮左右

[dou-xinyu]

尝试在.yaml 文件中将“is_poison”选项改为 True 后，使用“python training.py --params = utils/params.yaml”运行它，在我的项目中，主任务的测试准确率达到了 90%。 但是，我遇到了另一个问题。后门任务的准确率无法达到 100%，始终保持在 70% 左右。您有什么办法可以规定吗？

请问大概多少epoch之后可以跑到90%准确率呢

我跑的大概在1800轮左右

按照源代码，应该是指定一个攻击者并进行随机采样的，这部分代码对应的应该是paper里的重复攻击部分，70%左右的结果没有问题。 100%的后门任务准确率体现在的是paper的单次攻击部分，可以先将yaml文件里的is_poison改为false，如果使用的是cifar-10数据集，在运行到5000轮左右模型收敛，在这个时候进行单单次攻击能达到100%的后门任务准确率。 如果想要重复攻击部分实验达到较高的后门任务准确率，只能增加攻击者的个数，这在paper里也提及了，5个以上的攻击者个数可以达到90%以上的后门任务准确率，这需要修改number of adversary那个选项就可以了。

[lu0802mark]

try to run it with “python training.py --params = utils/params.yaml" after changing the "is_poison" option to True in the .yaml file, the test accuracy on main task reaches 90% on my machine.

However, i am having another problem. The accuracy of backdoor task cant reach 100%, it always remain to be around 70%. Do you have any ideas how to fix this?

您好，请问您还记得最后的后门准确率是如何达到100%的吗

[becomeGiant]

average_shrinkmodels function in helper.py have bug. Specifically, data.add(update_perlayer) have bug. If use type of int cant make model converge. So you should change it into data.float().add(update_per_layer).