Results: KRUM and MultiKrum vs blackbox attack

[anupamme]

We want to measure how do black-box attacks do against KRUM and MultiKrum defenses. For reading see the last paragraph of page 9 (of the paper). I am quoting the excerpt:

"Since the black-box attack does not have any norm difference constraint, training over the poisoned dataset usually leads to a large norm difference. Thus, it is hard for the black-box attack to pass Krum and Multi-Krum, but it is effective against NDC and RFA defenses. This is presumably because the attacker can still slowly inject a part of the backdoor via a series of attacks."

We want to verify these claims.

[anupamme]

The experiment setup we want to test is:

Tasks:

Task 1: CIFAR10 (Image Classification) | VGG-9 Task 2: EMNIST (Digit Classification) | LeNet Task 3: Sentiment 140 (Sentiment Classification) | LSTM Task 4: Reddit Dataset (Next word prediction) | LSTM

[anupamme]

Sentiment and Reddit dataset can be taken from here: https://github.com/TalwalkarLab/leaf/tree/master/data

[somiljain7]

hey! i am getting this error when i run run_simulated_averaging.sh file Traceback (most recent call last): File "simulated_averaging.py", line 165, in mode="targetted-task", dataset=args.dataset, poison_type=args.poison_type) File "C:\Users\Somil Jain\Downloads\Federated_AoT-master\fl_trainer.py", line 243, in test test_loss += criterion(output, target).item() File "C:\Users\Somil Jain\Anaconda3\lib\site-packages\torch\nn\modules\module.py", line 722, in _call_impl result = self.forward(*input, **kwargs) File "C:\Users\Somil Jain\Anaconda3\lib\site-packages\torch\nn\modules\loss.py", line 948, in forward ignore_index=self.ignore_index, reduction=self.reduction) File "C:\Users\Somil Jain\Anaconda3\lib\site-packages\torch\nn\functional.py", line 2422, in cross_entropy return nll_loss(log_softmax(input, 1), target, weight, None, ignore_index, None, reduction) File "C:\Users\Somil Jain\Anaconda3\lib\site-packages\torch\nn\functional.py", line 2218, in nll_loss ret = torch._C._nn.nll_loss(input, target, weight, _Reduction.get_enum(reduction), ignore_index) RuntimeError: Expected object of scalar type Long but got scalar type Int for argument #2 'target' in call to _thnn_nll_loss_forward 170500096it [03:01, 940706.36it/s]

[anupamme]

It is perhaps because of your pytorch version

It is running fine for me and I am using this:

torch==1.5.1 torchfile==0.1.0 torchvision==0.6.1