TrojanZoo provides a universal pytorch platform to conduct security researches (especially backdoor attacks/defenses) of image classification in deep learning.
...
mark, loss = self.optimize_mark(label, verbose=verbose, **kwargs)
if verbose:
asr, _ = self.attack.validate_fn(indent=4)
if not self.mark_random_pos:
select_num = self.attack.mark.mark_height * self.attack.mark.mark_width
overlap = mask_jaccard(self.attack.mark.get_mask(),
self.real_mask,
select_num=select_num)
prints(f'Jaccard index: {overlap:.3f}', indent=4)
else:
asr, _ = self.model._validate(get_data_fn=self.attack.get_data,
keep_org=False, poison_label=True,
verbose=False)
...
the get_data method only returns the data of the default target class (label 0). This infects defenses like Neural Cleanse, ABS. The outputs look like the following for a benign model:
It seems to be the case and modifying target_class in the loop shall fix the problem.
I'll later check details. The experiment results in paper are not affected because they are based on the old codes.
I found that in the implementation of ModelInspection class, the target_class is not changed when computing ASR for each reversed trigger.
The code computing the ASR: (from https://github.com/ain-soph/trojanzoo/blob/main/trojanvision/defenses/abstract.py#L366)
the get_data method only returns the data of the default target class (label 0). This infects defenses like Neural Cleanse, ABS. The outputs look like the following for a benign model:
Only the ASR of the default target class is meaningful, while the ASRs of other classes are almost zero.