ZSHsh98 / EPS-AD

This is the source code for Detecting Adversarial Data by Probing Multiple Perturbations Using Expected Perturbation Score (ICML2023).
33 stars 2 forks source link

The auroc of 128x128 diffusion is very poor, not reaching the level of 256x256 diffusion (not class conditional) #6

Open GsiWang opened 5 months ago

GsiWang commented 5 months ago

I use loader to read down sampled images of size 128, and then use a 128x128 diffusion (weights downloaded from https://github.com/openai/guided-diffusion )Why is the effect not as good as 256x256 diffusion (not class conditional)? Auroc is only about 0.5, which is equivalent to random guessing。 The parameters I used were basically the same, without using the category features of the diffusion model. I saw in the paper that there is 128*128 diffusion model and it has good results. Is it because I used class conditional instead of not class conditional that the quality of the generated diffusion images is not good, which leads to the inability to calculate scores well

ZSHsh98 commented 5 months ago

Yes, you are right. Class conditional diffusion models can be difficult to work for EPS. In this case, the model will be given a random class to obtain an EPS for an image, which is not able to be adapted for detecting adversaries with other classes.