cifar10

[renhl717445]

Could you please send the training and attack code of cifar10? I found that many papers on GAN generation adversarial samples used cifar10 data set for experiments, but basically there was no code that could be directly reproduced. I ran the code in advGan, and the effect was quite different from that of the original paper, so it was useless to adjust relevant parameters. Your paper also makes a comparison with advGan, and the effect is better.

[Scintillare]

Hi, @renhl717445! Firstly, I would like to mention that it's not my paper - this repo is not official. I just modify and combine code from the official advGan repo that uses TensorFlow and from another pytorch implementation to make it more convenient for my own research. And then introduce 2-phase training used in AIGan (advGan++).

My repo basically uses MNIST, but you can easily replace it with cifar: modify the used dataset in train_target_model.py and replace the target model with your own. Or just modify the MNIST_target_model input layer (replace 1 channel with 3).

Also, please pay attention that AIGan (advGan++) version is slightly different from advGan. The training procedure was changed according to the paper, it has 2 phases. You may check my other repos, I also have a fork from mathcbc/advGAN_pytorch with my fixes.

If your ablation study will be successful you may share the results, I'll add them to the readme :)

[Scintillare]

Oh, and also the same modifications (dataset and model) should be done for test_adversarial_examples.py (attack code)

[renhl717445]

Hi, @Scintillare, I am very glad to receive your reply. I have been changing this code for a long time, but no one has replied. I have tried many ways to modify the code, and the effect is not very ideal.

First of all, I modified the MNIST_target_model input layer (replace 1 channel with 3), but this will cause the two dimensions to not match when calculating loss, logits_model = 400*10, but train_labels = batch_size = 256.

At the same time, resnet18 provided by you is also used as the target model training, but the training accuracy is only about 80%, and different epochs are adjusted. I don't know if we also need to transform the data of cifar10.

Finally, I found that you have used imagenet and resnet50 to do relevant experiments. I want to ask how it works?

Looking forward to your reply!

[Scintillare]

@renhl717445 Hm, I'll try to modify it for cifar a bit later.

Yes, I implemented GAN architecture for big images from paper and tried it with 224x224 picture on my dataset. It worked for me. But actually sometimes it's hard to train GAN on bigger images. It may not converge and need restarts and tunning.. And it's take a lot of time to train this small GAN) Also, I provide code for experiments with activation functions. This was helpful for me! I tried them only on mnist, but convergence was much faster (less epochs from zero to needed results, like 33 vs 200). Adaptive one (Rational) was the best.

You may also be interested in RobGAN if you research this field, they solved some of this problems

[renhl717445]

Hi, @Scintillare， Thank you very much for your recommendation of RobGAN. I will read it carefully. At the same time, I would like to ask, in the mnist experiment of advGAN, I found that L_D_loss keeps declining and soon reaches 0, but L_G_fake_loss does keep rising and then approaches 1, while other losses keep declining. Is this a correct phenomenon? In my understanding, L_G_fake_loss should also keep falling and finally save stable, is this caused by too strong discriminator?

[renhl717445]

Hm，I am now running into some problems. Many of the tags generated against the sample are the same. Is this causing a model crash? Have you met it? Do you have any suggestions for modification

At 2023-05-09 17:49:58, "Diana Malyk" @.***> wrote:

@renhl717445 Hm, I'll try to modify it for cifar a bit later.

Yes, I implemented GAN architecture for big images from paper and tried it with 224x224 picture on my dataset. It worked for me. But actually sometimes it's hard to train GAN on bigger images. It may not converge and need restarts and tunning.. And it's take a lot of time to train this small GAN) Also, I provide code for experiments with activation functions. This was helpful for me! I tried them only on mnist, but convergence was much faster (less epochs from zero to needed results, like 33 vs 200). Adaptive one (Rational) was the best.

You may also be interested in RobGAN if you research this field, they solved some of this problems

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: @.***>

[Scintillare]

Hi, @renhl717445! Sorry for ghosting you, I was busy these days.

So, I tried to adapt it for cifar10. It was a few different approaches, but they all resulted in a failure. That's how the plots looked like last time:

Few comments: I modified the generator's decoder because it produced a checkerboard pattern, but that generator was also not strong enough. It seems that for cifar you need a stronger generator, maybe you need to modify some of the generators for Imagenet (now you can't directly use them, because they require pictures bigger than 224*224). But you may also try a few restarts with my last setup, maybe try to modify weights for losses (reduce beta, for example).

[Scintillare]

@renhl717445 Also, about your mnist question. I tested with mnist, that's my logs:

It's a short example, but it should continue like that. So, what we can see there:

• Face accuracy and adversarial loss decrease. They will look almost the same, accuracy just for additional info.
• Ideally, loss_D_fake, loss_D_real and loss_G_fake should be 0.25. But that is good enough if they are just stable around some value. But for the mnist it is almost always 0.25.
• Loss perturbation should decrease, but that's normal if it's stacked at some point if the images look good.
• loss_G it's a general loss for the GAN, that one is a combination of all the previous, so it should somehow decrease.

If you have some other behaviour, try to restart. If it's always not like that, it needs troubleshooting.

[Scintillare]

@renhl717445 is the samples just look the same? or it's actually the same data? how do you compare them? Probably, its mode collapse

[renhl717445]

I am testing the advgan experiment now, and the labels of the generated samples are basically the same. Is this normal?

At 2023-06-11 01:21:41, "Diana Malyk" @.***> wrote:

@renhl717445 is the samples just look the same? or it's actually the same data? how do you compare them? Probably, its mode collapse

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: @.***>