About Loss Function

[timho102003]

Hi Im new to GAN and also studying paper DRGAN. Im wondering that the loss function ur using in the implementation is the same as in the paper(GAN has it two player loss calculation but ur using cross entropy instead)? If Im wrong, please correct me tks!!!!!

loss_criterion = nn.CrossEntropyLoss() loss_criterion_gan = nn.BCEWithLogitsLoss()

[kayamin]

Hi, thank you for asking. DR-GAN's discriminator has 3 roles. Distinguish input face image ID _(LId), its pose _(Lpose) and whether it is real or synthesized image _(Lgan).

About their loss functions are explained as follows in the paper

D is trained to optimize Eqn. 7 by adding a fully connected layer with the softmax loss for real vs. fake, identity, and pose classifications respectively.

Discriminator returns Nd + 1(real/syn) + Np vector. Nd part is used for calculating _LId and Np part is used for _Lpose. Left 1 dimension is used for _L_gan_. I applied nn.CrossEntropyLoss() for _Lid and _L_pose to take LogSoftmax and nn.BCEWithLogitsLoss() for L_gan_ to take BCELoss with sigmoid layer. (There would be no need of softmax for binary classification) (･ω･)bｸﾞｯ

[timho102003]

Hi thanks for ur prompt attention, **1. CrossEntropy --> H(A,B) = -P(A)log(P(B))

2. GAN --> minmaxV(D,G) = E[log(D(x))]+E[log(1-D(G(x)))] What my understanding is that P(A)log(P(B)) is something like E[log(D(x))]**?? Thats why ur using nn.CrossEntropyLoss() as the loss function. I'm not quite sure if my understanding is correct

The second question is that DRGAN Team also release another Paper "Representation Learning by rotating Your Face" which is sort of an enhanced version of DRGAN. The paper mentioned a new way of improving Genc by iteratively switching (did't give an exact iteration)the models between Genc and D. As in ur code, there is also a D_strong so I'm wondering that if u also implement this part to ur code? Looking forward to ur reply,thanks!!!

[kayamin]

Yes. Here(http://pytorch.org/docs/master/nn.html), torch.nn.CrossEntropyLoss is explained as

loss(x, class) = -log(exp(x[class]) / (\sum_j exp(x[j])))
               = -x[class] + log(\sum_j exp(x[j]))

Softmax terms within log corresponds to what you mention as D(x). So this represents -log(D(x)). Calculating -log(D(x)) for every x in the data mean taking E[-log(D(x))], beacause x can be considered as sampled from data distribution p(x).

My implementation is based on "Representation Learning by rotating Your Face" but still haven't implemented "Model switch".

D_strong is used for controlling leaning ratio of D & G by checking D's output accuracy in Id, gan, pose as written in "Disentangled Representation Learning GAN for Pose-Invariant Face Recognition"

In conventional GANs, Goodfellow et al. [9] suggest to alternate between k (usually k = 1) steps of optimizing D and one step of optimizing G. This helps D to maintain near-optimal solution as long as G changes slowly. How- ever, in DR-GAN, D has strong supervisions thanks to the class labels. Thus, in later iterations, when D is close to the optimal solution, we update G more frequently than D, e.g., 4 steps for optimizing G and 1 for D.

[timho102003]

Thanks for your detail information. So far, I've modified the dataloader part and allow the library pandas to load image by using .csv file instead of .npy files with Multi-PIE Database as training data( roughly 400,000 images ). Another question is that paper only mentioned about the momentum=0.5; however, your recent modification add in the beta parameters. Many Thanks!

[kayamin]

In the paper, written as

Our implementation is extensively modified from a publicly available implementation of DC-GAN ~ Adam optimizer [62] is used with a learning rate of 0.0002 and momentum 0.5

Then in the DC-GAN paper

Additionally, we found leaving the momentum term β1 at the suggested value of 0.9 resulted in training oscillation and instability while reducing it to 0.5 helped stabilize training.

So I just applied beta1 = 0.5 for the Adam optimizer.

[timho102003]

I also implement dataloader part and allow the library pandas to load image by using .csv file instead of .npy files to Generate_Image.py. However, a new issue pop out which I can't figure it out the reason.

1. In training phase (take single-DRGAN as example):

generated = G_model(batch_image, pose_code, fixed_noise) # Generatorでイメージ生成 to generate fake images

2. While in -generate phase (take single-DRGAN as example):

main.py:

# Load Model (in my case --> snapshot/single/epoch99) D = torch.load('{}_D.pt'.format(args.snapshot)) G = torch.load('{}_G.pt'.format(args.snapshot))

Generate_Image.py:

# Generatorでイメージ生成 generated = G_model(batch_image, batch_pose_code, fixed_noise) When run to the code generated = G_model(batch_image, batch_pose_code, fixed_noise) in Generate_Image.py, an error occurred (The reason that I can't figure it out is that in training phase, it also get thru the same process and do not trigger the error down below): ERROR-->

File "main_v2.py", line 149, in features = Generate_Image(D, G, args) File "/home/avlab/TIM/DRGAN/Generate_Image.py", line 75, in Generate_Image generated = G_model(batch_image, batch_pose_code, fixed_noise) File "/usr/local/lib/python3.5/dist-packages/torch/nn/modules/module.py", line 325, in call result = self.forward(*input, kwargs) File "/home/avlab/TIM/DRGAN/model/single_DR_GAN_model.py", line 255, in forward x = self.G_enc_convLayers(input) # Bxchx96x96 -> Bx320x1x1 File "/usr/local/lib/python3.5/dist-packages/torch/nn/modules/module.py", line 325, in call result = self.forward(*input, *kwargs) File "/usr/local/lib/python3.5/dist-packages/torch/nn/modules/container.py", line 67, in forward input = module(input) File "/usr/local/lib/python3.5/dist-packages/torch/nn/modules/module.py", line 325, in call result = self.forward(input, kwargs) File "/usr/local/lib/python3.5/dist-packages/torch/nn/modules/padding.py", line 77, in forward return F.pad(input, self.padding, 'constant', self.value) File "/usr/local/lib/python3.5/dist-packages/torch/nn/modules/module.py", line 366, in getattr type(self).name, name)) AttributeError: 'ZeroPad2d' object has no attribute 'value'

Link to my code(Google Drive): https://drive.google.com/open?id=114VmFb6wddx7lQmi_a-EsAMMuR2toUcK Looking Forward to ur reply!!!

[timho102003]

Hi again When training DRGAN, I use two ways to load training data

1) Model1 --> Your code with .npy (small scale data --> 2000 images) 2) Model2 --> My modified code using .csv to load training data while training (large scale data --> 400,000)

While training, everything works just fine, the image data format, pose label... are all the same however in -generate mode (I also try two different code):

1) Load Model1 and generate image using ur code(.npy load image to be generate) --> It works(V) 2) Load Model2 and generate image using my code(.csv load image to be generate) --> Doesn't work(X) and pop out the error in my last comment.

So I'm wondering if theres any problem while going thru training part or is there any way to package large scale data thru .npy file since there is a limit for numpy Many Thanks!

[timho102003]

Problem Solved! Still thanks for ur hard work!

[Yagami123]

@timho102003 So, can you share how do you solve this problem? My project also occurs.

[zhengrongsheng]

@timho102003 Would you show me the way to load training data? And it would be helpful if you can provide me you training data, such as images.npy, ids.npy, yaws.npy. I am looking forward to you reply!

[zhengrongsheng]

@kayamin Hi When i run your code, i get a problem. Because i do not have any training data, so i want to ask for the data, such as images.npy, ids.npy, yaws.npy. thank you very much!

[timho102003]

@zhengrongsheng You can follow the tutorial from Pytorch --> https://pytorch.org/tutorials/beginner/data_loading_tutorial.html

[NomiMalik0207]

@timho102003 have you produced the results?