Which loss should I look for?

[nile649]

While training which loss should I look for to confirm whether model is actually learning or not?

[lyndonzheng]

@nile649 If you have do some work on GANs, you will find that it will be hard to find the model convergence by the loss. Hence, we judge the model by sampling some visual results.

[nile649]

I know, but loss gives some sense of security that it's learning something. I feel anxious on 0 -3 epoch bro. I have been affected by GAN violence.

[nile649]

Which loss will you suggest to just keep an eye on?

[lyndonzheng]

@nile649 This is the training snapshoot for our current project. If you think the GAN loss affect a lot, you can add the weight for reconstruction, where you need to make sure the img_d not be zero. Besides, you can also do a tiny modification for the Auto_Attn function in base_function.py: if type(mask) != type(None): pre = (1-mask) * x +mask*pre. Currently, we found this is helpful for arbitrary mask. We will release the improved code in the future.

[nile649]

only thing that we noticed from our research for a year has been we shouldn't use strong loss function in GAN, when I train my models I generally keep an eye on the ratio of Generator loss: Discriminator loss.

Also, I am using your project for a inpainting paper of ours, but what's your opinion on replacing progressive gan with style gan architecture? will it even make sense? Since inpainting task is very different from conditional generation...

[lyndonzheng]

Yes, I absolutely agree with you that the inpainting is different from other conditional generation. Because you need to ensure the generated patch very corresponding to the original visible regions. You not only need to generate the content (where style translation, super-resolution, colorlization just modfiy the apperance), but also need to ensure the generated content has global consistency with the visible part. I tried the progressive gan before, but the performance is not so good in our project.

[nile649]

One paper that I am writing this is actually an highlight, leave your paper rest all paper results are over fitting to the datasets that they have been trained on. Especially, contextual attention models are overfittting to the datasets. Treating inpainting as conditional where you have no idea of prior forces GAN to just memorize certain attributes.

Man, I am really impressed by your idea, I did try Cvae but not the other branch and I couldn't imagine myself thinking of the other branch as an enforcer.

I will definitely update you on my paper and results.

Thanks

[lyndonzheng]

The traditional image inpaiting do not know the semantic information. But for current machine learning based method, the semantic information is still very hard to define. If you mask partial object (not face), it's still difficult to fill it. How to make the network real konw the sematic inforamtion rather than remeber the dataset? It is a right thing we need to consider. Hope to see your new paper to throw the new question.