what's the role of reconstruction loss for image Synthesis ?

[qinhuafeng]

First, it is amazing work. Thank you to share your work. After reading your work, I am not clear for the role of the reconstruction loss. In your work, I known that one of role is that we create a sequence of images with gradually changed composition based on z_rec nosie， but I 'd like to known its contribution for improving quality of synthesis image. In many application, the reconstruction loss of MSE as global loss enables the model to produce consistent results. I am not sure that adapting the reconstruction loss here has similar effect or not. Can we remove the reconstruction loss for image synthesis?

[jwb95]

My guess at it is: If you start off the training at an initial resolution such that the receptive field of the first networks (which is defaulted to 11) captures the whole image at said resolution, the reconstruction loss is not necessary. The adversarial loss should in this case force the generator to produce the whole image. However, if this is not the case because the receptive field is smaller than the image, the generator is likely not to learn a distribution in which the original image is contained. By adding the reconstruction loss the generator is forced to include a representation of the original image in the latent space.

Also I could imagine that the mse loss speeds up the training (memorization of features in G) at the cost of fitting the data distribution better - but since the image quality is still very good this seems to be a good choice.

Any comments by the author would be appreciated! :)

[iris112358]

I tried a model without the reconstruction loss. It gives the following result on the balloon image This is quite interesting. I wonder why GAN is not able to correct the colors.