Discriminator update rule in domain adaptation and domain translation

[adrienchaton]

Dear M. Liu,

I am studying your paper that I found very interesting, thank you for sharing your research ! I read the other issues/questions asked, where you pointed that some explanations were not given in the paper, about that I would like to ask you more details about the domain adaptation experiment.

Could you explain the discriminator update rule and more precisely when you compute that please ? feature_loss_a = self._compute_ll_loss(fake_feat_ab - fake_feat_aa, dummy_variable) feature_loss_b = self._compute_ll_loss(fake_feat_ba - fake_feat_bb, dummy_variable) (cocogan_trainer_da.py lines 102-103)

Also, my experiments focus more on domain translation than domain adaptation. For that purpose, is this update rule still relevant ?

Thank you for your answer. Adrien

[mingyuliutw]

feature_loss_a and feature_loss_b are the additional losses used in the domain adaptation experiments in the main paper. The main idea is that if a pair of generated images in two domains are in correspondence we would like the domain-specific discriminators to extract a common feature representation for the two images. Minimizing the feature losses is a way to achieve the goal.

[adrienchaton]

Thank you for your quick answer and these explanations. Does this would also be relevant for domain translation like let's say cat2tiger where a part of facial features may also be corresponding (there are eyes, ears, mouth etc.) ?

Or do we see these features as encoded together in the shared top layers of the autoencoder and then discriminators will only serve the purpose of challenging the domain specific features (like the size of the face, the fur, the shape of the nose) so they are totally separate in that case (unlike domain adaptation) ?

My feeling is that the second is more right regarding domain translation.

I am asking because I gathered two datasets of cropped faces around 10k pictures each in 200px*200px, one domain is human faces, other is cat faces. I try to follow your implementation, it does reconstruct separately the images in their own domain but they are only very slightly altered when translating (taking the other generator) and it doesn't look like it tried to fake the other domain.

On the contrary, when I tried first MNIST-USPS domain adaptation, I had the separate domain reconstructed and the images adapted in the sense that MNIST digits adapted to USPS domain were looking more rough and USPS digits adapted to MNIST would look more clear (which is consistent with the overall difference between MNIST and USPS digits) while keeping the original digit shape.

Thank you in advance for any recommendation or remark you may have regarding my trial into domain translation.

[adrienchaton]

this is what I referred to as domain adaptation between MNIST and USPS which is rather consistent with what I thought of the experiment

[adrienchaton]

this is what I referred as my domain translation experiment, the third row should show the translated image from the other domain (eg. left sample of domain 2 translated to domain 1 in the bottom left of domain 1 plot) but it doesn't occur hence I am wondering what is preventing the human generator to decode cat latent representation into humain domain, and accordingly the cat generator to translate human latent representation into cat sample ..