search

mingyuliutw / UNIT

Unsupervised Image-to-Image Translation
Other
1.98k stars 360 forks source link

Some Puzzle about Shared Latent Representation #23

Closed ssdutHB closed 6 years ago

ssdutHB commented 6 years ago

In the paper, the author assumes that the image in two different domain can be coded into a common latent representation. What I what to know is that if the "Shared Latent Representation" assumption works for unaligned data. Because in the training process, the model must pick arbitrary image from each distribution, and if the data is unaligned, how to make the two images coded in to a common representation? In other words, whether the shared latent space assumption requires the aligned data?

mingyuliutw commented 6 years ago

Yes, the entire work/paper is about unaligned data. Alignment is not required.

mingyuliutw commented 6 years ago

The shared latent space assumption is an assumption saying that if there exists a pair of corresponding images in the two domains then the two images should have the same latent representation. During training, you don’t have aligned data (paired of corresponding images). The whole point of the paper is to use this constraint to encourage a solution that would be likely desired.

hsm207 commented 6 years ago

@mingyuliutw Could you please tell me which part of the model is preventing it from giving the same shared latent representation when the pair of input images is not a pair of corresponding image e.g. image 1 is the number 6 from SVHN and image 2 is the number 7 from MNIST?