search

mingyuliutw / UNIT

Unsupervised Image-to-Image Translation
Other
1.98k stars 360 forks source link

Diversity problem [ reduced VAE] #102

Closed asi95 closed 5 years ago

asi95 commented 5 years ago

Hello, thank you very much for your work.

I find your the UNIT model so impressive so i use it in a school project, it returned a good results but the issue that I am having is a diversity problem, given an input image, the output is always the same even with using a VAE (in your case, it's a reduced vae). so what i did is that i added a new share layer that compute the log variance and changed the KL loss function, but i had problems with exploding gradients. so i want to ask if it is possible to make the model returns multiple outputs for the same input ?

Thank you in advance, Regards

mingyuliutw commented 5 years ago

@asi95 Thanks for your feedback. The UNIT method is not good at generating diverse outputs. Please check our follow-up work on MUNIT (basically Mulitmodal UNIT) for generating diverse outputs. https://github.com/NVlabs/MUNIT Ideally, making the KL loss large should encourage diverse output. However, this also makes training harder as you experienced.

asi95 commented 5 years ago

thank you