search

wiseodd / generative-models

Collection of generative models, e.g. GAN, VAE in Pytorch and Tensorflow.
http://wiseodd.github.io
The Unlicense
7.33k stars 2.04k forks source link

Gibbsnet issue #44

Open LukasMosser opened 6 years ago

LukasMosser commented 6 years ago

I've been running the new gibbsnet implementation, but I can't find it to converge with the given settings.

Do you have any examples on what the results should look like?

wiseodd commented 6 years ago

My point on that implementation is to get the intuition of GibbsNet, and how to quickly implement it, esp. given ALI/BiGAN implementation. Given this intuition feel free to experiment with another architecture and hyperparams.

Cheers, Agustinus

-------- Original Message -------- Subject: [wiseodd/generative-models] Gibbsnet issue (#44) Local Time: December 25, 2017 12:32 PM UTC Time: December 25, 2017 11:32 AM From: notifications@github.com To: wiseodd/generative-models generative-models@noreply.github.com Subscribed subscribed@noreply.github.com

I've been running the new gibbsnet implementation, but I can't find it to converge with the given settings.

Do you have any examples on what the results should look like?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

LukasMosser commented 6 years ago

Yes, I understand. I just wanted to point out that it seems like this doesn't converge though. After 2000 iterations: 200

The loss of the generator is very high (18) and discriminator loss goes to zero. Have you gotten it to work with your code? I am using the code as is, no modifications.

wiseodd commented 6 years ago

Yeah, it's also what I experienced. It won't generate good result and the loss is blowing up. I suspect we need to have stronger networks + much lower learning rate.

However, as I'd like to have everything in this repo with very simple architecture, I decided to push it as is :)

-------- Original Message -------- Subject: Re: [wiseodd/generative-models] Gibbsnet issue (#44) Local Time: December 25, 2017 5:27 PM UTC Time: December 25, 2017 4:27 PM From: notifications@github.com To: wiseodd/generative-models generative-models@noreply.github.com Agustinus Kristiadi kristiadi@protonmail.com, Comment comment@noreply.github.com

Yes, I understand. I just wanted to point out that it seems like this doesn't converge though. After 2000 iterations: 200

The loss of the generator is very high (18) and discriminator loss goes to zero. Have you gotten it to work with your code? I am using the code as is, no modifications.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

LukasMosser commented 6 years ago

I tried an 8 layer MLP for each network, as suggested in the paper with lr of 1e-5. Still no convergence. Would be nice to get a version working. If I make any progress I'll update.