Open OwalnutO opened 6 years ago
Unfortunately I have the same problem, too. Do you solve it?
Did you do normalization on your own dataset before training?
I have this same problem in tensorflow. Is the output layer of the discriminator really supposed to be linear? If I use tanh I get reasonable values (d_loss=[-2,0], g_loss=[-1,1]) and the model trains nicely.
I have this same problem in tensorflow. Is the output layer of the discriminator really supposed to be linear? If I use tanh I get reasonable values (d_loss=[-2,0], g_loss=[-1,1]) and the model trains nicely.
@markemus could you comment on how exactly you solved this? I'm having a similar issue with 3x256x256 images using PyTorch, what exactly did you change? My loss is pretty erratic.
I have this same problem in tensorflow. Is the output layer of the discriminator really supposed to be linear? If I use tanh I get reasonable values (d_loss=[-2,0], g_loss=[-1,1]) and the model trains nicely.
@markemus could you comment on how exactly you solved this? I'm having a similar issue with 3x256x256 images using PyTorch, what exactly did you change? My loss is pretty erratic.
I used tanh as an activation function on the output layer, and it worked well. But I suspect it slows down training.
I have this same problem in tensorflow. Is the output layer of the discriminator really supposed to be linear? If I use tanh I get reasonable values (d_loss=[-2,0], g_loss=[-1,1]) and the model trains nicely.
@markemus could you comment on how exactly you solved this? I'm having a similar issue with 3x256x256 images using PyTorch, what exactly did you change? My loss is pretty erratic.
I used tanh as an activation function on the output layer, and it worked well. But I suspect it slows down training.
Sorry, what exactly did you use tanh on? The critic output?
@123epsilon yes, I used tanh as an activation function on the critic output layer to map the critic output from (-inf, inf) to [-1, 1]. With that the model was stable and trained well, but without it it was very unstable and did not converge IIRC.
However I suspect that mapping the output to that narrow range may have slowed down training- if possible it might be better to solve this issue by adjusting learning rates or using a different optimizer, but I wasn't able to find a set of hyperparameters that worked. Tanh, on the other hand, worked beautifully.
I reproduce the main code with tf.slim and try to train a wgan-gp model on my own dataset, image size is 128*128. But I get very large loss values during the training process, both G and D. Such as
d_loss: 514810.00000000, g_loss: -49040.92578125 d_loss: 76364.06250000, g_loss: -404828.46875000
And this problem starts very early. Could anyone give me some suggestions? Thanks!!!!!