I've noticed an inefficiency in the CGAN code. When we append the one-hot encoded labels to the image, they influence the training gradients a lot. Instead, I've noticed that scaling the one-hot encoded labels down by a factor of 0.01 or even 0.001 helps the CGAN converge around twice as fast.
That would mean changing opts.py's conv_cond_concat function. My hack was to change return concat([x, y*tf.ones([x_shapes[0], x_shapes[1], x_shapes[2], y_shapes[3]])], 3) to return concat([x, 0.001*y*tf.ones([x_shapes[0], x_shapes[1], x_shapes[2], y_shapes[3]])], 3) and that worked well for me. I'm not too sure about in general though, perhaps try adding batch norm?
I've noticed an inefficiency in the CGAN code. When we append the one-hot encoded labels to the image, they influence the training gradients a lot. Instead, I've noticed that scaling the one-hot encoded labels down by a factor of 0.01 or even 0.001 helps the CGAN converge around twice as fast.
That would mean changing opts.py's conv_cond_concat function. My hack was to change
return concat([x, y*tf.ones([x_shapes[0], x_shapes[1], x_shapes[2], y_shapes[3]])], 3)toreturn concat([x, 0.001*y*tf.ones([x_shapes[0], x_shapes[1], x_shapes[2], y_shapes[3]])], 3)and that worked well for me. I'm not too sure about in general though, perhaps try adding batch norm?