miss bias in the first layer of the discriminator

[udibr]

referring to https://github.com/Newmu/dcgan_code/blob/master/faces/train_uncond_dcgan.py#L118 I tried adding my own bias just before the lrelu, but it is hard to tell if it helps:

b = bias_ifn((ndf), 'db')
...
h = lrelu(dnn_conv(X, w, subsample=(2, 2), border_mode=(2, 2))+b.dimshuffle('x', 0, 'x', 'x'))

[axzml]

@udibr , I have some thoughts but I don't know whether they are correct.

Before using dnn_conv(X, w, subsample = (2, 2), border_mode=(2, 2)), the author used X = conv_cond_concat(X, yb), but in the deep learning tutorial ( http://deeplearning.net/tutorial/lenet.html#lenet ), X, the input, needs not to concatenate with yb. Meanwhile, yb has a special structure, so when we use dnn_conv(X, w, subsample = (2, 2), border_mode=(2, 2)), the result is something like conv(X1, w) + b.dimshuffle('x', 0, 'x', 'x'), here X1 is not concatenate with yb.

[udibr]

I am guessing the two ways of adding bias are equivalent. But if you search for conv_cond_concat you only find it in the mnist model so it is missing from the faces model. Perhaps there is a reason

[axzml]

Thank you for rereading the code. I find that in the faces model the batchnorm function is used with a bias parameter. However bias is still missing in some layers, like in discrim model, line 118.