search

mariolew / Deep-Alignment-Network-tensorflow

A re-implementation of Deep-Alignment-Network using TensorFlow
111 stars 30 forks source link

question about conv+bn #1

Open guyafeng opened 6 years ago

guyafeng commented 6 years ago

1、has this code reached the same accuracy as the theano ? 2、 i saw in theano document , bn(conv(relu)) means conv + bn +relu like this: >>> from lasagne.layers import InputLayer, DenseLayer, batch_norm

from lasagne.nonlinearities import tanh l1 = InputLayer((64, 768)) l2 = batch_norm(DenseLayer(l1, num_units=500, nonlinearity=tanh)) This introduces batch normalization right before its nonlinearity: from lasagne.layers import get_all_layers [l.class.name for l in get_all_layers(l2)] ['InputLayer', 'DenseLayer', 'BatchNormLayer', 'NonlinearityLayer']

 so , which is right about bn.
mariolew commented 6 years ago

Hi, @guyafeng Now, answer the questions.

  1. Since I have no access to a GPU this month, I haven't trained a model yet. But the I've tested the correctness of the layer functionalities. If you have a GPU, can you spend some time and try to train a model? I also want to know whether this code can reach the same performance as the paper described.

  2. There is no consensus yet on where to best place the BatchNorm layer. https://stackoverflow.com/questions/47143521/where-to-apply-batch-normalization-on-standard-cnns

You can refer to the link for more details.