search

taki0112 / Densenet-Tensorflow

Simple Tensorflow implementation of Densenet using Cifar10, MNIST
MIT License
506 stars 196 forks source link

batch normalize #4

Closed ydnaandy123 closed 6 years ago

ydnaandy123 commented 7 years ago

Wow excellent work! Thanks for sharing~ Just a quick question: is there any reason why you don't use the batch_normalization in tf.layers(https://www.tensorflow.org/api_docs/python/tf/layers/batch_normalization)? I'll be glad if you reply me~ thank you!

taki0112 commented 7 years ago

If you look at this url, when you use tf.layers.batch_normalization, you have to define update_ops

However, there were the following issues (Only in my case... NOT all people)

  1. There was a problem that the learning was not good(the accuracy was low).
  2. I personally like simple code. In that case, I wrote the code in a way that I would not add it because I do not think it would be well understood by other users. It seemed like other users did not know something about control_dependencies or get_collection

So I used batch_norm in contrib If you do updates_collections = None, the code is much simpler and easier to understand. Also, the performance has been increased, and if you gave None, it's the same as doing control_dependencies above.

thank you

ydnaandy123 commented 7 years ago

Oh wow! Many thanks for your detailed explanations! You are so friendly~ I'm surprised that the performance will be affected! Thanks for your sharing anyway.

BTW, for the Global_Average_Pooling I use tf.reduce_mean(input_tensor=x, axis=[1, 2], keep_dims=True) What do you think~

taki0112 commented 7 years ago

Oh good idea. Also, using reduce_mean is global_average_pooling.