search

CuriousAI / ladder

Ladder network is a deep learning algorithm that combines supervised and unsupervised learning
MIT License
516 stars 142 forks source link

Batch normalization #26

Open rslprpr opened 7 years ago

rslprpr commented 7 years ago

Could some one tell me what will happen if I use normal batch normalization not the one that developed by ladder network? And I have also don't understand why in evaluation process test and validation needs the statistics of train data?

hotloo commented 7 years ago

Hi!

Could you be more specific about what you meant by "normal batch normalization"?

In evaluation, because the sample normally comes one at a time, the statistics will be noisy. This noise will most likely destroy the network performance. Therefore, one often uses the batch statistics from training data which is much more robust to these type of problem.

Please let me know if I am not able to answer your question or answered it wrong. Hope this helps!