search

tensorflow / tensor2tensor

Library of deep learning models and datasets designed to make deep learning more accessible and accelerate ML research.
Apache License 2.0
15.5k stars 3.49k forks source link

*bug* batch normalization not appropriately applied #689

Open cbockman opened 6 years ago

cbockman commented 6 years ago

Description

Batch normalization gets applied here:

https://github.com/tensorflow/tensor2tensor/blob/master/tensor2tensor/layers/common_layers.py#L632

Per https://www.tensorflow.org/versions/master/api_docs/python/tf/layers/batch_normalization, training=True needs to be set while training (unless something really funky is happening upstream of apply_norm?)

Is a relatively easy fix; happy to issue PR, assuming I am not misunderstanding.

TensorFlow and tensor2tensor versions

t2t master

rsepassi commented 6 years ago

Would welcome a PR adding a is_training kwarg to apply_norm and updates around the codebase.