normalization modification

yd-yin commented 5 years ago

Checklist

[x] I've tested that my changes are compatible with the latest version of Tensorflow.
[x] I've read the Contribution Guidelines
[x] I've updated the documentation if necessary.

Motivation and Context

BatchNorm:

(code262) mean, var = tf.nn.moments(inputs, self.axes) When calculating the mean and variance of inputs, keepdims is set to False by default, returning a one-dimensional tensor. This tensor will be broadcasted along with the last dimension of inputs when doing batch_normalization, which is not suitable for channels first version.

InstanceNorm

The previous version of class InstanceNorm only supports InstanceNorm2d and static mode.

Description

Bug fixing of BatchNorm A new version of InstanceNorm Add *1d/2d/3d

zsdonghao commented 5 years ago

remember to change the changelog.md

zsdonghao commented 5 years ago

also, remember to correct the python format:

yapf -i xxx.py
# or for a folder
yapf -i --recursive folder

tensorlayer / TensorLayer