Closed yunlingz closed 4 years ago
https://github.com/yunjey/pytorch-tutorial/blob/master/tutorials/02-intermediate/convolutional_neural_network/main.py#L41-L42
nn.ReLU(), nn.BatchNorm2d(...),
BN should be used after ReLU. Features might be truncated by non-linearity like ReLU, so BN is used to normalize the distribution of features.
https://github.com/yunjey/pytorch-tutorial/blob/master/tutorials/02-intermediate/convolutional_neural_network/main.py#L41-L42
BN should be used after ReLU. Features might be truncated by non-linearity like ReLU, so BN is used to normalize the distribution of features.