search

leeyoshinari / YOLO_v2

The implementation of YOLO v2 with TensorFlow
GNU General Public License v3.0
80 stars 38 forks source link

About batch normalization #33

Closed Leoshiou closed 5 years ago

Leoshiou commented 5 years ago

Excuse me, can someone explain, why for batch normalization, it takes the maximum of the tensor of multiplying alpha and not as the following code.

    if batch_norm:
        depth = shape[3]
        scale = tf.Variable(tf.ones([depth, ], dtype='float32'), name='scale')
        shift = tf.Variable(tf.zeros([depth, ], dtype='float32'), name='shift')
        mean = tf.Variable(tf.ones([depth, ], dtype='float32'), name='rolling_mean')
        variance = tf.Variable(tf.ones([depth, ], dtype='float32'), name='rolling_variance')

        conv_bn = tf.nn.batch_normalization(conv, mean, variance, shift, scale, 1e-05)
        conv = tf.add(conv_bn, biases)
        **conv = tf.maximum(self.alpha * conv, conv)**
    else:
        conv = tf.add(conv, biases)
Leoshiou commented 5 years ago

Excuse me, can someone explain, why for batch normalization, it takes the maximum of the tensor of multiplying alpha and not as the following code.

    if batch_norm:
        depth = shape[3]
        scale = tf.Variable(tf.ones([depth, ], dtype='float32'), name='scale')
        shift = tf.Variable(tf.zeros([depth, ], dtype='float32'), name='shift')
        mean = tf.Variable(tf.ones([depth, ], dtype='float32'), name='rolling_mean')
        variance = tf.Variable(tf.ones([depth, ], dtype='float32'), name='rolling_variance')

        conv_bn = tf.nn.batch_normalization(conv, mean, variance, shift, scale, 1e-05)
        conv = tf.add(conv_bn, biases)
        **conv = tf.maximum(self.alpha * conv, conv)**
    else:
        conv = tf.add(conv, biases)

I found out that it was the leaky activation function.