wenxinxu / resnet-in-tensorflow

Re-implement Kaiming He's deep residual networks in tensorflow. Can be trained with cifar10.
MIT License
828 stars 276 forks source link

about weight initialization #3

Open pranshushah opened 7 years ago

pranshushah commented 7 years ago

why are you using tf.contrib.layers.xavier initializer instead of tf.contrib.layers.variance_scaling_initializer() ??

wenxinxu commented 7 years ago

There were no documentations for the tf.contrib.layers.variance_scaling_initializer() when I wrote the code. Basically the variance scaling initializer is a more customizable version of Xavier. They are based on the same idea that adjusts the initialization with the input channels. They should perform similarly in resnet. (You may change the code and play with it.)