Closed tcwicks closed 3 years ago
Is that what you want ?
Can you contact me in Gitter if you don't know how to participate this project?
@Oceania2018 I've seen GlorotUniform however there are a few differences. Variance Scaling Initializer has both the uniform as well as non uniform permutation. And each of these has the Xavier and the HE permutation In Math.Sqrt(1.3 * _Factor / n); Xavier uses a Factor of 1 HE uses a factor of
Also in many models the non uniform Glorot initializer performs better than GlorotUniform.
This person has done a decent job of explaining it much better than I can.
https://adventuresinmachinelearning.com/weight-initialization-tutorial-tensorflow/
Can you try latest version?
I would like to contribute the following. However: 1) I'm new to Github 2) I have no idea where it would go in SciSharp Tensorflow. 3) I'm not part of the project so I have no access anyway.
This is a port of tensorflow.contrib.variance_scaling_initializer trying to keep it as close as possible to the original in terms of style. Also along with the HE variant and the Xavier variant convenience methods.
Python URL: https://github.com/agrawalnishant/tensorflow/blob/da0a62b8c3d9e3357d41b5354acad3b5b25f7f95/tensorflow/contrib/layers/python/layers/initializers.py
Reason / Motivation: SciSharp Tensorflow does have an implementation of GlorotUniform however the variance scaling intializer has other permutations as well. Depending on the use case these can have a significant impact on training time.
`
`