Open kocmitom opened 6 years ago
I examined Nematus code: https://github.com/EdinburghNLP/nematus/blob/master/nematus/layers.py
which should be the same as ours: https://github.com/ufal/neuralmonkey/blob/master/neuralmonkey/nn/ortho_gru_cell.py#L57
They use ortho_weight and norm_weight. I did not go into details if ortho_weight is equal to our orthogonal initialization. But we are using tf.glorot_normal_initializer() but they are using: https://github.com/lium-lst/nmtpy/blob/master/nmtpy/nmtutils.py#L66
0.01 * np.random.randn(nin, nout)
I examined Nematus code: https://github.com/EdinburghNLP/nematus/blob/master/nematus/layers.py
which should be the same as ours: https://github.com/ufal/neuralmonkey/blob/master/neuralmonkey/nn/ortho_gru_cell.py#L57
They use ortho_weight and norm_weight. I did not go into details if ortho_weight is equal to our orthogonal initialization. But we are using tf.glorot_normal_initializer() but they are using: https://github.com/lium-lst/nmtpy/blob/master/nmtpy/nmtutils.py#L66
0.01 * np.random.randn(nin, nout)