search

HUJI-Deep / simnets-tf

SimNets implementation in TensorFlow
MIT License
7 stars 3 forks source link

Default initializers #7

Closed orsharir closed 7 years ago

orsharir commented 7 years ago

Beyond the fact we need to fix issue #3, I believe the current default initializers for the parameters of the layers are not right.

For the weights, the default initializer should be ones, and not uniform. For the templates the default initializer should be normal gaussian. For the MEX layers, each offset vector (the rows of the weight matrix) should be drawn from a Dirichlet distribution of dimension equal to the vector and alpha = 1. You can implement the latter with numpy.random.dirichlet([1] * k), where k is the dimension of each vector.

elhanan7 commented 7 years ago

solved with 559663bf60a647f232e81adc3692c40cf473d661