search

rwth-i6 / returnn_common

Common building blocks for RETURNN configs, such as models, training concepts, etc
7 stars 4 forks source link

LSTM param init wrong #217

Closed albertz closed 1 year ago

albertz commented 1 year ago

We initialize all params with Glorot currently. In RETURNN, the bias is initialized with 0. instead but all other params are also Gorot by default, so only the bias is wrong.

albertz commented 1 year ago

@JackTemaki @Atticus1806 this might change the behavior of your setups. Now the bias is initialized with 0., as in RETURNN.