search

clab / dynet

DyNet: The Dynamic Neural Network Toolkit
Apache License 2.0
3.42k stars 704 forks source link

"empty" RNNStates should be rigged to return zero vectors on .s(), .h(), etc #1322

Open gailweiss opened 6 years ago

gailweiss commented 6 years ago

having .s() return nothing on an RNN's initial state is very confusing (at least to me), as there must be some kind of value from which the RNN starts and not revealing it just hides what's going on (making things like automaton-extraction very difficult).

to my understanding, RNNStates with empty .s() are effectively treated as having zero-vectors in .add_input() computations (the matrix multiplication with the missing c/h vectors is simply skipped). It would be nice if .s(), .h(), .output(), etc reflected this.

neubig commented 6 years ago

Yes, I think this is a good idea.

Mouray-Hutchinson commented 2 years ago

Hello, I would like to fix this bug in RNN for a SWE class. Can this be assigned to me?