search

clab / dynet

DyNet: The Dynamic Neural Network Toolkit
Apache License 2.0
3.42k stars 704 forks source link

Behavior of s-unit setter/getter in state-less RNNBuilders #489

Open odashi opened 7 years ago

odashi commented 7 years ago

In some state-less RNNBuilder implementations (e.g. SimpleRNNBuilder or GRUBuilder) also have get_s/set_s_impl overridings, and they are basically coded as the redirect to get_h/set_h_impl. This (especially the setter) brings a side-effect against h-units, and maybe it sometimes could be an undesirable behavior.

I guess it is better that the get_s returns 0 and set_s_impl behaves no action if the RNN cell does not have s-units. How do maintainers think?

neubig commented 7 years ago

I think this is a documentation problem. The actual semantics of get_s and set_s are "set all of the information that the RNN needs." In the case of RNNs and GRUs, this is just h. In the case of LSTMs, this is h and c. The documentation is saying something different though, so we should fix this.

pmichel31415 commented 7 years ago

We could add set_c and get_c methods to lstms. Would be a bit rebarbative to code but it's mostly copy pasting of set_h/get_h. Would also need subclass specific bindings in python