Only copy the state of top_h?

karpathy / char-rnn

Multi-layer Recurrent Neural Networks (LSTM, GRU, RNN) for character-level language models in Torch

11.59k stars 2.58k forks source link

Only copy the state of top_h? #91

Closed ffmpbgrnn closed 9 years ago

ffmpbgrnn commented 9 years ago

Hi, in train.lua#L276, I found the code only copy the top_h to the next batches. Why you didn't copy other hs to the next batch?

Thank you!

karpathy commented 9 years ago

the state of the RNN is summarized in that last set of states, the other states are not necessary once we've performed a backward pass on that batch. We use that last state to "initialize" the state for next batch in sequence.

ffmpbgrnn commented 9 years ago

Thank you, I get it!