search

vikrant7 / mobile-vod-bottleneck-lstm

Implementation of Mobile Video Object Detection with Temporally-Aware Feature Maps using PyTorch
131 stars 37 forks source link

Is it necessary to re-initilize hidden state and cell state after detach_() ? #6

Closed turboxin closed 5 years ago

turboxin commented 5 years ago

Hi @vikrant7 , thanks for this great implementation, I have learned a lot from it.

After each iteration of a sequence, you detach the hidden state and cell state of the lstm cell from the graph to cut backpropagation, do you think it's also necessary to re-initilize them, since hidden state and cell state should be zero for the start of a new sequence?

vikrant7 commented 5 years ago

Hi @turboxin, Why do you think the cell state and hidden state should be zero at the starting of the sequence? I will suggest you go through this thread: here Here they have mentioned why we learn initial states.