I want to implement a seq2seq architecture wherein I can do multiple forward calls, average the loss and then backprop each individually. I've been trying to modify the seq2seq architecture for this using nn.SeqLSTM. here is some code which I think is the culprit
This doesn't seem possible with the current architecture because the above nn.Sequencer() seem to store timestep state internally which gets reset to 0 after 1 backward pass. Hence it crashes when I try to backprop the next time.
Is there someway I can achieve this using rnn?
PS: I don't think this is a github issue but I didn't find any mailing list for RNN library. Let me know if there is a better medium to ask this question.
I want to implement a seq2seq architecture wherein I can do multiple forward calls, average the loss and then backprop each individually. I've been trying to modify the seq2seq architecture for this using nn.SeqLSTM. here is some code which I think is the culprit
This doesn't seem possible with the current architecture because the above
nn.Sequencer()seem to store timestep state internally which gets reset to 0 after 1 backward pass. Hence it crashes when I try to backprop the next time.Is there someway I can achieve this using rnn?
PS: I don't think this is a github issue but I didn't find any mailing list for RNN library. Let me know if there is a better medium to ask this question.