Hi, I found a similar issue that was closed earlier. I did go over the discussion, however, I still think that the implementation is incorrect.
This is the original implementation. Here, self.weight is is not updated during the forward pass. So, if we do 2 consecutive forward passes, in both the forward passes, we start with the same value for self.weight. While we need the value of self.weight to be updated at the end of first forward pass, and use the updated value for the second forward pass.
Also, in the earlier issue the following explanation was given to justify the existing implementation. However, it is not a correct justification, as in the existing implementation we do not index W while assigning the new values, instead we simply assign W to the output of the recurrent layer.
Following two code snippets are the proposed changes that work correctly.
Hi, I found a similar issue that was closed earlier. I did go over the discussion, however, I still think that the implementation is incorrect.
This is the original implementation. Here,
self.weightis is not updated during the forward pass. So, if we do 2 consecutive forward passes, in both the forward passes, we start with the same value forself.weight. While we need the value ofself.weightto be updated at the end of first forward pass, and use the updated value for the second forward pass.Also, in the earlier issue the following explanation was given to justify the existing implementation. However, it is not a correct justification, as in the existing implementation we do not index W while assigning the new values, instead we simply assign W to the output of the recurrent layer.
Following two code snippets are the proposed changes that work correctly.