yzhangcs / parser

:rocket: State-of-the-art parsers for natural language.
https://parser.yzhang.site/
MIT License
827 stars 139 forks source link

directions in LSTM #48

Closed attardi closed 3 years ago

attardi commented 3 years ago

In lstm.py, shouldn't be the dimensions of h and c depend on the number of directions, as mentioned in the comment? "h of shape [num_layers*num_directions, batch_size, hidden_size] holds the initial hidden state ... c of shape [num_layers*num_directions, batch_size, hidden_size] holds the initial cell state"

Hence, it should be:

num_directions = 1 + self.bidirectional
if hx is None:
        ih = x.new_zeros(self.num_layers * num_directions, batch_size, self.hidden_size)
        h, c = ih, ih
    else:
        h, c = self.permute_hidden(hx, sequence.sorted_indices)
    h = h.view(self.num_layers, num_directions, batch_size, self.hidden_size)
    c = c.view(self.num_layers, num_directions, batch_size, self.hidden_size)
yzhangcs commented 3 years ago

Yeah, u're right. Thank you!

attardi commented 3 years ago

Shouldn't also be:

    if hx is None:
        ih = x.new_zeros(self.num_layers * self.num_directions, batch_size, self.hidden_size)