gau820827 / AI-writer_Data2Doc

PyTorch Implementation of NBA game summary generator.
Apache License 2.0
83 stars 22 forks source link

Fix Multilayer GRU #13

Closed yinhsunhuang closed 6 years ago

yinhsunhuang commented 6 years ago

Multilayer GRU seems wrong. In the original implementation, the weight of each layer is the same.

class EncoderRNN(nn.Module):
    def __init__(self, hidden_size, embedding_layer, n_layers=LAYER_DEPTH):
       ...
        self.gru = nn.GRUCell(hidden_size, hidden_size)

    def forward(self, rt, re, rm, hidden):
        embedded = self.embedding(rt, re, rm)
        output = embedded

        for i in range(self.n_layers):
            output = self.gru(output, hidden)

        return output

the for loop in forward() is using the same gru in each iteration/layer.

yinhsunhuang commented 6 years ago

I've sent the PR(#14).

gau820827 commented 6 years ago
for i in range(self.n_layers):
    output = F.relu(output)
    layer_fnc = getattr(self, "gru"+str(i))
    output = layer_fnc(output, hidden[i,:,:])
    nh[i,:,:] = output

Why there's a Relu function between each layer?