ShulinCao / OpenNRE-PyTorch

Neural Relation Extraction implemented in PyTorch
MIT License
219 stars 45 forks source link

Question about dropout in selector #16

Open WindChimeRan opened 5 years ago

WindChimeRan commented 5 years ago

https://github.com/ShulinCao/OpenNRE-PyTorch/blob/master/networks/selector.py#L70

You use dropout after sen_matrix in the training stage, and not use it in the test stage.

I think the nuance of dropout should be manipulated by model.eval(), rather than by applying another similar function without dropout.

class One(Selector):
    def forward(self, x):
        tower_logits = []
        for i in range(len(self.scope) - 1):
            sen_matrix = x[self.scope[i] : self.scope[i + 1]]
            sen_matrix = self.dropout(sen_matrix)
            logits = self.get_logits(sen_matrix)
            score = F.softmax(logits, 1)
            _, k = torch.max(score, dim = 0)
            k = k[self.label[i]]
            tower_logits.append(logits[k])
        return torch.cat(tower_logits, 0)
    def test(self, x):
        tower_score = []
        for i in range(len(self.scope) - 1):
            sen_matrix = x[self.scope[i] : self.scope[i + 1]]
            logits = self.get_logits(sen_matrix)
            score = F.softmax(logits, 1)
            score, _ = torch.max(score, 0)
            tower_score.append(score)
        tower_score = torch.stack(tower_score)
        return list(tower_score.data.cpu().numpy())