search

timbmg / Sentence-VAE

PyTorch Re-Implementation of "Generating Sentences from a Continuous Space" by Bowman et al 2015 https://arxiv.org/abs/1511.06349
587 stars 153 forks source link

Maybe a bug in constructing hidden states. #20

Open leix28 opened 4 years ago

leix28 commented 4 years ago 
(1) hidden = self.latent2hidden(z)

(2) if self.bidirectional or self.num_layers > 1:
(3)     hidden = hidden.view(self.hidden_factor, batch_size, self.hidden_size)
(4) else:
(5)     hidden = hidden.unsqueeze(0)

I think line (3) should be

hidden = hidden.view(batch_size, self.hidden_factor, self.hidden_size).transpose(0, 1)

This snap of code appears in both forward and inference.

rocreguant commented 3 years ago

I tried, there is a dimensionality missmatch with your line...

Quasimondo commented 3 years ago

I also encountered a problem with inference of a 2-layer model. The fix for me was changing these lines in the inference() code in model.py:

if self.bidirectional or self.num_layers > 1:
    # unflatten hidden state
    hidden = hidden.view(self.hidden_factor, batch_size, self.hidden_size)

hidden = hidden.unsqueeze(0)

to

if self.bidirectional or self.num_layers > 1:
    # unflatten hidden state
    hidden = hidden.view(self.hidden_factor, batch_size, self.hidden_size)
if self.num_layers == 1:   
    hidden = hidden.unsqueeze(0)