clovaai / FocusSeq2Seq

[EMNLP 2019] Mixture Content Selection for Diverse Sequence Generation (Question Generation / Abstractive Summarization)
https://arxiv.org/abs/1909.01953
MIT License
113 stars 20 forks source link

hidden size doesn't match #6

Closed Yupei-Du closed 4 years ago

Yupei-Du commented 4 years ago

Hi, thanks for open-source the code!

However, when I change the 'dec_hidden_size' to 256, an error occurs:

Traceback (most recent call last):
  File "train.py", line 351, in <module>
    train=True)
  File "/search/data3/xxx/py3_pytorch_venv/lib/python3.7/site-packages/torch/nn/modules/module.py", line 547, in __call__
    result = self.forward(*input, **kwargs)
  File "/search/data3/xxx/FocusSeq2Seq/models.py", line 272, in forward
    diversity_lambda=diversity_lambda)
  File "/search/data3/xxx/py3_pytorch_venv/lib/python3.7/site-packages/torch/nn/modules/module.py", line 547, in __call__
    result = self.forward(*input, **kwargs)
  File "/search/data3/xxx/FocusSeq2Seq/layers/decoder.py", line 296, in forward
    s = self.rnncell(dec_input, s)
  File "/search/data3/xxx/py3_pytorch_venv/lib/python3.7/site-packages/torch/nn/modules/module.py", line 547, in __call__
    result = self.forward(*input, **kwargs)
  File "/search/data3/xxx/py3_pytorch_venv/lib/python3.7/site-packages/torch/nn/modules/rnn.py", line 1015, in forward
    self.check_forward_input(input)
  File "/search/data3/xxx/py3_pytorch_venv/lib/python3.7/site-packages/torch/nn/modules/rnn.py", line 761, in check_forward_input
    input.size(1), self.input_size))
RuntimeError: input has inconsistent input_size: got 812, expected 556

I guess it is because the 'context' doesn't 'bridge' to the right size. How should I fix it? Thank you so much.

j-min commented 4 years ago

As you see here, the size of context is the size of encoder outputs (512 = bidirectional rnn of 256). And decoder input size is 812 = 300 (word embedding size) + 512 (encoder output size) here So you need to change the size of encoder to 256 as well to match it. The authors of NQG++ used 512-dim decoder for match dimensions of encoder and decoder

Yupei-Du commented 4 years ago

Thanks a lot, I'd like to close this issue 👍