spro / practical-pytorch

Go to https://github.com/pytorch/tutorials - this repo is deprecated and no longer maintained
MIT License
4.52k stars 1.11k forks source link

seq2seq-translation-batched: Bahdanau attention does not work #82

Open juditacs opened 6 years ago

juditacs commented 6 years ago

__init__.py fails with AttributeError, max_length does not exist. Fixing this results in a concat error in the Attn class:

    45         elif self.method == 'concat':
---> 46             energy = self.attn(torch.cat((hidden, encoder_output), 1))
     47             energy = self.v.dot(energy)
     48             return energy

RuntimeError: dimension out of range (expected to be in range of [-1, 0], but got 1)

replacing the dimension in line 46 to 0 results in this error:

    44 
     45         elif self.method == 'concat':
---> 46             energy = self.attn(torch.cat((hidden, encoder_output), 0))
     47             energy = self.v.dot(energy)
     48             return energy

RuntimeError: inconsistent tensor sizes at /opt/conda/conda-bld/pytorch_1512386481460/work/torch/lib/THC/generic/THCTensorMath.cu:157
aevilorz commented 6 years ago

I think there could be a solution to attention calculation in matrix form in #107 .

anantzoid commented 6 years ago

119 addresses some of the mentioned issues.