All translated sentences start with a same word

[zhangdistephen]

I wonder has anyone run the code and whether encounter the same problem? @ZiJianZhao

[juditacs]

We are experiencing the same issue for morphological reinflection.

k l l a m i g a z g a t á s b a n </s>
k a c s k a e l e d e l h e z </s>
k o m m u n i s t á v á </s>
k g y o n v á g n á k </s>
k o k o z t a t o k </s>
k e r e m t é s r e </s>
k a r n i s n á l </s>
k z é r i á é r t </s>
k a b o l t á t o k </s>

[darkcoderrises]

Anyone found any solution for this?

[JulianRMedina]

I think part of the reason is that the training set mostly starts with the word 'A'. So it's most likely learning to put 'ein'. I could be wrong, but this is my hypothesis.

Also note that in the references file most of the Germen translations start with 'Ein'.

[juditacs]

There is no such pattern in our training set. "k" is indeed frequent, but not overwhelmingly.

[JulianRMedina]

Only way to be certain is to try another dataset, or remove all those starting with 'k'. You could try that just to see how it affects it?

[juditacs]

Unfortunately I don't have time for that right now, and it is obviously a bug in the code, "k" replaces valid characters/words. In the sample I posted above, the first letters should be: á k k a f t k ? r, otherwise the output seems to be correct (reinflection is a much easier task than machine translation). I can send you the data to test it.

[JulianRMedina]

I forked this repo to my own and will be fixing things with the model in general. If you want to help with that process please see the repo here: https://github.com/JulianRMedina/attention-is-all-you-need-pytorch

[juditacs]

I will be paying attention your work.

I can't promise direct help right now but I would be happy to share some morphological data that is not a a toy dataset but still very easy and the model should be able to learn it quickly (the standard seq2seq converges in a few epochs and performs with over 90% accuracy).

[kannadaraj]

Same Issue. for me too.. All words are starting with same word. What might be the possible reason...?

[saitarslanboun]

I solved the problem.

If you change

# wrap into a Variable
dec_partial_seq = Variable(dec_partial_seq, volatile=True)

with

# wrap into a Variable
dec_partial_seq = Variable(dec_partial_seq, volatile=True)
     if i == 0:
          s = torch.zeros(dec_partial_seq.size()).cuda().fill_(2).long()
          dec_partial_seq = torch.cat((dec_partial_seq, s), dim=1)

in "transformer/Translator.py" file, the problem is solved.

Best,

[yueyongjiao]

I wonder has anyone run the code and whether encounter the same problem? @ZiJianZhao

[saitarslanboun]

I wrote it down on up. If you go to the version on the date when I wrote, you can do the change what I have told, and you will solve the problem.

[yueyongjiao]

May this code can be used in the task where the source text length is larger than the length of the target text，such as summarization generation ?

[saitarslanboun]

It seems to me it is not pretty straightforward thing to do. You have to make some modifications in encoder part of the code.

[nhh1501]

I try to create chatbot. But all translated sentencest always same result

i m not not not not not not . i m not not not not not not . i m not not not not not not . i m not not not not not not . i m not not not not not not . i m not not not not not not . i m not not not not not not . i m not not not not not not . i m not not not not not not . i m not not not not not not . i m not not not not not not . i m not not not not not not . i m not not not not not not . i m not not not not not not . i m not not not not not not . i m not not not not not not . i m not not not not not not . i m not not not not not not . i m not not not not not not . i m not not not not not not . i m not not not not not not . i m not not not not not not . i m not not not not not not . i m not not not not not not . i m not not not not not not . i m not not not not not not .

[rush86999]

i m having the same issue with same translation, any ideas would be helpful

[marvinzh]

I try to create chatbot. But all translated sentencest always same result

i m not not not not not not . i m not not not not not not . i m not not not not not not . i m not not not not not not . i m not not not not not not . i m not not not not not not . i m not not not not not not . i m not not not not not not . i m not not not not not not . i m not not not not not not . i m not not not not not not . i m not not not not not not . i m not not not not not not . i m not not not not not not . i m not not not not not not . i m not not not not not not . i m not not not not not not . i m not not not not not not . i m not not not not not not . i m not not not not not not . i m not not not not not not . i m not not not not not not . i m not not not not not not . i m not not not not not not . i m not not not not not not . i m not not not not not not .

i m having the same issue with same translation, any ideas would be helpful

same issue, solved by writing beam search function by myself

[Abigale001]

I try to create chatbot. But all translated sentencest always same result i m not not not not not not . i m not not not not not not . i m not not not not not not . i m not not not not not not . i m not not not not not not . i m not not not not not not . i m not not not not not not . i m not not not not not not . i m not not not not not not . i m not not not not not not . i m not not not not not not . i m not not not not not not . i m not not not not not not . i m not not not not not not . i m not not not not not not . i m not not not not not not . i m not not not not not not . i m not not not not not not . i m not not not not not not . i m not not not not not not . i m not not not not not not . i m not not not not not not . i m not not not not not not . i m not not not not not not . i m not not not not not not . i m not not not not not not .

i m having the same issue with same translation, any ideas would be helpful

same issue, solved by writing beam search function by myself

Hi @marvinzh , is it possible to share your beam search? And if possible, my email address is liyicong123@outlook.com. Thank you very much for your kind help.

[jadore801120]

Hi all, please take a look on the newest code. I made the beam search more concise and clear. Hope it helps.