search

ufal / neuralmonkey

An open-source tool for sequence learning in NLP built on TensorFlow.
BSD 3-Clause "New" or "Revised" License
410 stars 106 forks source link

Neural Machine Translation with Word Predictions #514

Open jlibovicky opened 6 years ago

jlibovicky commented 6 years ago

Paper Neural Machine Translation with Word Predictions by Weng et al. (https://arxiv.org/pdf/1708.01771.pdf) improves the decoder training by adding an additional objective by predicting the remaining target sentence words from the decoders' states. In this way there were able improve by 4.5 BLEU points in Chinese-English translation and 1.3 BLEU points in German-English translation. Similar trick helped with encoder.

This can be added to Neural Monkey by implementing a new decoder operating on states of the recurrent decoder/encoder and do multi-task training in the standard way.

guotong1988 commented 6 years ago

Have you finished the implementation? Thank you!

jindrahelcl commented 6 years ago

No, nor we are planning to right now. You can contribute if you want :)

guotong1988 commented 6 years ago

I confuse about the paper: the author want to predict the vocabulary OR just another additional objective designed by the author?