search

OpenNMT / OpenNMT-py

Open Source Neural Machine Translation and (Large) Language Models in PyTorch
https://opennmt.net/
MIT License
6.72k stars 2.25k forks source link

[feature request - Document level context] #1212

Closed vince62s closed 5 years ago

vince62s commented 5 years ago

Hi guys, If someone is interested, there are some papers that we could implement. Research is definitely going that way and it is part of the WMT19 task.

This paper: http://aclweb.org/anthology/D18-1049 We could implement it. One aspect is that it requires 2 steps training. Maybe we can avoid that by turning off context attention when not applicable (we need to separate the 2 soources: sentence level, document level).

There is another interesting paper here: https://publications.idiap.ch/downloads/papers/2018/Miculicich_EMNLP_2018.pdf There is an implementation based on an old version of onmt-py. We could implement it too but it also require 2 or 3 step training.

if anyone is interested, let me know.

LauraYu commented 5 years ago

I've been playing with the code of the HAN model (https://publications.idiap.ch/downloads/papers/2018/Miculicich_EMNLP_2018.pdf). As you said, it's based on an old version of onmt-py and pytorch 0.3, which makes it very difficult to adapt to. It'll be really helpful to have it implemented in onmt.

vince62s commented 5 years ago

closing this while expecting more convincing papers.

ling-pan commented 5 years ago

Hi,

I wonder whether there are implementations for these two papers in current OpenNMT?

Thanks!

vince62s commented 5 years ago

No, I am waiting for papers submitted at WMT19

JOHW85 commented 7 months ago

Any interest in this: https://aclanthology.org/2021.acl-long.267/