NMT Inference: Chunk overlength sequences and translate in sequence - Githubissues

dmlc / gluon-nlp

NLP made easy

https://nlp.gluon.ai/

Apache License 2.0

2.55k stars 538 forks source link

NMT Inference: Chunk overlength sequences and translate in sequence #1424

Open leezu opened 3 years ago

leezu commented 3 years ago

We may add an equivalent functionality to the NMT evaluation scripts

https://github.com/awslabs/sockeye/blob/3736f18e3286895cb1eb29817396a8b508edd802/sockeye/inference.py#L657-L659

sxjscience commented 3 years ago

To add this functionality in the existing NMT example, we may revise the argument here https://github.com/dmlc/gluon-nlp/blob/20321598a732d254d4a9e036eecbe0ae8abb48a0/scripts/machine_translation/evaluate_transformer.py#L23-L90 and try to revise the translation logic.