search

OpenNMT / OpenNMT-tf

Neural machine translation and sequence learning using TensorFlow
https://opennmt.net/
MIT License
1.45k stars 390 forks source link

translation with language models #495

Open VictorChen2012 opened 4 years ago

VictorChen2012 commented 4 years ago

I'm wondering if it is possible to combine an LM with a seq2seq model under OpenNMT-tf, e.g. shallow fusion, deep fusion or cold fusion.

Currently, vars and ops of LM decoder and seq2seq decoder are in different name scope. It's too complicated to directly load and merge two pretrained models, i.e. LM and the seq2seq model under the same name scope.

Any suggestions to the goal above?

guillaumekln commented 4 years ago

Shallow fusion should be the most accessible but it may not be easy to integrate at this time. However, there are some incoming changes that should facilitate such combinations.

I'm interested in supporting shallow fusion in the near future.

VictorChen2012 commented 4 years ago

Shallow fusion should be the most accessible but it may not be easy to integrate at this time. However, there are some incoming changes that should facilitate such combinations.

I'm interested in supporting shallow fusion in the near future.

Thanks for your quick reply! I'll see if I can contribute then.

lkluo commented 3 years ago

Any update?

guillaumekln commented 3 years ago

No one is currently working on this as far as I know.