search

tensorflow / nmt

TensorFlow Neural Machine Translation Tutorial
Apache License 2.0
6.35k stars 1.96k forks source link

How to use BERT embedding? #486

Open nashid opened 2 years ago

nashid commented 2 years ago

Is it possible to use BERT word embeddings along with this NMT implementation?

The goal is to use a pre-trained BERT language model so the contextualized embedding could be leveraged.

I am wondering whether anyone implemented or was able to run this model with any other contextualized embedding like ELMO or BERT.

nashid commented 2 years ago

@mommi84 any idea?

namavar-marjane commented 2 years ago

That would be a very useful feature.

nashid commented 2 years ago

This would be very useful as contextual embedding became a norm nowadays. Wondering anyone implemented or was able to run this model with any contextualized embedding?