search

tensorflow / nmt

TensorFlow Neural Machine Translation Tutorial
Apache License 2.0
6.36k stars 1.96k forks source link

OOV solution #357

Open yapingzhao opened 6 years ago

yapingzhao commented 6 years ago

Hi, I am a beginner of nmt.I have a question: How to solve the problem of OOV in translation translation? Can I add a large dictionary? How to add a dictionary in nmt? Looking forward to your advice or answers. Best regards, Thank you very much!

thormacy commented 6 years ago

reduce batch size, hidden layer number, hidden state number, etc.

hpulfc commented 6 years ago

@yapingzhao first , you should have a better Word segmentation tools. then use the output summary to find the alignment , the replace unk word

thormacy commented 6 years ago

A precise Word segmentation tool is definitely helpful to reduce the vocabulary size. Alternative ways: try copy net

yapingzhao commented 5 years ago

@hpulfc ，thanks!

VieZhong commented 5 years ago

I find the solution about this question ---- CopyNet. The code is here.

But I wonder that, its vocabulary size is fixed, and there's no array or no list to store the oov words, so how does the code solve the oov problem?

paulpaul91 commented 5 years ago

I find the solution about this question ---- CopyNet. The code is here.

But I wonder that, its vocabulary size is fixed, and there's no array or no list to store the oov words, so how does the code solve the oov problem?

have you understand the copynet in nmt?