search

daniel-kukiela / nmt-chatbot

NMT Chatbot
GNU General Public License v3.0
385 stars 213 forks source link

Adjusting word representations #96

Open Alekos92 opened 5 years ago

Alekos92 commented 5 years ago

I have been using your version of NMT for a while now, and I find it quite successful in learning reddit conversations.

One modification I'm working on is to adjust word embeddings in order to improve the proposed stack, for example by concatenating topic information, or by translating the corresponding vectors.

What would be the suggested way to incorporate such adjustments to the code? Just concatenating the desired vectors in the encoder and decoder builders leads to sizing errors. Would it be better if the system was fed using pre-trained embeddings that already include the desired adjustments, if such a modification is supported?

daniel-kukiela commented 5 years ago

I haven't look that deep into Google's NMT to answer your questions. My fork contains just couple of fixes/tweaks/changes. We have a plan of replacing NMT with own seq2seq model.