search

daniel-kukiela / nmt-chatbot

NMT Chatbot
GNU General Public License v3.0
385 stars 212 forks source link

Repeated Answers given by Chatbot #159

Closed VPHarish closed 1 year ago

VPHarish commented 4 years ago

Hi I am actually having this problem for every models I train. the model just train well enough to answer in just same sentences, and i definitely didn't overfit it. I trained it on Google Colab, databases I used were : 2011-03(60k pairs), 2015-01(3M pairs), 2015-05(3M pairs). I used these databases to train different models and it all outputs like this. I don't know where I made mistake. Here are the outputs from the nmt in training process:

src: > Alacaugalidaseeppianna newlinechar newlinechar I spent more time than I'm willing to admit trying to pronounce that, and figure out your state order. ref: I threw in the Muskogee 'cauga' phonetic in there, just to make it sound like a plausible location. nmt: **I'm not sure that's a lot of time.**

src: Alright awesome, thanks! Will I notice much of a loading time difference over using a thunderbolt connected hard drive over a USB 3 hard drive or not really? ref: I'm not sure, I think Thunderbolt (if it's faster than USB 3) but I doubt you'd notice a real world difference. nmt: **I'm not sure if you're going to be a good idea.**

src: littlebrother ref: works on lockscreen too and 6+'s? nmt: **I'm not sure if**

src: That beer sounds PHENOMENAL. ref: 'Tis. I'm pretty sad it was only produced once. Extremely full bodied and tasty. nmt: **I'm not sure if it's a 'fist'.**

I'm new to machine learning and was interested to learn this stuff, but stuck with these answers from the chatbot:

Hello

  • I'm not sure what you mean. [3.55]

Who are you?

  • **I'm not sure if you'** [-94.5]

May I know your name?

  • **I'm not sure if you're not a fan** [-95.7]

How are you?

  • **I'm not sure if you'** [-94.5]

I'm not sure what to do next, please help! Thank You.

M0rica commented 4 years ago

For how long has the model been training(how many epochs)? Because this normaly happens if the model didn't train that long, like less than 2 epochs.

VPHarish commented 4 years ago

For every database I trained fully. But for some reason it gives same type of answer for the same question in different database trained models. I don't know what to do now, I am currently trying to increase my database but Google Colab started crashing on me for more GPU memory usage.

M0rica commented 4 years ago

Thatt's weird, an other possibility why a model responses with "I'm not sure" (at least in my experiments) is that it simply had to few examples for your question and doesn't know how to reply, but i don't think that's the case with your 6 million pairs...

VPHarish commented 4 years ago

I am currently trying to train a new model from scratch with nearly 25 - 35 million pairs, but Google Colab definitely won't work. So I am going to train it on a local GPU host service and within few days I will come up with my new trained model and update about it. Thank you .

Nathan-Chell commented 4 years ago

Are you using the new fork of the code? You said 'I trained fully', what does the entail exactly?

How many epochs have you trained for?

VPHarish commented 4 years ago

I can't understand what new fork means. First, I trained for 3 epochs, then for the remaining 2 databases I trained for 4.

Nathan-Chell commented 4 years ago

I mean are you using the most current code?

VPHarish commented 4 years ago

yes, I do

astrickash commented 4 years ago

I get rid of his "I'm not sure but" sentence by removing it from answers_subsentence_score.txt. I don't know what exactly is the role of this file but removing the line and running prepare_data.py and then train.py helps me in answering properly. FYI I trained it for 150k steps on 6.5M rows