Conchylicultor
commented
7 years ago I'm not sure what is your problem. If your dataset is only composed of 12 samples, it's normal that the model won't generalize well.
Open yogesh-0586 opened 7 years ago
Conchylicultor
commented
7 years ago I'm not sure what is your problem. If your dataset is only composed of 12 samples, it's normal that the model won't generalize well.
yogesh-0586
commented
7 years ago This is my input sets
{ "question" : "what is your name", "answer" : "LOLA" } { "question" : "what is your age", "answer" : "Don't know" } { "question" : "Current time", "answer" : "Don't know" } { "question" : "Current year", "answer" : "2017" } { "question" : "Current month", "answer" : "June" } { "question" : "how to find current month", "answer" : "current month is June" } { "question" : "how to find current time", "answer" : "Don't know" } { "question" : "your city", "answer" : "universe" } { "question" : "your city name", "answer" : "universe" } { "question" : "your city Zip code", "answer" : "universe" } { "question" : "What do you know about Java?", "answer" : "Java is a high-level programming language originally developed by Sun Microsystems and released in 1995. Java runs on a variety of platforms, such as Windows, Mac OS, and the various versions of UNIX." } { "question" : "What are the supported platforms by Java Programming Language?", "answer" : "Java runs on a variety of platforms, such as Windows, Mac OS, and the various versions of UNIX/Linux like HP-Unix, Sun Solaris, Redhat Linux, Ubuntu, CentOS, etc." } { "question" : "List any five features of Java?", "answer" : "Some features include Object Oriented, Platform Independent, Robust, Interpreted, Multi-threaded" } '
Put all configuration parameter as same in chatbot.py just increase numEpochs to 100 and learningRate to 0.001
And this will generate in my model_prediction.txt file
Q: what is your name A: Lola
Q: what is your age A: Do n't know
Q: Current time A: Do n't know
Q: Current year A: 2017
Q: Current month A: June
Q: how to find current month A: Current month is june
Q: how to find current time A: Do n't know
Q: your city A: Universe
Q: your city name A: Universe
Q: your city Zip code A: Universe
Q: What do you know about Java? A: Do n't know
Q: What are the supported platforms by Java Programming Language? A: Do n't know
Q: List any five features of Java? A: Some features include object oriented, platform,,,,,,
Q: java A: 2017
Q: features A: 2017
Q: Programming A: 2017
Q: Language A: 2017
Q: platforms A: 2017
Q: fiv A: 2017
bshao001
commented
7 years ago The weights trained are more sensitive to the question length than any words in the question. For example, if you had a training record like: Q: Something A: My answer. And if that was the only one word question data in your training set, the answers for all your last a few one word question would have been: My answer. That's one type of typical overfitting when you do not have enough training data.
In CornellData.py file I update some code and passed my data for QA format like this
I passed only 10-12 questions for testing. If I passed exact sentences for testing it return proper answers, but for single word or combination of two words it returns different answers. In my 10-12 questions one questions having all unique words which not presents in any other questions but for testing with that unique words it show other answers.