search

gunthercox / chatterbot-corpus

A multilingual dialog corpus
http://chatterbot-corpus.readthedocs.io
BSD 3-Clause "New" or "Revised" License
1.36k stars 1.15k forks source link

Fixing bug in french corpus #164

Closed gwilherm closed 4 years ago

gwilherm commented 4 years ago

Hello,

I noticed that french bot sometimes answers with a simple letter, Below an extract from the database just after training the bot with french corpus:

$ sqlite3 db.sqlite3
sqlite> select * from statement;
[...]
71|Sur quoi travailles-tu ?|NNP:quoi VBD:travailles-tu|training|2019-11-08 00:51:21.479493|Je travaille sur un projet|NNP:travaille NN:sur NN:un JJ:projet|
72|Q|q|training|2019-11-08 00:51:21.479952|||
73|u|u|training|2019-11-08 00:51:21.480344|Q|q|
74|e|e|training|2019-11-08 00:51:21.480913|u|u|
75|l|l|training|2019-11-08 00:51:21.481112|e|e|
76|s|s|training|2019-11-08 00:51:21.481659|l|l|
77|||training|2019-11-08 00:51:21.481685|s|s|
78|l|l|training|2019-11-08 00:51:21.481869|||
79|a|a|training|2019-11-08 00:51:21.481981|l|l|
80|n|n|training|2019-11-08 00:51:21.482473|a|a|
81|g|g|training|2019-11-08 00:51:21.483138|n|n|
82|a|a|training|2019-11-08 00:51:21.483254|g|g|
83|g|g|training|2019-11-08 00:51:21.483420|a|a|
84|e|e|training|2019-11-08 00:51:21.483585|g|g|
85|s|s|training|2019-11-08 00:51:21.483746|e|e|
86|||training|2019-11-08 00:51:21.483773|s|s|
87|u|u|training|2019-11-08 00:51:21.483928|||
88|t|t|training|2019-11-08 00:51:21.484366|u|u|
89|i|i|training|2019-11-08 00:51:21.484735|t|t|
90|l|l|training|2019-11-08 00:51:21.484908|i|i|
91|i|i|training|2019-11-08 00:51:21.485070|l|l|
92|s|s|training|2019-11-08 00:51:21.485228|i|i|
93|e|e|training|2019-11-08 00:51:21.485389|s|s|
94|s|s|training|2019-11-08 00:51:21.485544|e|e|
95|-|-|training|2019-11-08 00:51:21.485632|s|s|
96|t|t|training|2019-11-08 00:51:21.485781|-|-|
97|u|u|training|2019-11-08 00:51:21.485931|t|t|
98|||training|2019-11-08 00:51:21.485950|u|u|
99|?|?|training|2019-11-08 00:51:21.486038|||
100|||training|2019-11-08 00:51:21.486055|?|?|
101|-|-|training|2019-11-08 00:51:21.486134|||
102|||training|2019-11-08 00:51:21.486150|-|-|
103|J|j|training|2019-11-08 00:51:21.486449|||
104|'|'|training|2019-11-08 00:51:21.486621|J|j|
105|u|u|training|2019-11-08 00:51:21.486777|'|'|
106|t|t|training|2019-11-08 00:51:21.486931|u|u|
107|i|i|training|2019-11-08 00:51:21.487085|t|t|
108|l|l|training|2019-11-08 00:51:21.487237|i|i|
109|i|i|training|2019-11-08 00:51:21.487390|l|l|
110|s|s|training|2019-11-08 00:51:21.487543|i|i|
111|e|e|training|2019-11-08 00:51:21.487697|s|s|
112|||training|2019-11-08 00:51:21.487716|e|e|
113|P|p|training|2019-11-08 00:51:21.487985|||
114|y|y|training|2019-11-08 00:51:21.488250|P|p|
115|t|t|training|2019-11-08 00:51:21.488419|y|y|
116|h|h|training|2019-11-08 00:51:21.488798|t|t|
117|o|o|training|2019-11-08 00:51:21.489188|h|h|
118|n|n|training|2019-11-08 00:51:21.489465|o|o|
119|,|,|training|2019-11-08 00:51:21.489569|n|n|
120|||training|2019-11-08 00:51:21.489589|,|,|
121|H|h|training|2019-11-08 00:51:21.489751|||
122|a|a|training|2019-11-08 00:51:21.489850|H|h|
123|v|v|training|2019-11-08 00:51:21.491224|a|a|
124|a|a|training|2019-11-08 00:51:21.491400|v|v|
125|||training|2019-11-08 00:51:21.491423|a|a|
126|e|e|training|2019-11-08 00:51:21.491614|||
127|t|t|training|2019-11-08 00:51:21.491797|e|e|
128|||training|2019-11-08 00:51:21.491818|t|t|
129|C|c|training|2019-11-08 00:51:21.492613|||
130|+|+|training|2019-11-08 00:51:21.492785|C|c|
131|+|+|training|2019-11-08 00:51:21.492940|+|+|
132|||training|2019-11-08 00:51:21.492961|+|+|
133|a|a|training|2019-11-08 00:51:21.493058|||
134|s|s|training|2019-11-08 00:51:21.493220|a|a|
135|s|s|training|2019-11-08 00:51:21.493379|s|s|
136|e|e|training|2019-11-08 00:51:21.493543|s|s|
137|z|z|training|2019-11-08 00:51:21.493976|e|e|
138|||training|2019-11-08 00:51:21.494013|z|z|
139|s|s|training|2019-11-08 00:51:21.494292|||
140|o|o|training|2019-11-08 00:51:21.494549|s|s|
141|u|u|training|2019-11-08 00:51:21.494835|o|o|
142|v|v|training|2019-11-08 00:51:21.495113|u|u|
143|e|e|training|2019-11-08 00:51:21.495406|v|v|
144|n|n|training|2019-11-08 00:51:21.495615|e|e|
145|t|t|training|2019-11-08 00:51:21.495799|n|n|
146|.|.|training|2019-11-08 00:51:21.495896|t|t|
147|||training|2019-11-08 00:51:21.495916|.|.|
148|-|-|training|2019-11-08 00:51:21.495999|||
149|||training|2019-11-08 00:51:21.496016|-|-|
150|J|j|training|2019-11-08 00:51:21.496165|||
151|'|'|training|2019-11-08 00:51:21.496321|J|j|
152|u|u|training|2019-11-08 00:51:21.496474|'|'|
153|t|t|training|2019-11-08 00:51:21.496627|u|u|
154|i|i|training|2019-11-08 00:51:21.496781|t|t|
155|l|l|training|2019-11-08 00:51:21.496937|i|i|
156|i|i|training|2019-11-08 00:51:21.497087|l|l|
157|s|s|training|2019-11-08 00:51:21.497242|i|i|
158|e|e|training|2019-11-08 00:51:21.497400|s|s|
159|||training|2019-11-08 00:51:21.497419|e|e|
160|s|s|training|2019-11-08 00:51:21.497570|||
161|u|u|training|2019-11-08 00:51:21.497722|s|s|
162|r|r|training|2019-11-08 00:51:21.498104|u|u|
163|t|t|training|2019-11-08 00:51:21.498282|r|r|
164|o|o|training|2019-11-08 00:51:21.498446|t|t|
165|u|u|training|2019-11-08 00:51:21.498601|o|o|
166|t|t|training|2019-11-08 00:51:21.498778|u|u|
167|||training|2019-11-08 00:51:21.498799|t|t|
168|P|p|training|2019-11-08 00:51:21.498958|||
169|y|y|training|2019-11-08 00:51:21.499112|P|p|
170|t|t|training|2019-11-08 00:51:21.499264|y|y|
171|h|h|training|2019-11-08 00:51:21.499425|t|t|
172|o|o|training|2019-11-08 00:51:21.499579|h|h|
173|n|n|training|2019-11-08 00:51:21.499736|o|o|
174|.|.|training|2019-11-08 00:51:21.499842|n|n|
[...]
gunthercox commented 4 years ago

@gwilherm Thank you for the contribution. I'm certain there are a number of developers in the community who will appreciate this fix.