[BUG]: NLU Spellchecker correcting numbers

botpress / nlu

This repo contains every ML/NLU related code written by Botpress in the NodeJS environment. This includes the Botpress Standalone NLU Server.

23 stars 21 forks source link

[BUG]: NLU Spellchecker correcting numbers #90

Closed sebburon closed 3 years ago

sebburon commented 3 years ago

There is an issue with the spellchecker that makes it so numbers are getting "corrected". As an example, if I ask the user for a number and the user enters 1005. The NLU will "correct" this number to 1000 and the saved value for the entry will be 1000.

franklevasseur commented 3 years ago

This is a known and very old bug that I simply didn't have time to fix.

The spellchecker performs really badly as shown by our e2e scores and metrics. It only knows the vocabulary of the training set you train with...

My suggestions for now are:

If you directly use the nlu-server POST /predict route, just don't use the spellcheck field.
If you use the nlu server through botpress, set the NLU_SKIP_SPELLCHECK to true. This variable is used here.

franklevasseur commented 3 years ago

Fixed by #119