NLU does not work correctly with the Russian language

[SergeyKhodusov]

Describe the bug NLU does not work correctly with the Russian language.

To Reproduce Steps to reproduce the behavior:

1. Created a bot
2. Added 1 intent
3. Added 1 question Q&A
4. I am writing a message that should not fall into either the intent or the Q&A at all.

Gives match percentages for intent. Neither in the intent nor in the QnA is there a match even for one word.

Expected behavior none equal to approximately 0%

Screenshots

Environment (please complete the following information):

• OS: Windows 11
• Browser Avast
• Browser Version 96.1.13589.111
• Botpress Version 12.26.10
• NodeJS 18.4.0

Additional context If I create more intents and QnA questions, I always get false positives of 50%+.

[SergeyKhodusov]

Prepared a better example. Created QnA on the topic of what the bot can do. Questions like: "Tell me about your abilities?" "Tell me what you can do?" "How else can you help me?" Wrote a text that has no relation in meaning to the questions asked (Like "Lorem Ipsum"). Gives a 75% match with QnA. It is wrong.

[SergeyKhodusov]

There are indeed matches for several words such as "What", "You" and "How". But these are words that do not make much sense in the question. They occur in almost every question or intent and should not affect the outcome.