search

RasaHQ / rasa

💬 Open source machine learning framework to automate text- and voice-based conversations: NLU, dialogue management, connect to Slack, Facebook, and more - Create chatbots and voice assistants
https://rasa.com/docs/rasa/
Apache License 2.0
18.89k stars 4.63k forks source link

Variations in Confusion Matrix with changing Kernel for Intent Classifier Sklearn #843

Closed AbdurRub closed 6 years ago

AbdurRub commented 6 years ago

Rasa NLU version (e.g. 0.7.3): 0.11

Used backend / pipeline (mitie, spacy_sklearn, ...): spacy_sklearn

Operating system (windows, osx, ...): Ubuntu 17.04

Issue: Attached is the picture of Intent Confusion Matrix using all 3 kernels (Linear , RBF , Polynomial) There is slight variation in Linear and RBF but huge variation in Polynomial kernel usage.

I want to ask which one is better to use and what this picture is showing regarding kernel usage. I am confused which one is better in my use case.

I am building chatbot for automobile industry assume it. Data contains different type of messages. falling in Sales Service Parts Intents.

abdur 2018-02-13 12-02-36

tmbo commented 6 years ago

Either Linear or RBF look fine, the polynomial kernel doesn't seem to get the difference between service and anything else. The confusion matrix shows you the prediction of the model on the X axis and the expected label on the y axis. So the diagonal shows the number of examples where the model predicted the label that was expected. Anything that is not on the diagonal is an example where the model predicted the wrong label.