How does this work?

[knkarthik]

Amazing work! It would be helpful if you can explain how the tool works - the ML/NLP stuff behind this so that others can modify it if needed.

Thanks.

[alfredfrancis]

We have two main NLU modules

Intent classifier module

It uses scikit learn SVM for text classification and uses NLTK for preprocessing and feature extraction.

Following preprocessing tasks are performed for the classification,

• Tokenizing
• stopwords removal
• Stemming
• Lemmatizing

NER module

NER module uses PyCRF suite for Named Entity Recognition. It takes following features,

• POS tags
• Word order Word position in the sentence