search

Colin-Codes / IntentClassifier-ML-Project

Pyhton, Keras, SciKit-Learn, Matplotlib: Machine learning research project around classification of intent behind tech support emails in order to enable automatic follow up.
0 stars 0 forks source link

Scoring algorithm #27

Open Colin-Codes opened 4 years ago

Colin-Codes commented 4 years ago

#Predicted react, correctly reacted: 1

Predicted react, incorrectly reacted: -1

#Predicted react, should have ignored: -1
#Predicted ignore, should have reacted: -1
#Predicted ignore, should have ignored: 1</strike>

Consider a threshold value - we only want to answer if we are really sure.

Colin-Codes commented 4 years ago

This should be based on prediction results of actionable classes, weighted by occurrence in the dataset.

This can then be used in cross validation scoring and model evaluation

Colin-Codes commented 4 years ago

Can link with cross validation using:

https://scikit-learn.org/stable/modules/model_evaluation.html#scoring-parameter

Colin-Codes commented 4 years ago
  1. Scoring algorithm: This should be based on precision results of actionable classes, weighted by occurrence of those classes in the dataset. This can then be used in cross validation scoring and model evaluation. I'm confident I can handle this.