Optimise KNN TFIDF - Githubissues

Colin-Codes commented 4 years ago

by experimentation

Colin-Codes commented 4 years ago

TFIDF k = 1 (54.2),5(52.6),10(52.1),15(51.5)

Using TensorFlow backend. Validation accuracy: 0.972013422818792 Validation accuracy: 0.952248322147651 Validation accuracy: 0.9519463087248322 Validation accuracy: 0.9335234899328859 Confusion Matrix: [[ 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 67 0 1 0 0 0 0 0 0 0 0 0 0 3 0 0 3 0 0 0 0 0 0 0] [ 0 1 0 0 0 1 1 1 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1] [ 0 1 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 1 1 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 1 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 4 0 0 1 0 1 1 0 0 0 0 0 0 2 0 0] [ 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 1 1 0 0 0 0 1 0 1 0 1 3 2 5 2 1 0 0 0 0 1 0 2 2 0] [ 0 1 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 3 0 0 0 0 1 0 0 0 0 0 2 0 0 1 0 0 0 0 0 0 0 1 0] [ 0 2 0 0 0 0 0 0 0 0 0 0 0 0 4 0 0 0 0 0 0 0 0 0 1] [ 0 3 0 0 0 0 0 2 0 0 0 0 0 2 0 3 0 0 0 0 3 0 3 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0] [ 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0] [ 0 2 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 3 0 1 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 1 0 1 1 0 0 0 0 0 0 2 0 0 0 0] [ 0 0 2 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0] [ 0 1 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 2 0 0 0 0 0 1 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 2]] Classification Report: C:\Users\colin\Anaconda3\lib\site-packages\sklearn\metrics\classification.py:1143: UndefinedMetricWarning: Precision and F-score are ill-defined and being set to 0.0 in labels with no predicted samples. 'precision', 'predicted', average, warn_for) precision recall f1-score support

   Access       0.50      1.00      0.67         1
  Account       0.81      0.91      0.85        74
   Action       0.00      0.00      0.00         6
    Admin       0.00      0.00      0.00         5

Authorisation 0.00 0.00 0.00 1 Availability 0.50 0.50 0.50 2 Callback 0.00 0.00 0.00 1 Colour 0.40 1.00 0.57 2 Delivery 0.60 1.00 0.75 3 Documents 1.00 0.44 0.62 9 EqualGlass 0.50 1.00 0.67 2 Error 1.00 0.13 0.23 23 Feedback 0.14 0.50 0.22 2 Forward 0.00 0.00 0.00 8 Gables 0.31 0.57 0.40 7 Information 0.50 0.19 0.27 16 Leaver 1.00 1.00 1.00 1 Logo 0.25 0.67 0.36 3 Pricing 1.00 0.38 0.55 8 Project 1.00 1.00 1.00 1 Reminder 0.25 0.40 0.31 5 Report 0.00 0.00 0.00 3 Status 0.11 1.00 0.20 1 Template 0.20 0.20 0.20 5 Weight 0.50 0.67 0.57 3

micro avg       0.54      0.54      0.54       192
macro avg       0.42      0.50      0.40       192

weighted avg 0.63 0.54 0.52 192

Accuracy Score: 0.5416666666666666 Confusion Matrix: [[ 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 65 0 0 0 0 0 0 0 0 0 0 0 0 5 0 0 2 0 2 0 0 0 0 0] [ 0 2 0 0 0 1 0 1 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 1] [ 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 1 1 1] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 1 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0] [ 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 3 0 0 0 0 0 0 0 3 0 1 0 0 0 0 0 0 0 0 0 0 2 0 0] [ 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 1 1 0 0 0 0 0 1 0 1 2 2 5 2 2 0 0 0 0 1 0 2 3 0] [ 0 0 0 0 0 0 0 0 0 0 1 0 1 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 3 0 0 0 0 0 0 0 0 0 0 2 0 1 1 0 0 0 0 0 0 0 1 0] [ 0 3 0 0 0 0 0 0 0 0 0 0 0 0 3 0 0 0 0 0 0 0 0 0 1] [ 0 4 0 0 0 0 0 2 0 0 0 1 0 1 0 3 0 0 0 0 2 0 3 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0] [ 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0] [ 0 2 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 4 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 1 0 1 1 0 0 0 0 0 0 2 0 0 0 0] [ 0 0 2 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0] [ 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 2 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 2]] Classification Report: precision recall f1-score support

   Access       1.00      1.00      1.00         1
  Account       0.76      0.88      0.81        74
   Action       0.00      0.00      0.00         6
    Admin       0.00      0.00      0.00         5

Authorisation 0.00 0.00 0.00 1 Availability 0.50 0.50 0.50 2 Callback 0.00 0.00 0.00 1 Colour 0.40 1.00 0.57 2 Delivery 0.60 1.00 0.75 3 Documents 1.00 0.33 0.50 9 EqualGlass 0.40 1.00 0.57 2 Error 0.50 0.09 0.15 23 Feedback 0.14 0.50 0.22 2 Forward 0.00 0.00 0.00 8 Gables 0.23 0.43 0.30 7 Information 0.50 0.19 0.27 16 Leaver 1.00 1.00 1.00 1 Logo 0.29 0.67 0.40 3 Pricing 1.00 0.50 0.67 8 Project 0.25 1.00 0.40 1 Reminder 0.33 0.40 0.36 5 Report 0.00 0.00 0.00 3 Status 0.11 1.00 0.20 1 Template 0.29 0.40 0.33 5 Weight 0.40 0.67 0.50 3

micro avg       0.53      0.53      0.53       192
macro avg       0.39      0.50      0.38       192

weighted avg 0.55 0.53 0.49 192

Accuracy Score: 0.5260416666666666 Confusion Matrix: [[ 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 61 0 0 0 0 0 0 0 0 0 0 0 0 8 0 0 2 0 2 0 0 1 0 0] [ 0 2 0 0 0 1 0 1 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 1] [ 0 1 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 1 1 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0] [ 0 1 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0] [ 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 2 0 0 0 0 0 0 0 3 0 1 0 0 0 1 0 0 0 0 0 0 2 0 0] [ 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 1 1 0 0 0 0 1 1 0 1 4 2 4 1 2 0 0 0 0 1 0 3 1 0] [ 0 0 0 0 0 0 0 0 0 0 1 0 1 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 3 0 0 0 0 1 0 0 0 0 0 2 0 0 1 0 0 0 0 0 0 0 1 0] [ 0 3 0 0 0 0 0 0 0 0 0 0 0 0 3 0 0 0 0 0 0 0 0 0 1] [ 0 4 0 0 0 0 0 2 0 0 0 1 0 1 0 3 0 0 0 0 2 0 3 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0] [ 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0] [ 0 2 0 0 0 0 0 0 0 0 0 0 0 1 0 1 0 0 4 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 1 0 1 1 0 0 0 0 0 0 2 0 0 0 0] [ 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0] [ 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 2 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 2]] Classification Report: precision recall f1-score support

   Access       1.00      1.00      1.00         1
  Account       0.74      0.82      0.78        74
   Action       0.00      0.00      0.00         6
    Admin       1.00      0.20      0.33         5

Authorisation 0.00 0.00 0.00 1 Availability 0.50 0.50 0.50 2 Callback 0.00 0.00 0.00 1 Colour 0.33 1.00 0.50 2 Delivery 0.75 1.00 0.86 3 Documents 1.00 0.33 0.50 9 EqualGlass 0.40 1.00 0.57 2 Error 0.67 0.17 0.28 23 Feedback 0.14 0.50 0.22 2 Forward 0.00 0.00 0.00 8 Gables 0.23 0.43 0.30 7 Information 0.38 0.19 0.25 16 Leaver 1.00 1.00 1.00 1 Logo 0.29 0.67 0.40 3 Pricing 1.00 0.50 0.67 8 Project 0.25 1.00 0.40 1 Reminder 0.33 0.40 0.36 5 Report 0.00 0.00 0.00 3 Status 0.09 1.00 0.17 1 Template 0.40 0.40 0.40 5 Weight 0.40 0.67 0.50 3

micro avg       0.52      0.52      0.52       192
macro avg       0.44      0.51      0.40       192

weighted avg 0.59 0.52 0.51 192

Accuracy Score: 0.5208333333333334 Confusion Matrix: [[ 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 60 0 1 0 0 0 0 0 0 0 0 0 0 8 0 0 2 0 2 0 0 1 0 0] [ 0 2 0 0 0 1 0 1 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 1] [ 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 1 1 1] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0] [ 0 1 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 1 0 0 0 0 0 0 0 3 0 1 0 0 1 1 0 0 0 0 0 0 2 0 0] [ 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 1 1 0 0 0 0 1 1 0 1 3 2 4 1 2 0 0 0 0 1 0 3 2 0] [ 0 0 0 0 0 0 0 0 0 0 1 0 1 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 3 0 0 0 0 1 0 0 0 0 0 2 0 0 1 0 0 0 0 0 0 0 1 0] [ 0 3 0 0 0 0 0 0 0 0 0 0 0 0 4 0 0 0 0 0 0 0 0 0 0] [ 0 2 0 0 0 0 0 2 0 0 0 1 0 2 1 3 0 0 0 0 2 0 3 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3 0 0 0 0 0 0 0] [ 0 1 0 0 0 0 0 0 0 0 0 0 0 1 1 0 0 0 4 0 0 0 0 1 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0] [ 0 0 0 0 0 0 0 1 0 0 1 0 0 1 0 0 0 0 0 0 2 0 0 0 0] [ 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0] [ 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 2 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 2]] Classification Report: precision recall f1-score support

   Access       1.00      1.00      1.00         1
  Account       0.79      0.81      0.80        74
   Action       0.00      0.00      0.00         6
    Admin       0.00      0.00      0.00         5

Authorisation 0.00 0.00 0.00 1 Availability 0.50 0.50 0.50 2 Callback 0.00 0.00 0.00 1 Colour 0.25 1.00 0.40 2 Delivery 0.75 1.00 0.86 3 Documents 1.00 0.33 0.50 9 EqualGlass 0.40 1.00 0.57 2 Error 0.60 0.13 0.21 23 Feedback 0.17 0.50 0.25 2 Forward 0.00 0.00 0.00 8 Gables 0.24 0.57 0.33 7 Information 0.43 0.19 0.26 16 Leaver 1.00 1.00 1.00 1 Logo 0.38 1.00 0.55 3 Pricing 1.00 0.50 0.67 8 Project 0.33 1.00 0.50 1 Reminder 0.33 0.40 0.36 5 Report 0.00 0.00 0.00 3 Status 0.09 1.00 0.17 1 Template 0.29 0.40 0.33 5 Weight 0.40 0.67 0.50 3

micro avg       0.52      0.52      0.52       192
macro avg       0.40      0.52      0.39       192

weighted avg 0.58 0.52 0.50 192

Accuracy Score: 0.515625

Colin-Codes commented 4 years ago

TFIDF k = 10 (52.1),50(41.7),100(39.6),150(40.6)

Confusion Matrix: [[ 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 61 0 0 0 0 0 0 0 0 0 0 0 0 8 0 0 2 0 2 0 0 1 0 0] [ 0 2 0 0 0 1 0 1 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 1] [ 0 1 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 1 1 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0] [ 0 1 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0] [ 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 2 0 0 0 0 0 0 0 3 0 1 0 0 0 1 0 0 0 0 0 0 2 0 0] [ 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 1 1 0 0 0 0 1 1 0 1 4 2 4 1 2 0 0 0 0 1 0 3 1 0] [ 0 0 0 0 0 0 0 0 0 0 1 0 1 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 3 0 0 0 0 1 0 0 0 0 0 2 0 0 1 0 0 0 0 0 0 0 1 0] [ 0 3 0 0 0 0 0 0 0 0 0 0 0 0 3 0 0 0 0 0 0 0 0 0 1] [ 0 4 0 0 0 0 0 2 0 0 0 1 0 1 0 3 0 0 0 0 2 0 3 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0] [ 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0] [ 0 2 0 0 0 0 0 0 0 0 0 0 0 1 0 1 0 0 4 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 1 0 1 1 0 0 0 0 0 0 2 0 0 0 0] [ 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0] [ 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 2 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 2]] Classification Report: C:\Users\colin\Anaconda3\lib\site-packages\sklearn\metrics\classification.py:1143: UndefinedMetricWarning: Precision and F-score are ill-defined and being set to 0.0 in labels with no predicted samples. 'precision', 'predicted', average, warn_for) precision recall f1-score support

   Access       1.00      1.00      1.00         1
  Account       0.74      0.82      0.78        74
   Action       0.00      0.00      0.00         6
    Admin       1.00      0.20      0.33         5

Authorisation 0.00 0.00 0.00 1 Availability 0.50 0.50 0.50 2 Callback 0.00 0.00 0.00 1 Colour 0.33 1.00 0.50 2 Delivery 0.75 1.00 0.86 3 Documents 1.00 0.33 0.50 9 EqualGlass 0.40 1.00 0.57 2 Error 0.67 0.17 0.28 23 Feedback 0.14 0.50 0.22 2 Forward 0.00 0.00 0.00 8 Gables 0.23 0.43 0.30 7 Information 0.38 0.19 0.25 16 Leaver 1.00 1.00 1.00 1 Logo 0.29 0.67 0.40 3 Pricing 1.00 0.50 0.67 8 Project 0.25 1.00 0.40 1 Reminder 0.33 0.40 0.36 5 Report 0.00 0.00 0.00 3 Status 0.09 1.00 0.17 1 Template 0.40 0.40 0.40 5 Weight 0.40 0.67 0.50 3

micro avg       0.52      0.52      0.52       192
macro avg       0.44      0.51      0.40       192

weighted avg 0.59 0.52 0.51 192

Accuracy Score: 0.5208333333333334 Confusion Matrix: [[ 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 41 0 3 0 0 0 0 0 0 0 0 0 0 22 0 0 3 0 3 0 0 2 0 0] [ 0 0 0 0 0 1 0 1 0 0 0 0 1 1 2 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 1 1 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 1 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 3 0 0 1 0 2 0 0 0 0 0 0 0 3 0 0] [ 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 1 0 0 0 0 1 1 0 1 3 2 5 2 2 0 0 0 0 1 0 2 2 0] [ 0 0 0 0 0 0 0 0 0 0 1 0 0 0 1 0 0 0 0 0 0 0 0 0 0] [ 0 2 0 0 0 0 1 0 0 0 0 0 2 0 2 0 0 0 0 0 0 0 0 1 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 7 0 0 0 0 0 0 0 0 0 0] [ 0 1 0 0 0 0 0 2 0 0 1 0 0 4 3 0 0 0 0 0 3 0 2 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 0 0 1 4 0 0 0 0 1 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0] [ 0 0 0 0 0 0 0 1 0 0 1 0 0 1 0 0 0 0 0 0 2 0 0 0 0] [ 0 0 2 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 2 0 0 0 0 0 2 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 2]] Classification Report: precision recall f1-score support

   Access       1.00      1.00      1.00         1
  Account       0.93      0.55      0.69        74
   Action       0.00      0.00      0.00         6
    Admin       0.25      0.20      0.22         5

Authorisation 0.00 0.00 0.00 1 Availability 0.50 0.50 0.50 2 Callback 0.00 0.00 0.00 1 Colour 0.29 1.00 0.44 2 Delivery 0.75 1.00 0.86 3 Documents 1.00 0.33 0.50 9 EqualGlass 0.33 1.00 0.50 2 Error 1.00 0.13 0.23 23 Feedback 0.00 0.00 0.00 2 Forward 0.00 0.00 0.00 8 Gables 0.16 1.00 0.27 7 Information 0.00 0.00 0.00 16 Leaver 1.00 1.00 1.00 1 Logo 0.27 1.00 0.43 3 Pricing 1.00 0.50 0.67 8 Project 0.25 1.00 0.40 1 Reminder 0.29 0.40 0.33 5 Report 0.00 0.00 0.00 3 Status 0.09 1.00 0.17 1 Template 0.29 0.40 0.33 5 Weight 1.00 0.67 0.80 3

micro avg       0.42      0.42      0.42       192
macro avg       0.42      0.51      0.37       192

weighted avg 0.65 0.42 0.44 192

Accuracy Score: 0.4166666666666667 Confusion Matrix: [[ 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 38 0 2 0 0 0 0 0 0 0 0 0 0 25 0 0 4 0 3 0 0 2 0 0] [ 0 0 0 0 0 1 0 1 0 0 0 0 1 0 3 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 1 1 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 1 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 3 0 0 1 0 2 0 0 0 0 0 0 0 3 0 0] [ 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 1 0 0 1 0 1 1 0 2 1 2 2 2 1 0 0 0 0 1 0 6 2 0] [ 0 0 0 0 0 0 0 0 0 0 1 0 1 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 1 0 0 0 0 1 0 0 0 0 0 2 0 3 0 0 0 0 0 0 0 0 1 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 7 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 3 0 0 1 0 2 1 5 0 0 0 0 0 2 0 2 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 0 0 1 4 0 0 0 0 1 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0] [ 0 0 0 0 0 0 0 1 0 0 1 0 0 1 0 0 0 0 0 0 2 0 0 0 0] [ 0 0 2 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 2 0 0 0 0 0 2 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 2]] Classification Report: precision recall f1-score support

   Access       1.00      1.00      1.00         1
  Account       0.97      0.51      0.67        74
   Action       0.00      0.00      0.00         6
    Admin       0.33      0.20      0.25         5

Authorisation 0.00 0.00 0.00 1 Availability 0.33 0.50 0.40 2 Callback 0.00 0.00 0.00 1 Colour 0.22 1.00 0.36 2 Delivery 0.75 1.00 0.86 3 Documents 1.00 0.33 0.50 9 EqualGlass 0.29 1.00 0.44 2 Error 1.00 0.04 0.08 23 Feedback 0.11 0.50 0.18 2 Forward 0.00 0.00 0.00 8 Gables 0.13 1.00 0.24 7 Information 0.00 0.00 0.00 16 Leaver 1.00 1.00 1.00 1 Logo 0.25 1.00 0.40 3 Pricing 1.00 0.50 0.67 8 Project 0.25 1.00 0.40 1 Reminder 0.33 0.40 0.36 5 Report 0.00 0.00 0.00 3 Status 0.07 1.00 0.12 1 Template 0.29 0.40 0.33 5 Weight 1.00 0.67 0.80 3

micro avg       0.40      0.40      0.40       192
macro avg       0.41      0.52      0.36       192

weighted avg 0.67 0.40 0.41 192

Accuracy Score: 0.3958333333333333 Confusion Matrix: [[ 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 38 0 2 0 0 0 0 0 0 0 0 0 0 19 0 0 7 0 3 0 0 5 0 0] [ 0 0 0 0 0 1 0 1 0 0 0 0 1 0 3 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 1 1 1] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 1 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 4 0 0 1 0 1 0 0 0 0 0 0 0 3 0 0] [ 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 1 0 0 1 0 2 1 0 2 1 2 1 2 1 0 0 0 0 1 0 6 2 0] [ 0 0 0 0 0 0 0 0 0 0 1 0 1 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 1 0 0 0 0 1 0 0 0 0 0 2 0 3 0 0 1 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 7 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 3 0 0 1 0 2 1 3 1 0 0 0 0 2 0 3 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 2 4 0 0 0 0 1 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0] [ 0 0 0 0 0 0 0 2 0 0 1 0 0 1 0 0 0 0 0 0 1 0 0 0 0] [ 0 0 1 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 1 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 2 0 0 0 0 0 2 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3]] Classification Report: precision recall f1-score support

   Access       1.00      1.00      1.00         1
  Account       0.97      0.51      0.67        74
   Action       0.00      0.00      0.00         6
    Admin       0.00      0.00      0.00         5

Authorisation 0.00 0.00 0.00 1 Availability 0.33 0.50 0.40 2 Callback 0.00 0.00 0.00 1 Colour 0.18 1.00 0.31 2 Delivery 0.60 1.00 0.75 3 Documents 1.00 0.44 0.62 9 EqualGlass 0.29 1.00 0.44 2 Error 1.00 0.04 0.08 23 Feedback 0.11 0.50 0.18 2 Forward 0.00 0.00 0.00 8 Gables 0.17 1.00 0.29 7 Information 0.50 0.06 0.11 16 Leaver 1.00 1.00 1.00 1 Logo 0.18 1.00 0.30 3 Pricing 1.00 0.50 0.67 8 Project 0.25 1.00 0.40 1 Reminder 0.25 0.20 0.22 5 Report 1.00 0.33 0.50 3 Status 0.05 1.00 0.10 1 Template 0.33 0.40 0.36 5 Weight 0.75 1.00 0.86 3

micro avg       0.41      0.41      0.41       192
macro avg       0.44      0.54      0.37       192

weighted avg 0.71 0.41 0.43 192

Accuracy Score: 0.40625

Colin-Codes commented 4 years ago

TFIDF k = 1(54.2),2(55.2),3(55.2),4(54.7)

Validation accuracy: 0.9603691275167785 Validation accuracy: 0.951979865771812 Validation accuracy: 0.9694295302013423 Validation accuracy: 0.9281543624161074 Confusion Matrix: [[ 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 67 0 1 0 0 0 0 0 0 0 0 0 0 3 0 0 3 0 0 0 0 0 0 0] [ 0 1 0 0 0 1 1 1 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1] [ 0 1 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 1 1 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 1 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 4 0 0 1 0 1 1 0 0 0 0 0 0 2 0 0] [ 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 1 1 0 0 0 0 1 0 1 0 1 3 2 5 2 1 0 0 0 0 1 0 2 2 0] [ 0 1 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 3 0 0 0 0 1 0 0 0 0 0 2 0 0 1 0 0 0 0 0 0 0 1 0] [ 0 2 0 0 0 0 0 0 0 0 0 0 0 0 4 0 0 0 0 0 0 0 0 0 1] [ 0 3 0 0 0 0 0 2 0 0 0 0 0 2 0 3 0 0 0 0 3 0 3 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0] [ 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0] [ 0 2 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 3 0 1 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 1 0 1 1 0 0 0 0 0 0 2 0 0 0 0] [ 0 0 2 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0] [ 0 1 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 2 0 0 0 0 0 1 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 2]] Classification Report: C:\Users\colin\Anaconda3\lib\site-packages\sklearn\metrics\classification.py:1143: UndefinedMetricWarning: Precision and F-score are ill-defined and being set to 0.0 in labels with no predicted samples. 'precision', 'predicted', average, warn_for) precision recall f1-score support

   Access       0.50      1.00      0.67         1
  Account       0.81      0.91      0.85        74
   Action       0.00      0.00      0.00         6
    Admin       0.00      0.00      0.00         5

Authorisation 0.00 0.00 0.00 1 Availability 0.50 0.50 0.50 2 Callback 0.00 0.00 0.00 1 Colour 0.40 1.00 0.57 2 Delivery 0.60 1.00 0.75 3 Documents 1.00 0.44 0.62 9 EqualGlass 0.50 1.00 0.67 2 Error 1.00 0.13 0.23 23 Feedback 0.14 0.50 0.22 2 Forward 0.00 0.00 0.00 8 Gables 0.31 0.57 0.40 7 Information 0.50 0.19 0.27 16 Leaver 1.00 1.00 1.00 1 Logo 0.25 0.67 0.36 3 Pricing 1.00 0.38 0.55 8 Project 1.00 1.00 1.00 1 Reminder 0.25 0.40 0.31 5 Report 0.00 0.00 0.00 3 Status 0.11 1.00 0.20 1 Template 0.20 0.20 0.20 5 Weight 0.50 0.67 0.57 3

micro avg       0.54      0.54      0.54       192
macro avg       0.42      0.50      0.40       192

weighted avg 0.63 0.54 0.52 192

Accuracy Score: 0.5416666666666666 Confusion Matrix: [[ 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 69 0 0 0 0 0 0 0 0 0 0 0 0 2 1 0 2 0 0 0 0 0 0 0] [ 0 2 0 0 0 1 1 1 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0] [ 0 1 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 1 1 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 1 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 2 0 0 0 0 0 0 0 4 0 0 1 0 0 0 0 0 0 0 0 0 2 0 0] [ 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 1 3 0 0 0 0 1 0 1 0 1 4 2 4 1 1 0 0 0 0 1 0 1 2 0] [ 0 1 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 3 0 0 0 0 1 0 0 0 0 0 2 0 0 1 0 0 0 0 0 0 0 1 0] [ 0 2 0 0 0 0 0 0 0 0 0 0 0 0 4 0 0 0 0 0 0 0 0 0 1] [ 0 4 0 0 0 0 0 2 0 0 0 0 0 2 0 3 0 0 0 0 2 0 3 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0] [ 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0] [ 0 2 0 0 0 0 0 0 0 0 0 0 0 4 0 0 0 0 2 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 1 0 1 1 0 0 0 0 0 0 2 0 0 0 0] [ 0 0 2 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0] [ 0 2 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 1 0 0 0 0 0 1 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 2]] Classification Report: precision recall f1-score support

   Access       0.50      1.00      0.67         1
  Account       0.75      0.93      0.83        74
   Action       0.00      0.00      0.00         6
    Admin       0.00      0.00      0.00         5

Authorisation 0.00 0.00 0.00 1 Availability 0.50 0.50 0.50 2 Callback 0.00 0.00 0.00 1 Colour 0.33 1.00 0.50 2 Delivery 0.60 1.00 0.75 3 Documents 1.00 0.44 0.62 9 EqualGlass 0.50 1.00 0.67 2 Error 1.00 0.17 0.30 23 Feedback 0.14 0.50 0.22 2 Forward 0.00 0.00 0.00 8 Gables 0.40 0.57 0.47 7 Information 0.50 0.19 0.27 16 Leaver 1.00 1.00 1.00 1 Logo 0.33 0.67 0.44 3 Pricing 1.00 0.25 0.40 8 Project 1.00 1.00 1.00 1 Reminder 0.33 0.40 0.36 5 Report 0.00 0.00 0.00 3 Status 0.12 1.00 0.22 1 Template 0.20 0.20 0.20 5 Weight 0.67 0.67 0.67 3

micro avg       0.55      0.55      0.55       192
macro avg       0.44      0.50      0.40       192

weighted avg 0.62 0.55 0.52 192

Accuracy Score: 0.5520833333333334 Confusion Matrix: [[ 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 68 0 0 0 0 0 0 0 0 0 0 0 0 2 1 0 2 0 1 0 0 0 0 0] [ 0 2 0 0 0 1 1 1 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0] [ 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 1 1 1] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 1 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 2 0 0 0 0 0 0 0 4 0 0 1 0 0 0 0 0 0 0 0 0 2 0 0] [ 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 1 1 0 0 0 0 1 0 1 0 1 3 2 5 1 2 0 0 0 0 1 0 2 2 0] [ 0 0 0 0 0 0 0 0 0 0 1 0 1 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 3 0 0 0 0 0 0 0 0 0 0 2 0 1 1 0 0 0 0 0 0 0 1 0] [ 0 2 0 0 0 0 0 0 0 0 0 0 0 0 4 0 0 0 0 0 0 0 0 0 1] [ 0 4 0 0 0 0 0 2 0 0 0 0 0 2 0 3 0 0 0 0 2 0 3 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0] [ 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0] [ 0 2 0 0 0 0 0 0 0 0 0 0 0 3 0 0 0 0 3 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 1 0 1 1 0 0 0 0 0 0 2 0 0 0 0] [ 0 0 2 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0] [ 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 2 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 2]] Classification Report: precision recall f1-score support

   Access       0.50      1.00      0.67         1
  Account       0.77      0.92      0.84        74
   Action       0.00      0.00      0.00         6
    Admin       0.00      0.00      0.00         5

Authorisation 0.00 0.00 0.00 1 Availability 0.50 0.50 0.50 2 Callback 0.00 0.00 0.00 1 Colour 0.33 1.00 0.50 2 Delivery 0.60 1.00 0.75 3 Documents 1.00 0.44 0.62 9 EqualGlass 0.40 1.00 0.57 2 Error 1.00 0.13 0.23 23 Feedback 0.14 0.50 0.22 2 Forward 0.00 0.00 0.00 8 Gables 0.36 0.57 0.44 7 Information 0.43 0.19 0.26 16 Leaver 1.00 1.00 1.00 1 Logo 0.33 0.67 0.44 3 Pricing 1.00 0.38 0.55 8 Project 0.50 1.00 0.67 1 Reminder 0.33 0.40 0.36 5 Report 0.00 0.00 0.00 3 Status 0.11 1.00 0.20 1 Template 0.33 0.40 0.36 5 Weight 0.50 0.67 0.57 3

micro avg       0.55      0.55      0.55       192
macro avg       0.41      0.51      0.39       192

weighted avg 0.62 0.55 0.52 192

Accuracy Score: 0.5520833333333334 Confusion Matrix: [[ 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 68 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 2 0 2 0 0 0 0 0] [ 0 2 0 0 0 1 1 1 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0] [ 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 1 1 1] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 1 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0] [ 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 3 0 0 0 0 0 0 0 3 0 1 0 0 0 0 0 0 0 0 0 0 2 0 0] [ 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 1 1 0 0 0 0 1 0 1 1 1 3 2 5 1 1 0 0 0 0 1 0 2 2 0] [ 0 0 0 0 0 0 0 0 0 0 1 0 1 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 3 0 0 0 0 0 0 0 0 0 0 2 0 1 1 0 0 0 0 0 0 0 1 0] [ 0 3 0 0 0 0 0 0 0 0 0 0 0 0 3 0 0 0 0 0 0 0 0 0 1] [ 0 4 0 0 0 0 0 2 0 0 0 1 0 1 0 3 0 0 0 0 2 0 3 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0] [ 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0] [ 0 2 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 4 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 1 0 1 1 0 0 0 0 0 0 2 0 0 0 0] [ 0 0 2 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0] [ 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 2 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 2]] Classification Report: precision recall f1-score support

   Access       0.50      1.00      0.67         1
  Account       0.76      0.92      0.83        74
   Action       0.00      0.00      0.00         6
    Admin       0.00      0.00      0.00         5

Authorisation 0.00 0.00 0.00 1 Availability 0.50 0.50 0.50 2 Callback 0.00 0.00 0.00 1 Colour 0.40 1.00 0.57 2 Delivery 0.60 1.00 0.75 3 Documents 0.75 0.33 0.46 9 EqualGlass 0.40 1.00 0.57 2 Error 0.60 0.13 0.21 23 Feedback 0.17 0.50 0.25 2 Forward 0.00 0.00 0.00 8 Gables 0.30 0.43 0.35 7 Information 0.60 0.19 0.29 16 Leaver 1.00 1.00 1.00 1 Logo 0.33 0.67 0.44 3 Pricing 1.00 0.50 0.67 8 Project 0.25 1.00 0.40 1 Reminder 0.33 0.40 0.36 5 Report 0.00 0.00 0.00 3 Status 0.11 1.00 0.20 1 Template 0.33 0.40 0.36 5 Weight 0.50 0.67 0.57 3

micro avg       0.55      0.55      0.55       192
macro avg       0.38      0.51      0.38       192

weighted avg 0.57 0.55 0.51 192

Accuracy Score: 0.546875

Colin-Codes / IntentClassifier-ML-Project

Optimise KNN TFIDF #64