Optimise KNN BoW

[Colin-Codes]

optimise by iterating over K, and cross validating

[Colin-Codes]

BoW k = 1(51.6),2(53.1),3(54.7),4(53.6)

Validation accuracy: 0.9840268456375839 Validation accuracy: 0.9821140939597316 Validation accuracy: 0.9758389261744966 Validation accuracy: 0.9684563758389262 Confusion Matrix: [[ 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 64 0 0 0 0 1 0 0 0 0 0 0 2 3 0 0 3 0 1 0 0 0 0 0] [ 0 0 0 0 0 1 1 1 0 0 0 1 0 0 0 0 0 0 1 0 0 0 0 0 1] [ 0 1 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 1 0 0 1 0 0 1 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 1 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 1 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 5 0 1 1 0 1 1 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0] [ 1 1 0 0 0 0 3 1 0 1 1 4 1 2 2 2 0 0 1 0 1 0 1 0 1] [ 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 1 0 0 0 0 0 1 2 0 0 0 1 0 0 1 0 0 1 1] [ 0 2 0 0 0 0 0 0 0 0 0 0 0 0 3 0 0 0 0 0 0 0 0 0 2] [ 0 1 0 0 0 0 0 0 0 1 0 2 0 5 0 4 0 0 0 0 3 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0] [ 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0] [ 0 1 0 0 0 0 1 0 0 0 0 1 0 2 0 0 0 1 2 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 1 0 0 0 2 0 0 0 0 0 0 2 0 0 0 0] [ 0 0 1 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0] [ 0 1 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 1 1] [ 0 0 0 0 0 0 0 0 0 0 0 1 0 1 0 0 0 0 0 0 0 0 0 0 1]] Classification Report: C:\Users\colin\Anaconda3\lib\site-packages\sklearn\metrics\classification.py:1143: UndefinedMetricWarning: Precision and F-score are ill-defined and being set to 0.0 in labels with no predicted samples. 'precision', 'predicted', average, warn_for) precision recall f1-score support

   Access       0.50      1.00      0.67         1
  Account       0.89      0.86      0.88        74
   Action       0.00      0.00      0.00         6
    Admin       0.00      0.00      0.00         5

Authorisation 0.00 0.00 0.00 1 Availability 0.50 0.50 0.50 2 Callback 0.00 0.00 0.00 1 Colour 0.33 0.50 0.40 2 Delivery 0.75 1.00 0.86 3 Documents 0.56 0.56 0.56 9 EqualGlass 0.50 0.50 0.50 2 Error 0.36 0.17 0.24 23 Feedback 0.00 0.00 0.00 2 Forward 0.12 0.25 0.17 8 Gables 0.30 0.43 0.35 7 Information 0.50 0.25 0.33 16 Leaver 1.00 1.00 1.00 1 Logo 0.20 0.67 0.31 3 Pricing 0.50 0.25 0.33 8 Project 0.00 0.00 0.00 1 Reminder 0.22 0.40 0.29 5 Report 0.00 0.00 0.00 3 Status 0.50 1.00 0.67 1 Template 0.33 0.20 0.25 5 Weight 0.14 0.33 0.20 3

micro avg       0.52      0.52      0.52       192
macro avg       0.33      0.39      0.34       192

weighted avg 0.55 0.52 0.52 192

Accuracy Score: 0.515625 Confusion Matrix: [[ 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 64 0 0 0 0 1 0 0 0 0 0 0 3 2 0 0 3 0 1 0 0 0 0 0] [ 0 0 0 0 0 1 1 1 0 0 0 1 0 0 0 1 0 0 1 0 0 0 0 0 0] [ 0 1 0 0 0 0 1 0 0 0 0 0 0 1 0 0 0 1 0 0 0 0 0 1 0] [ 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 1 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 1 0 5 0 1 1 0 0 1 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 1 1 0 0 0 0 4 0 0 1 1 5 1 2 2 2 0 0 1 0 1 0 1 0 0] [ 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 1 0 0 0 0 0 1 3 0 0 0 1 0 0 2 0 0 0 0] [ 0 3 0 0 0 0 0 0 0 0 0 0 0 0 3 0 0 0 0 0 0 0 0 0 1] [ 0 1 0 1 0 0 0 0 0 0 0 3 0 5 0 3 0 0 0 0 3 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0] [ 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0] [ 0 1 0 0 0 0 1 0 0 0 0 1 0 2 0 0 0 0 2 1 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 1 0 0 0 3 0 0 0 0 0 0 1 0 0 0 0] [ 0 0 1 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0] [ 0 1 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 1 1] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 2]] Classification Report: precision recall f1-score support

   Access       0.50      1.00      0.67         1
  Account       0.88      0.86      0.87        74
   Action       0.00      0.00      0.00         6
    Admin       0.00      0.00      0.00         5

Authorisation 0.00 0.00 0.00 1 Availability 0.50 0.50 0.50 2 Callback 0.00 0.00 0.00 1 Colour 0.50 1.00 0.67 2 Delivery 0.75 1.00 0.86 3 Documents 0.62 0.56 0.59 9 EqualGlass 0.67 1.00 0.80 2 Error 0.42 0.22 0.29 23 Feedback 0.00 0.00 0.00 2 Forward 0.15 0.38 0.21 8 Gables 0.43 0.43 0.43 7 Information 0.38 0.19 0.25 16 Leaver 1.00 1.00 1.00 1 Logo 0.22 0.67 0.33 3 Pricing 0.50 0.25 0.33 8 Project 0.00 0.00 0.00 1 Reminder 0.14 0.20 0.17 5 Report 0.00 0.00 0.00 3 Status 0.50 1.00 0.67 1 Template 0.50 0.20 0.29 5 Weight 0.50 0.67 0.57 3

micro avg       0.53      0.53      0.53       192
macro avg       0.37      0.44      0.38       192

weighted avg 0.56 0.53 0.53 192

Accuracy Score: 0.53125 Confusion Matrix: [[ 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 65 0 0 0 0 1 0 0 0 0 0 0 2 2 0 0 2 0 2 0 0 0 0 0] [ 0 0 0 0 0 1 1 1 0 0 0 2 0 0 0 0 0 0 1 0 0 0 0 0 0] [ 0 1 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 1 0 0 0 0 0 1 1] [ 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 1 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 5 0 1 1 0 1 1 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0] [ 1 1 0 0 0 0 3 1 0 1 1 5 1 2 1 3 0 0 1 0 1 0 1 0 0] [ 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 1 0 0 0 0 0 1 2 0 0 0 1 0 0 1 0 0 1 1] [ 0 2 0 0 0 0 0 0 0 0 0 0 0 0 3 0 0 0 0 0 0 0 0 0 2] [ 0 1 0 1 0 0 0 0 0 0 0 3 0 5 0 3 0 0 0 0 3 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0] [ 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0] [ 0 1 0 0 0 0 1 0 0 0 0 1 0 1 0 0 0 0 3 1 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 1 0 0 0 1 0 0 0 0 0 0 3 0 0 0 0] [ 0 0 1 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0] [ 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 2 0 0 0 0 0 1 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3]] Classification Report: precision recall f1-score support

   Access       0.50      1.00      0.67         1
  Account       0.90      0.88      0.89        74
   Action       0.00      0.00      0.00         6
    Admin       0.00      0.00      0.00         5

Authorisation 0.00 0.00 0.00 1 Availability 0.50 0.50 0.50 2 Callback 0.00 0.00 0.00 1 Colour 0.50 1.00 0.67 2 Delivery 0.75 1.00 0.86 3 Documents 0.62 0.56 0.59 9 EqualGlass 0.50 0.50 0.50 2 Error 0.38 0.22 0.28 23 Feedback 0.00 0.00 0.00 2 Forward 0.14 0.25 0.18 8 Gables 0.38 0.43 0.40 7 Information 0.38 0.19 0.25 16 Leaver 1.00 1.00 1.00 1 Logo 0.22 0.67 0.33 3 Pricing 0.60 0.38 0.46 8 Project 0.00 0.00 0.00 1 Reminder 0.33 0.60 0.43 5 Report 0.00 0.00 0.00 3 Status 0.50 1.00 0.67 1 Template 0.33 0.20 0.25 5 Weight 0.43 1.00 0.60 3

micro avg       0.55      0.55      0.55       192
macro avg       0.36      0.45      0.38       192

weighted avg 0.56 0.55 0.54 192

Accuracy Score: 0.546875 Confusion Matrix: [[ 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 65 0 0 0 0 1 0 0 0 0 0 0 2 2 0 0 3 0 1 0 0 0 0 0] [ 0 0 0 0 0 1 1 1 0 0 0 1 0 0 0 0 0 0 1 0 0 0 0 0 1] [ 0 1 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 1 0 0 0 0 0 1 1] [ 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 1 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 5 0 1 1 0 1 1 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0] [ 1 1 0 0 0 0 3 1 0 1 1 4 1 3 1 3 0 0 1 0 1 0 1 0 0] [ 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 1 0 0 0 0 0 1 3 0 0 0 1 0 0 1 0 0 0 1] [ 0 3 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 2] [ 0 1 0 0 0 0 0 0 0 1 0 3 0 5 0 3 0 0 0 0 3 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0] [ 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0] [ 0 2 0 0 0 0 1 0 0 0 0 1 0 1 0 0 0 0 3 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 3 0 0 0 0 0 0 2 0 0 0 0] [ 0 0 1 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0] [ 0 1 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 1 1] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3]] Classification Report: precision recall f1-score support

   Access       0.50      1.00      0.67         1
  Account       0.87      0.88      0.87        74
   Action       0.00      0.00      0.00         6
    Admin       0.00      0.00      0.00         5

Authorisation 0.00 0.00 0.00 1 Availability 0.50 0.50 0.50 2 Callback 0.00 0.00 0.00 1 Colour 0.50 1.00 0.67 2 Delivery 0.75 1.00 0.86 3 Documents 0.62 0.56 0.59 9 EqualGlass 0.50 0.50 0.50 2 Error 0.36 0.17 0.24 23 Feedback 0.00 0.00 0.00 2 Forward 0.17 0.38 0.23 8 Gables 0.33 0.29 0.31 7 Information 0.38 0.19 0.25 16 Leaver 1.00 1.00 1.00 1 Logo 0.22 0.67 0.33 3 Pricing 0.60 0.38 0.46 8 Project 0.00 0.00 0.00 1 Reminder 0.25 0.40 0.31 5 Report 0.00 0.00 0.00 3 Status 0.50 1.00 0.67 1 Template 0.50 0.20 0.29 5 Weight 0.33 1.00 0.50 3

micro avg       0.54      0.54      0.54       192
macro avg       0.36      0.44      0.37       192

weighted avg 0.55 0.54 0.52 192

Accuracy Score: 0.5364583333333334

[Colin-Codes]

BoW k = 1(53.1),5(50.5),10(52.1),15(51.0)

Validation accuracy: 0.9823154362416108 Validation accuracy: 0.9659060402684564 Validation accuracy: 0.9413422818791947 Validation accuracy: 0.9058389261744967 Confusion Matrix: [[ 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 64 0 0 0 0 1 0 0 0 0 0 0 3 2 0 0 3 0 1 0 0 0 0 0] [ 0 0 0 0 0 1 1 1 0 0 0 1 0 0 0 0 0 0 1 0 0 0 0 0 1] [ 0 1 0 0 0 0 1 0 0 0 0 0 0 1 0 0 0 1 0 0 0 0 0 1 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 1 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 1 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 5 0 1 1 0 1 1 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0] [ 1 1 0 0 0 0 2 1 0 1 1 4 1 3 2 2 1 0 1 0 1 0 1 0 0] [ 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 1 0 0 0 0 0 1 3 0 0 0 1 0 0 1 0 0 0 1] [ 0 2 0 0 0 0 0 0 0 0 0 0 0 0 3 0 0 0 0 0 0 0 0 0 2] [ 0 1 0 0 0 0 0 0 0 1 0 2 0 5 0 4 0 0 0 0 3 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0] [ 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0] [ 0 2 0 0 0 0 0 0 0 0 0 1 0 1 0 0 0 0 3 0 1 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 1 0 0 0 3 0 0 0 0 0 0 1 0 0 0 0] [ 0 0 1 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0] [ 0 1 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 1 1] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3]] Classification Report: C:\Users\colin\Anaconda3\lib\site-packages\sklearn\metrics\classification.py:1143: UndefinedMetricWarning: Precision and F-score are ill-defined and being set to 0.0 in labels with no predicted samples. 'precision', 'predicted', average, warn_for) precision recall f1-score support

   Access       0.50      1.00      0.67         1
  Account       0.88      0.86      0.87        74
   Action       0.00      0.00      0.00         6
    Admin       0.00      0.00      0.00         5

Authorisation 0.00 0.00 0.00 1 Availability 0.50 0.50 0.50 2 Callback 0.00 0.00 0.00 1 Colour 0.33 0.50 0.40 2 Delivery 0.75 1.00 0.86 3 Documents 0.62 0.56 0.59 9 EqualGlass 0.50 0.50 0.50 2 Error 0.40 0.17 0.24 23 Feedback 0.00 0.00 0.00 2 Forward 0.15 0.38 0.21 8 Gables 0.33 0.43 0.38 7 Information 0.50 0.25 0.33 16 Leaver 0.50 1.00 0.67 1 Logo 0.22 0.67 0.33 3 Pricing 0.60 0.38 0.46 8 Project 0.00 0.00 0.00 1 Reminder 0.12 0.20 0.15 5 Report 0.00 0.00 0.00 3 Status 0.50 1.00 0.67 1 Template 0.50 0.20 0.29 5 Weight 0.38 1.00 0.55 3

micro avg       0.53      0.53      0.53       192
macro avg       0.33      0.42      0.35       192

weighted avg 0.56 0.53 0.53 192

Accuracy Score: 0.53125 Confusion Matrix: [[ 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 61 0 0 0 0 1 0 0 0 0 0 0 2 6 0 0 2 0 2 0 0 0 0 0] [ 0 0 0 0 0 1 1 1 0 0 0 1 0 0 0 0 0 0 1 0 0 0 0 0 1] [ 0 1 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 1 0 0 0 0 0 1 1] [ 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 1 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 1 0 5 0 1 1 0 0 1 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0] [ 1 1 0 0 0 0 3 1 0 1 1 4 1 3 1 3 0 0 1 0 1 0 1 0 0] [ 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 1 0 0 0 0 0 1 3 0 0 0 1 0 0 1 0 0 0 1] [ 0 3 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 2] [ 0 1 0 0 0 0 0 0 0 1 0 3 0 5 0 3 0 0 0 0 3 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0] [ 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0] [ 0 2 0 0 0 0 1 0 0 0 0 1 0 1 0 0 0 0 3 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 3 0 0 0 0 0 0 2 0 0 0 0] [ 0 0 1 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0] [ 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 2 0 0 0 0 0 1 0] [ 0 0 0 0 0 0 0 0 0 0 0 1 0 1 0 0 0 0 0 0 0 0 0 0 1]] Classification Report: precision recall f1-score support

   Access       0.50      1.00      0.67         1
  Account       0.87      0.82      0.85        74
   Action       0.00      0.00      0.00         6
    Admin       0.00      0.00      0.00         5

Authorisation 0.00 0.00 0.00 1 Availability 0.50 0.50 0.50 2 Callback 0.00 0.00 0.00 1 Colour 0.40 1.00 0.57 2 Delivery 0.75 1.00 0.86 3 Documents 0.62 0.56 0.59 9 EqualGlass 0.50 0.50 0.50 2 Error 0.33 0.17 0.23 23 Feedback 0.00 0.00 0.00 2 Forward 0.16 0.38 0.22 8 Gables 0.20 0.29 0.24 7 Information 0.38 0.19 0.25 16 Leaver 1.00 1.00 1.00 1 Logo 0.22 0.67 0.33 3 Pricing 0.60 0.38 0.46 8 Project 0.00 0.00 0.00 1 Reminder 0.25 0.40 0.31 5 Report 0.00 0.00 0.00 3 Status 0.50 1.00 0.67 1 Template 0.50 0.20 0.29 5 Weight 0.17 0.33 0.22 3

micro avg       0.51      0.51      0.51       192
macro avg       0.34      0.42      0.35       192

weighted avg 0.54 0.51 0.51 192

Accuracy Score: 0.5052083333333334 Confusion Matrix: [[ 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 60 0 0 0 0 1 0 0 0 0 0 0 2 7 0 0 2 0 2 0 0 0 0 0] [ 0 0 0 0 0 1 1 1 0 0 0 1 0 0 0 0 0 0 1 0 0 0 0 0 1] [ 0 1 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 1 0 0 0 0 0 1 1] [ 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 1 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 1 0 0 0 0 0 0 0 5 0 1 1 0 0 1 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0] [ 1 1 0 0 0 0 3 0 0 1 1 5 1 2 1 3 0 0 1 0 2 0 1 0 0] [ 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 1 0 0 0 0 0 1 3 0 0 0 1 0 0 1 0 0 0 1] [ 0 1 0 0 0 0 0 0 0 0 0 0 0 0 4 0 0 0 0 0 0 0 0 0 2] [ 0 1 0 0 0 0 0 0 0 1 0 3 0 5 0 3 0 0 0 0 3 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0] [ 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0] [ 0 2 0 0 0 0 1 0 0 0 0 1 0 1 0 0 0 0 3 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 3 0 0 0 0] [ 0 0 1 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0] [ 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 1 1] [ 0 0 0 0 0 0 0 0 0 0 0 1 0 1 0 0 0 0 0 0 0 0 0 0 1]] Classification Report: precision recall f1-score support

   Access       0.50      1.00      0.67         1
  Account       0.88      0.81      0.85        74
   Action       0.00      0.00      0.00         6
    Admin       0.00      0.00      0.00         5

Authorisation 0.00 0.00 0.00 1 Availability 0.50 0.50 0.50 2 Callback 0.00 0.00 0.00 1 Colour 0.67 1.00 0.80 2 Delivery 0.75 1.00 0.86 3 Documents 0.62 0.56 0.59 9 EqualGlass 0.50 0.50 0.50 2 Error 0.38 0.22 0.28 23 Feedback 0.00 0.00 0.00 2 Forward 0.18 0.38 0.24 8 Gables 0.33 0.57 0.42 7 Information 0.38 0.19 0.25 16 Leaver 1.00 1.00 1.00 1 Logo 0.22 0.67 0.33 3 Pricing 0.60 0.38 0.46 8 Project 0.00 0.00 0.00 1 Reminder 0.30 0.60 0.40 5 Report 0.00 0.00 0.00 3 Status 0.50 1.00 0.67 1 Template 0.50 0.20 0.29 5 Weight 0.14 0.33 0.20 3

micro avg       0.52      0.52      0.52       192
macro avg       0.36      0.44      0.37       192

weighted avg 0.56 0.52 0.52 192

Accuracy Score: 0.5208333333333334 Confusion Matrix: [[ 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 59 0 0 0 0 0 0 0 0 0 0 0 4 7 0 0 2 0 2 0 0 0 0 0] [ 0 0 0 0 0 1 0 1 0 0 0 1 0 1 0 0 0 0 1 0 0 0 0 0 1] [ 0 1 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 1 0 0 0 0 0 1 1] [ 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 1 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 5 0 1 1 0 1 1 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0] [ 1 1 0 0 0 0 3 0 0 1 1 5 1 3 1 3 0 0 1 0 1 0 1 0 0] [ 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 1 0 0 0 0 0 0 3 0 0 0 2 0 0 1 0 0 0 1] [ 0 1 0 0 0 0 0 0 0 0 0 0 0 0 4 0 0 0 0 0 0 0 0 0 2] [ 0 1 0 0 0 0 0 0 0 1 0 3 0 5 0 3 0 0 0 0 3 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0] [ 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0] [ 0 1 0 0 0 0 1 0 0 0 0 1 0 1 0 0 0 0 3 1 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 1 0 0 0 2 0 0 0 0 0 0 2 0 0 0 0] [ 0 0 1 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0] [ 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 2 0 1 0 0 0 1 0] [ 0 0 0 0 0 0 0 0 0 0 0 1 0 1 0 0 0 0 0 0 0 0 0 0 1]] Classification Report: precision recall f1-score support

   Access       0.50      1.00      0.67         1
  Account       0.91      0.80      0.85        74
   Action       0.00      0.00      0.00         6
    Admin       0.00      0.00      0.00         5

Authorisation 0.00 0.00 0.00 1 Availability 0.50 0.50 0.50 2 Callback 0.00 0.00 0.00 1 Colour 0.67 1.00 0.80 2 Delivery 0.75 1.00 0.86 3 Documents 0.56 0.56 0.56 9 EqualGlass 0.50 0.50 0.50 2 Error 0.38 0.22 0.28 23 Feedback 0.00 0.00 0.00 2 Forward 0.14 0.38 0.21 8 Gables 0.31 0.57 0.40 7 Information 0.38 0.19 0.25 16 Leaver 1.00 1.00 1.00 1 Logo 0.20 0.67 0.31 3 Pricing 0.60 0.38 0.46 8 Project 0.00 0.00 0.00 1 Reminder 0.25 0.40 0.31 5 Report 0.00 0.00 0.00 3 Status 0.50 1.00 0.67 1 Template 0.50 0.20 0.29 5 Weight 0.17 0.33 0.22 3

micro avg       0.51      0.51      0.51       192
macro avg       0.35      0.43      0.36       192

weighted avg 0.56 0.51 0.52 192

Accuracy Score: 0.5104166666666666

[Colin-Codes]

BoW k = 10(51.0),50(49.5),100(42.2),150(39.6)

Validation accuracy: 0.938489932885906 Validation accuracy: 0.7415771812080537 Validation accuracy: 0.6299664429530202 Validation accuracy: 0.5556375838926174 Confusion Matrix: [[ 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 60 0 0 0 0 1 0 0 0 0 0 0 2 7 0 0 2 0 2 0 0 0 0 0] [ 0 0 0 0 0 1 0 1 0 0 0 2 0 1 0 0 0 0 0 0 0 0 0 0 1] [ 0 1 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 1 0 0 0 0 0 1 1] [ 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 1 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 1 0 0 0 0 0 0 0 5 0 1 1 0 0 1 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0] [ 1 1 0 0 0 0 3 0 0 1 1 5 1 3 1 3 0 0 1 0 1 0 1 0 0] [ 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 1 0 0 0 0 0 0 3 0 0 0 2 0 0 1 0 0 0 1] [ 0 1 0 0 0 0 0 0 0 0 0 0 0 0 4 0 0 0 0 0 0 0 0 0 2] [ 0 1 0 0 0 0 0 0 0 1 0 3 0 5 0 3 0 0 0 0 3 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0] [ 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0] [ 0 1 0 0 0 0 1 0 0 0 0 1 0 1 0 0 0 0 3 1 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 1 0 0 0 3 0 0 0 0 0 0 1 0 0 0 0] [ 0 0 1 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0] [ 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 1 1] [ 0 0 0 0 0 0 0 0 0 0 0 1 0 1 0 0 0 0 0 0 0 0 0 0 1]] Classification Report: C:\Users\colin\Anaconda3\lib\site-packages\sklearn\metrics\classification.py:1143: UndefinedMetricWarning: Precision and F-score are ill-defined and being set to 0.0 in labels with no predicted samples. 'precision', 'predicted', average, warn_for) precision recall f1-score support

   Access       0.50      1.00      0.67         1
  Account       0.90      0.81      0.85        74
   Action       0.00      0.00      0.00         6
    Admin       0.00      0.00      0.00         5

Authorisation 0.00 0.00 0.00 1 Availability 0.50 0.50 0.50 2 Callback 0.00 0.00 0.00 1 Colour 0.67 1.00 0.80 2 Delivery 0.75 1.00 0.86 3 Documents 0.56 0.56 0.56 9 EqualGlass 0.50 0.50 0.50 2 Error 0.36 0.22 0.27 23 Feedback 0.00 0.00 0.00 2 Forward 0.15 0.38 0.21 8 Gables 0.33 0.57 0.42 7 Information 0.38 0.19 0.25 16 Leaver 1.00 1.00 1.00 1 Logo 0.20 0.67 0.31 3 Pricing 0.75 0.38 0.50 8 Project 0.00 0.00 0.00 1 Reminder 0.14 0.20 0.17 5 Report 0.00 0.00 0.00 3 Status 0.50 1.00 0.67 1 Template 0.50 0.20 0.29 5 Weight 0.14 0.33 0.20 3

micro avg       0.51      0.51      0.51       192
macro avg       0.35      0.42      0.36       192

weighted avg 0.56 0.51 0.52 192

Accuracy Score: 0.5104166666666666 Confusion Matrix: [[ 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 54 0 0 0 0 0 0 0 0 0 0 0 5 10 0 0 2 0 2 1 0 0 0 0] [ 0 0 0 0 0 1 0 1 0 0 0 2 0 1 0 0 0 0 1 0 0 0 0 0 0] [ 0 1 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 1 0 0 0 0 0 1 1] [ 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 1 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 1 0 0 0 0 1 0 0 5 0 0 1 0 0 1 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0] [ 1 0 0 0 0 0 3 1 0 1 1 3 1 3 3 2 0 0 1 0 2 0 1 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 2 0 0 0 0 0 0 3 0 0 0 2 0 0 1 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 6 0 0 0 0 0 0 0 0 0 1] [ 0 1 0 0 0 0 0 0 0 1 0 2 0 6 0 3 0 0 0 0 3 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0] [ 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 1 0 0 0 0 0 0 1 1 1 0 0 3 1 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 1 0 0 0 1 0 0 0 0 0 0 3 0 0 0 0] [ 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0] [ 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 2 0 1 0 0 0 1 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 2]] Classification Report: precision recall f1-score support

   Access       0.50      1.00      0.67         1
  Account       0.93      0.73      0.82        74
   Action       0.00      0.00      0.00         6
    Admin       0.00      0.00      0.00         5

Authorisation 0.00 0.00 0.00 1 Availability 0.50 0.50 0.50 2 Callback 0.00 0.00 0.00 1 Colour 0.50 1.00 0.67 2 Delivery 0.75 1.00 0.86 3 Documents 0.56 0.56 0.56 9 EqualGlass 0.50 0.50 0.50 2 Error 0.38 0.13 0.19 23 Feedback 0.00 0.00 0.00 2 Forward 0.14 0.38 0.20 8 Gables 0.29 0.86 0.43 7 Information 0.38 0.19 0.25 16 Leaver 1.00 1.00 1.00 1 Logo 0.20 0.67 0.31 3 Pricing 0.60 0.38 0.46 8 Project 0.00 0.00 0.00 1 Reminder 0.25 0.60 0.35 5 Report 0.00 0.00 0.00 3 Status 0.50 1.00 0.67 1 Template 0.50 0.20 0.29 5 Weight 0.50 0.67 0.57 3

micro avg       0.49      0.49      0.49       192
macro avg       0.36      0.45      0.37       192

weighted avg 0.57 0.49 0.50 192

Accuracy Score: 0.4947916666666667 Confusion Matrix: [[ 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 42 0 1 0 0 0 0 0 0 0 0 1 6 16 0 0 2 0 4 1 0 0 1 0] [ 0 0 0 0 0 1 0 1 0 0 0 3 0 1 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 1 0 0 0 0 1 0 0 0 1 0 0 0 0 0 1 1] [ 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 1 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 1 0 0 5 0 0 1 0 1 1 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 1 0 0 0 0 0 3 2 0 1 1 2 2 3 3 1 0 0 1 0 2 0 1 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 1 0 0 0 0 0 0] [ 0 0 0 0 0 0 2 0 0 0 0 0 1 3 0 0 0 2 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 6 0 0 0 0 0 0 0 0 0 1] [ 0 1 0 0 0 0 0 0 0 0 0 3 0 5 0 2 0 0 0 0 5 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 1 0 0 0 0 0 0 2 1 0 0 0 2 1 0 0 1 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 1 0 0 0 1 0 0 0 0 0 0 3 0 0 0 0] [ 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0] [ 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 2 0 1 0 0 0 1 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 2]] Classification Report: precision recall f1-score support

   Access       0.50      1.00      0.67         1
  Account       0.98      0.57      0.72        74
   Action       0.00      0.00      0.00         6
    Admin       0.00      0.00      0.00         5

Authorisation 0.00 0.00 0.00 1 Availability 0.50 0.50 0.50 2 Callback 0.00 0.00 0.00 1 Colour 0.33 1.00 0.50 2 Delivery 0.60 1.00 0.75 3 Documents 0.71 0.56 0.63 9 EqualGlass 0.67 1.00 0.80 2 Error 0.22 0.09 0.12 23 Feedback 0.00 0.00 0.00 2 Forward 0.12 0.38 0.19 8 Gables 0.21 0.86 0.34 7 Information 0.50 0.12 0.20 16 Leaver 1.00 1.00 1.00 1 Logo 0.20 0.67 0.31 3 Pricing 0.50 0.25 0.33 8 Project 0.00 0.00 0.00 1 Reminder 0.25 0.60 0.35 5 Report 0.00 0.00 0.00 3 Status 0.33 1.00 0.50 1 Template 0.33 0.20 0.25 5 Weight 0.50 0.67 0.57 3

micro avg       0.42      0.42      0.42       192
macro avg       0.34      0.46      0.35       192

weighted avg 0.57 0.42 0.44 192

Accuracy Score: 0.421875 Confusion Matrix: [[ 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 40 0 2 0 0 3 0 0 0 0 0 1 2 15 0 0 3 0 4 3 0 0 1 0] [ 0 0 0 0 0 1 1 1 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 1] [ 0 0 0 0 0 0 0 0 1 0 0 0 0 1 0 0 0 1 0 0 0 0 0 1 1] [ 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 1 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 1 0 0 5 0 0 1 0 1 0 0 0 0 1 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 1 0 0 0 0 0 3 2 0 1 1 2 2 3 2 1 0 0 0 0 3 0 1 0 1] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 2 0 0 0 0 0 1 3 0 0 0 2 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 6 0 0 0 0 0 0 0 0 0 1] [ 0 1 0 0 0 0 0 0 0 0 0 3 1 4 1 1 0 0 0 0 5 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 1 0 0 0 0 0 0 2 1 0 0 0 2 1 0 0 1 0 0] [ 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 3 0 0 1 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0] [ 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0] [ 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 2 0 1 0 0 0 1 0] [ 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2]] Classification Report: precision recall f1-score support

   Access       0.50      1.00      0.67         1
  Account       0.98      0.54      0.70        74
   Action       0.00      0.00      0.00         6
    Admin       0.00      0.00      0.00         5

Authorisation 0.00 0.00 0.00 1 Availability 0.50 0.50 0.50 2 Callback 0.00 0.00 0.00 1 Colour 0.33 1.00 0.50 2 Delivery 0.60 1.00 0.75 3 Documents 0.71 0.56 0.63 9 EqualGlass 0.67 1.00 0.80 2 Error 0.25 0.09 0.13 23 Feedback 0.00 0.00 0.00 2 Forward 0.19 0.38 0.25 8 Gables 0.22 0.86 0.35 7 Information 0.50 0.06 0.11 16 Leaver 1.00 1.00 1.00 1 Logo 0.18 0.67 0.29 3 Pricing 1.00 0.25 0.40 8 Project 0.00 0.00 0.00 1 Reminder 0.08 0.20 0.11 5 Report 0.00 0.00 0.00 3 Status 0.33 1.00 0.50 1 Template 0.33 0.20 0.25 5 Weight 0.33 0.67 0.44 3

micro avg       0.40      0.40      0.40       192
macro avg       0.35      0.44      0.33       192

weighted avg 0.59 0.40 0.42 192

Accuracy Score: 0.3958333333333333