search

Colin-Codes / IntentClassifier-ML-Project

Pyhton, Keras, SciKit-Learn, Matplotlib: Machine learning research project around classification of intent behind tech support emails in order to enable automatic follow up.
0 stars 0 forks source link

Optimise GRU #65

Open Colin-Codes opened 4 years ago

Colin-Codes commented 4 years ago

Optimise by experiment

Colin-Codes commented 4 years ago
GRU 300 dims, 1 epoch, 32 batch, 30 sentence size, 56.25% accuracy

Confusion Matrix: [[ 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 58 0 1 0 0 1 0 0 0 0 0 0 0 5 4 0 1 3 0 1 0 0 0 0] [ 0 0 0 0 1 1 0 0 0 0 0 0 0 0 0 2 0 0 2 0 0 0 0 0 0] [ 0 1 0 1 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 1 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 2 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 4 0 2 0 0 0 3 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 1 1 1 1 0 0 0 0 0 0 1 12 0 0 0 2 0 0 2 1 0 0 0 1 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 1 0 0 0 0 0 0 0 2 1 1 1 1 0 0 0 0 0 0 0 1 0] [ 0 3 0 0 0 0 0 0 0 0 0 0 0 0 3 1 0 0 0 0 0 0 0 0 0] [ 0 0 0 1 0 0 0 1 0 0 0 4 0 0 0 7 0 0 0 0 0 0 1 2 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0] [ 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 3 0 0 0 1 0 0 4 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 1 1 0 2 0 0 0 0 0 0 1 0 0 0 0] [ 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0] [ 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 3 0 0 0 0 0 0 0 1 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 2]] Classification Report: precision recall f1-score support Access 0.50 1.00 0.67 1 Account 0.91 0.78 0.84 74 Action 0.00 0.00 0.00 6 Admin 0.17 0.20 0.18 5 Authorisation 0.00 0.00 0.00 1 Availability 0.50 0.50 0.50 2 Callback 0.00 0.00 0.00 1 Colour 0.33 0.50 0.40 2 Delivery 1.00 0.67 0.80 3 Documents 1.00 0.44 0.62 9 EqualGlass 0.50 1.00 0.67 2 Error 0.43 0.52 0.47 23 Feedback 0.67 1.00 0.80 2 Forward 0.33 0.12 0.18 8 Gables 0.33 0.43 0.38 7 Information 0.28 0.44 0.34 16 Leaver 0.50 1.00 0.67 1 Logo 0.67 0.67 0.67 3 Pricing 0.36 0.50 0.42 8 Project 0.00 0.00 0.00 1 Reminder 0.33 0.20 0.25 5 Report 1.00 0.33 0.50 3 Status 0.25 1.00 0.40 1 Template 0.17 0.20 0.18 5 Weight 1.00 0.67 0.80 3 micro avg 0.56 0.56 0.56 192 macro avg 0.45 0.49 0.43 192 weighted avg 0.61 0.56 0.57 192 Accuracy Score: 0.5625

Colin-Codes commented 4 years ago
GRU 300 dims, 1 epoch, 32 batch, 15 sentence size, 56.25% accuracy

Confusion Matrix: [[ 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 52 0 4 1 0 0 0 0 0 0 0 1 1 7 1 0 1 0 1 1 0 0 4 0] [ 0 0 2 0 0 1 1 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 1 0 0 0 0 0 0 1 0 0 0 1 0 0 0 0 0 0 0 0 1 0 1] [ 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 1 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 1 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 2 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 1 0 0 0 0 0 0 0 5 1 2 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 1 1 1 0 0 0 0 0 0 1 0 14 0 1 0 0 0 0 2 0 1 0 0 1 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0] [ 2 0 0 0 0 0 0 0 0 0 0 0 1 2 0 1 0 0 0 0 1 0 0 1 0] [ 0 2 0 1 0 0 0 0 0 0 0 0 0 0 4 0 0 0 0 0 0 0 0 0 0] [ 0 0 1 0 0 0 0 0 0 0 0 4 0 6 1 4 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0] [ 0 0 0 1 0 0 0 0 0 0 0 0 0 0 1 0 0 1 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 1 0 2 0 0 0 0 5 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 1 0 1 0 0 1 1 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0] [ 0 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 2 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 1]] Classification Report: precision recall f1-score support Access 0.25 1.00 0.40 1 Account 0.91 0.70 0.79 74 Action 0.29 0.33 0.31 6 Admin 0.00 0.00 0.00 5 Authorisation 0.00 0.00 0.00 1 Availability 0.50 0.50 0.50 2 Callback 0.50 1.00 0.67 1 Colour 1.00 0.50 0.67 2 Delivery 1.00 0.67 0.80 3 Documents 0.62 0.56 0.59 9 EqualGlass 0.50 1.00 0.67 2 Error 0.61 0.61 0.61 23 Feedback 0.50 1.00 0.67 2 Forward 0.13 0.25 0.17 8 Gables 0.31 0.57 0.40 7 Information 0.57 0.25 0.35 16 Leaver 1.00 1.00 1.00 1 Logo 0.33 0.33 0.33 3 Pricing 0.62 0.62 0.62 8 Project 0.00 0.00 0.00 1 Reminder 0.17 0.20 0.18 5 Report 0.75 1.00 0.86 3 Status 0.50 1.00 0.67 1 Template 0.25 0.40 0.31 5 Weight 0.50 0.33 0.40 3 micro avg 0.56 0.56 0.56 192 macro avg 0.47 0.55 0.48 192 weighted avg 0.64 0.56 0.58 192 Accuracy Score: 0.5625

Colin-Codes commented 4 years ago
GRU 300 dims, 1 epoch, 32 batch, 5 sentence size, 40.625% accuracy

Confusion Matrix: [[ 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 42 0 1 0 0 0 0 0 3 0 3 1 5 2 10 0 0 1 0 2 0 0 4 0] [ 0 0 0 0 0 1 0 1 0 0 0 0 1 1 0 1 0 0 1 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 2 0 0 0 2 0 0] [ 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 1 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 1 0 0 0 0 0 0] [ 0 0 1 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 1] [ 0 0 0 0 0 0 0 0 0 3 0 6 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0] [ 0 0 2 2 0 0 0 0 0 1 0 11 0 0 1 1 0 0 3 0 0 1 0 1 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 1 0] [ 0 1 0 1 0 0 0 0 0 0 0 1 1 0 0 1 0 0 1 0 1 0 0 1 0] [ 0 1 0 1 0 0 0 0 0 0 0 0 0 1 2 1 0 0 1 0 0 0 0 0 0] [ 0 0 0 0 1 0 0 0 0 0 0 3 0 2 1 4 0 0 1 0 2 0 1 1 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0] [ 0 0 0 1 0 0 0 0 0 0 0 0 0 0 1 0 0 1 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 1 0 1 0 1 0 1 0 0 4 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 1 0 0 3 0 0 0 0 0 0 1 0 0 0 0] [ 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0] [ 0 1 1 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 1 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 1]] Classification Report: C:\Users\colin\Anaconda3\lib\site-packages\sklearn\metrics\classification.py:1143: UndefinedMetricWarning: Precision and F-score are ill-defined and being set to 0.0 in labels with no predicted samples. 'precision', 'predicted', average, warn_for) precision recall f1-score support Access 1.00 1.00 1.00 1 Account 0.91 0.57 0.70 74 Action 0.00 0.00 0.00 6 Admin 0.00 0.00 0.00 5 Authorisation 0.00 0.00 0.00 1 Availability 0.50 0.50 0.50 2 Callback 0.00 0.00 0.00 1 Colour 0.00 0.00 0.00 2 Delivery 0.00 0.00 0.00 3 Documents 0.38 0.33 0.35 9 EqualGlass 0.50 0.50 0.50 2 Error 0.39 0.48 0.43 23 Feedback 0.25 0.50 0.33 2 Forward 0.00 0.00 0.00 8 Gables 0.29 0.29 0.29 7 Information 0.19 0.25 0.22 16 Leaver 1.00 1.00 1.00 1 Logo 1.00 0.33 0.50 3 Pricing 0.27 0.50 0.35 8 Project 0.00 0.00 0.00 1 Reminder 0.11 0.20 0.14 5 Report 0.67 0.67 0.67 3 Status 0.25 1.00 0.40 1 Template 0.11 0.20 0.14 5 Weight 0.50 0.33 0.40 3 micro avg 0.41 0.41 0.41 192 macro avg 0.33 0.35 0.32 192 weighted avg 0.52 0.41 0.44 192 Accuracy Score: 0.40625

Colin-Codes commented 4 years ago
GRU 300 dims, 1 epoch, 32 batch, 20 sentence size, 63.02%; 63.54%; 58.9% accuracy

Confusion Matrix: [[ 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 66 0 2 0 0 0 0 1 0 0 0 0 1 1 1 0 1 0 0 1 0 0 0 0] [ 0 0 0 1 0 2 0 0 0 0 0 1 0 0 0 1 0 0 0 0 1 0 0 0 0] [ 0 0 0 0 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 1 1 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0] [ 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 5 0 2 0 1 0 0 0 0 0 0 0 0 1 0 0] [ 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 1 1 0 0 0 0 1 0 0 11 0 0 0 4 0 0 1 0 0 0 1 3 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 1 0 1 0 0 1 1 1 0 0 0 0 1 0 1 0 0 1 0] [ 0 3 0 1 0 0 0 0 0 0 0 0 0 0 3 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 1 0 1 0 0 6 0 1 0 5 0 0 1 0 0 1 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3 0 0 0 0 0 0 0] [ 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 5 0 0 1 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 1 1 0 0 0 0 0 0 0 0 3 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0] [ 0 1 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 1 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 2]] Classification Report: C:\Users\colin\Anaconda3\lib\site-packages\sklearn\metrics\classification.py:1143: UndefinedMetricWarning: Precision and F-score are ill-defined and being set to 0.0 in labels with no predicted samples. 'precision', 'predicted', average, warn_for) precision recall f1-score support Access 1.00 1.00 1.00 1 Account 0.93 0.89 0.91 74 Action 0.00 0.00 0.00 6 Admin 0.00 0.00 0.00 5 Authorisation 0.00 0.00 0.00 1 Availability 0.33 0.50 0.40 2 Callback 0.00 0.00 0.00 1 Colour 1.00 1.00 1.00 2 Delivery 0.38 1.00 0.55 3 Documents 1.00 0.56 0.71 9 EqualGlass 0.67 1.00 0.80 2 Error 0.46 0.48 0.47 23 Feedback 0.67 1.00 0.80 2 Forward 0.20 0.12 0.15 8 Gables 0.60 0.43 0.50 7 Information 0.45 0.31 0.37 16 Leaver 1.00 1.00 1.00 1 Logo 0.75 1.00 0.86 3 Pricing 0.56 0.62 0.59 8 Project 0.00 0.00 0.00 1 Reminder 0.38 0.60 0.46 5 Report 0.50 1.00 0.67 3 Status 0.25 1.00 0.40 1 Template 0.17 0.20 0.18 5 Weight 1.00 0.67 0.80 3 micro avg 0.63 0.63 0.63 192 macro avg 0.49 0.58 0.50 192 weighted avg 0.65 0.63 0.63 192 Accuracy Score: 0.6302083333333334 Confusion Matrix: [[ 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 66 0 1 0 0 0 0 0 1 0 1 1 1 1 0 0 1 1 0 0 0 0 0 0] [ 0 0 1 0 0 0 0 0 0 0 0 3 0 0 0 2 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 1 1 0 0 0 0 1 0 1 0 0 0 0 0 0 0 1 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 1 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 2 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 6 0 2 0 0 0 1 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 1 1 0 0 0 0 0 1 0 0 13 0 0 0 5 0 0 1 0 1 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 1 0 0 0 0 1 1 1 0 2 0 0 0 0 1 0 0 1 0] [ 0 3 0 0 0 0 0 0 0 0 0 0 0 0 3 0 0 0 0 1 0 0 0 0 0] [ 0 0 0 0 0 0 0 1 0 0 0 4 0 0 1 7 0 0 2 0 1 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 0 1 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 6 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 1 1 0 0 0 2 0 0 0 0 1 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0] [ 0 1 0 1 0 0 0 0 0 0 0 1 0 0 0 1 0 0 0 0 0 0 0 1 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 2]] Classification Report: C:\Users\colin\Anaconda3\lib\site-packages\sklearn\metrics\classification.py:1143: UndefinedMetricWarning: Precision and F-score are ill-defined and being set to 0.0 in labels with no predicted samples. 'precision', 'predicted', average, warn_for) precision recall f1-score support Access 1.00 1.00 1.00 1 Account 0.92 0.89 0.90 74 Action 0.50 0.17 0.25 6 Admin 0.00 0.00 0.00 5 Authorisation 0.00 0.00 0.00 1 Availability 1.00 0.50 0.67 2 Callback 0.00 0.00 0.00 1 Colour 0.33 0.50 0.40 2 Delivery 0.50 0.67 0.57 3 Documents 0.86 0.67 0.75 9 EqualGlass 0.67 1.00 0.80 2 Error 0.42 0.57 0.48 23 Feedback 0.50 1.00 0.67 2 Forward 0.33 0.12 0.18 8 Gables 0.50 0.43 0.46 7 Information 0.32 0.44 0.37 16 Leaver 0.50 1.00 0.67 1 Logo 0.50 0.33 0.40 3 Pricing 0.55 0.75 0.63 8 Project 0.00 0.00 0.00 1 Reminder 0.20 0.20 0.20 5 Report 1.00 1.00 1.00 3 Status 1.00 1.00 1.00 1 Template 0.33 0.20 0.25 5 Weight 1.00 0.67 0.80 3 micro avg 0.64 0.64 0.64 192 macro avg 0.52 0.52 0.50 192 weighted avg 0.64 0.64 0.63 192 Accuracy Score: 0.6354166666666666 Confusion Matrix: [[ 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 64 0 1 0 0 0 0 0 0 0 1 0 2 1 0 0 1 1 0 1 0 2 0 0] [ 0 1 0 1 0 1 0 0 0 0 0 2 0 0 0 0 0 0 0 0 1 0 0 0 0] [ 0 0 0 0 0 0 0 0 1 0 0 0 0 2 0 0 0 0 1 0 0 0 1 0 0] [ 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 1 0 0 1 0 0 0 0 0 0 0 0 1 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 5 1 3 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 1 1 1 0 0 0 0 1 0 0 13 0 0 0 3 0 0 0 0 1 0 0 1 1] [ 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 1 0 0 0 0 0 0 1 0 0 0 1 0 0 1 0 0 0 0 3 0 0 1 0] [ 0 3 0 1 0 0 0 0 0 0 0 0 0 0 3 0 0 0 0 0 0 0 0 0 0] [ 0 2 0 0 0 0 0 0 0 1 0 6 0 1 0 2 0 0 2 0 2 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 1 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 1 0 2 0 0 0 0 5 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 1 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 3 0 0 0 0] [ 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0] [ 0 1 0 0 0 1 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 2 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 2]] Classification Report: C:\Users\colin\Anaconda3\lib\site-packages\sklearn\metrics\classification.py:1143: UndefinedMetricWarning: Precision and F-score are ill-defined and being set to 0.0 in labels with no predicted samples. 'precision', 'predicted', average, warn_for) precision recall f1-score support Access 0.50 1.00 0.67 1 Account 0.86 0.86 0.86 74 Action 0.00 0.00 0.00 6 Admin 0.00 0.00 0.00 5 Authorisation 0.00 0.00 0.00 1 Availability 0.33 0.50 0.40 2 Callback 0.00 0.00 0.00 1 Colour 1.00 1.00 1.00 2 Delivery 0.25 0.33 0.29 3 Documents 0.83 0.56 0.67 9 EqualGlass 0.50 0.50 0.50 2 Error 0.42 0.57 0.48 23 Feedback 0.67 1.00 0.80 2 Forward 0.00 0.00 0.00 8 Gables 0.75 0.43 0.55 7 Information 0.33 0.12 0.18 16 Leaver 1.00 1.00 1.00 1 Logo 0.67 0.67 0.67 3 Pricing 0.50 0.62 0.56 8 Project 0.00 0.00 0.00 1 Reminder 0.20 0.60 0.30 5 Report 1.00 0.67 0.80 3 Status 0.25 1.00 0.40 1 Template 0.50 0.40 0.44 5 Weight 0.67 0.67 0.67 3 micro avg 0.59 0.59 0.59 192 macro avg 0.45 0.50 0.45 192 weighted avg 0.59 0.59 0.58 192 Accuracy Score: 0.5885416666666666

Colin-Codes commented 4 years ago
GRU 300 dims, 1 epoch, 32 batch, 25 sentence size, 53.6% 58.33% accuracy

Confusion Matrix: [[ 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 53 0 0 0 0 0 0 0 0 0 3 0 13 2 0 0 1 1 0 1 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 3 0 1 0 1 0 0 0 0 1 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 4 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 1 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 6 0 3 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 1 0 0 0 0 0 0 0 0 0] [ 0 1 0 1 1 0 0 0 0 2 0 14 0 0 0 1 1 0 1 0 1 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 1 0] [ 0 0 0 0 0 0 1 0 0 0 0 1 1 1 0 1 0 0 0 0 2 0 0 1 0] [ 0 1 0 0 0 0 0 0 0 0 0 1 0 2 2 0 0 0 1 0 0 0 0 0 0] [ 0 1 0 0 0 0 0 0 0 0 0 4 0 3 0 2 0 0 0 0 6 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 1 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 3 0 0 0 1 0 0 3 0 1 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 4 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 2 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0] [ 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 3 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 2]] Classification Report: C:\Users\colin\Anaconda3\lib\site-packages\sklearn\metrics\classification.py:1143: UndefinedMetricWarning: Precision and F-score are ill-defined and being set to 0.0 in labels with no predicted samples. 'precision', 'predicted', average, warn_for) precision recall f1-score support Access 1.00 1.00 1.00 1 Account 0.93 0.72 0.81 74 Action 0.00 0.00 0.00 6 Admin 0.00 0.00 0.00 5 Authorisation 0.00 0.00 0.00 1 Availability 1.00 0.50 0.67 2 Callback 0.00 0.00 0.00 1 Colour 0.00 0.00 0.00 2 Delivery 1.00 1.00 1.00 3 Documents 0.60 0.67 0.63 9 EqualGlass 1.00 0.50 0.67 2 Error 0.39 0.61 0.47 23 Feedback 0.33 0.50 0.40 2 Forward 0.05 0.12 0.07 8 Gables 0.40 0.29 0.33 7 Information 0.25 0.12 0.17 16 Leaver 0.50 1.00 0.67 1 Logo 0.67 0.67 0.67 3 Pricing 0.43 0.38 0.40 8 Project 0.00 0.00 0.00 1 Reminder 0.18 0.80 0.30 5 Report 1.00 0.67 0.80 3 Status 1.00 1.00 1.00 1 Template 0.60 0.60 0.60 5 Weight 1.00 0.67 0.80 3 micro avg 0.54 0.54 0.54 192 macro avg 0.49 0.47 0.46 192 weighted avg 0.60 0.54 0.55 192 Accuracy Score: 0.5364583333333334 Confusion Matrix: [[ 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 57 1 1 0 0 0 0 0 0 0 1 0 0 2 11 0 0 0 0 0 0 0 1 0] [ 0 0 0 0 0 1 0 0 0 0 0 2 0 0 0 1 0 0 1 0 1 0 0 0 0] [ 0 0 0 1 0 0 0 0 0 0 0 0 0 0 1 0 0 0 1 0 0 0 1 1 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 1 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 2 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 7 0 1 0 0 0 1 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 1 0 1 0 0 0 0 0 0 2 0 8 0 0 0 8 0 0 2 0 0 1 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 1 0 0 1 0 0 0 0 0 1 1 0 2 0 0 0 0 1 0 0 1 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 4 2 0 0 0 0 0 0 1 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 1 0 1 1 10 0 1 2 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0] [ 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 1 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 6 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 1 1 0 0 0 2 0 0 0 0 1 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 1 0 0 2 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 1]] Classification Report: C:\Users\colin\Anaconda3\lib\site-packages\sklearn\metrics\classification.py:1143: UndefinedMetricWarning: Precision and F-score are ill-defined and being set to 0.0 in labels with no predicted samples. 'precision', 'predicted', average, warn_for) precision recall f1-score support Access 0.50 1.00 0.67 1 Account 0.98 0.77 0.86 74 Action 0.00 0.00 0.00 6 Admin 0.33 0.20 0.25 5 Authorisation 0.00 0.00 0.00 1 Availability 0.50 0.50 0.50 2 Callback 0.00 0.00 0.00 1 Colour 0.00 0.00 0.00 2 Delivery 1.00 0.67 0.80 3 Documents 0.70 0.78 0.74 9 EqualGlass 0.67 1.00 0.80 2 Error 0.50 0.35 0.41 23 Feedback 0.67 1.00 0.80 2 Forward 0.50 0.12 0.20 8 Gables 0.44 0.57 0.50 7 Information 0.23 0.62 0.34 16 Leaver 1.00 1.00 1.00 1 Logo 0.50 0.33 0.40 3 Pricing 0.43 0.75 0.55 8 Project 0.00 0.00 0.00 1 Reminder 0.17 0.20 0.18 5 Report 0.75 1.00 0.86 3 Status 0.33 1.00 0.50 1 Template 0.40 0.40 0.40 5 Weight 1.00 0.33 0.50 3 micro avg 0.58 0.58 0.58 192 macro avg 0.46 0.50 0.45 192 weighted avg 0.65 0.58 0.59 192 Accuracy Score: 0.5833333333333334

Colin-Codes commented 4 years ago
GRU 300 dims, 1 epoch, 32 batch, 18 sentence size, 55.2% accuracy

Confusion Matrix: [[ 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 61 0 1 0 0 0 1 0 0 0 1 1 3 1 2 0 1 0 0 0 0 0 2 0] [ 0 0 1 0 0 1 0 0 0 0 0 4 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 1 1 0 0 0 0 1 0 0 0 1 0 0 0 0 1 0 0] [ 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 1 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 1 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 1 0 0 0 0 0 0 3 0 4 0 0 0 1 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 1 0 0 0 0 0 0 0 0 0] [ 1 1 1 1 0 0 0 1 0 1 0 11 0 0 0 4 0 0 1 0 0 0 0 0 1] [ 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 2 0 0 0 0 1 1 1 0 1 0 0 0 0 1 0 0 1 0] [ 0 2 0 1 0 0 0 0 0 0 0 0 0 1 2 0 0 0 0 0 0 1 0 0 0] [ 0 0 0 0 1 0 0 1 0 0 0 5 0 2 1 3 0 0 1 0 2 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 1 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 1 0 0 5 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 1 1 0 2 0 1 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0] [ 0 1 0 1 0 0 0 0 0 0 0 1 0 0 0 1 0 0 0 0 0 0 0 1 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3]] Classification Report: C:\Users\colin\Anaconda3\lib\site-packages\sklearn\metrics\classification.py:1143: UndefinedMetricWarning: Precision and F-score are ill-defined and being set to 0.0 in labels with no predicted samples. 'precision', 'predicted', average, warn_for) precision recall f1-score support Access 0.50 1.00 0.67 1 Account 0.94 0.82 0.88 74 Action 0.33 0.17 0.22 6 Admin 0.00 0.00 0.00 5 Authorisation 0.00 0.00 0.00 1 Availability 0.50 0.50 0.50 2 Callback 0.00 0.00 0.00 1 Colour 0.14 0.50 0.22 2 Delivery 0.75 1.00 0.86 3 Documents 0.75 0.33 0.46 9 EqualGlass 0.50 0.50 0.50 2 Error 0.34 0.48 0.40 23 Feedback 0.50 1.00 0.67 2 Forward 0.10 0.12 0.11 8 Gables 0.50 0.29 0.36 7 Information 0.19 0.19 0.19 16 Leaver 1.00 1.00 1.00 1 Logo 0.33 0.33 0.33 3 Pricing 0.56 0.62 0.59 8 Project 0.00 0.00 0.00 1 Reminder 0.00 0.00 0.00 5 Report 0.75 1.00 0.86 3 Status 0.50 1.00 0.67 1 Template 0.25 0.20 0.22 5 Weight 0.75 1.00 0.86 3 micro avg 0.55 0.55 0.55 192 macro avg 0.41 0.48 0.42 192 weighted avg 0.58 0.55 0.56 192 Accuracy Score: 0.5520833333333334

Colin-Codes commented 4 years ago
GRU 300 dims, 1 epoch, 32 batch, 18 sentence size, 55.2% accuracy

Confusion Matrix: [[ 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 58 0 1 0 0 0 0 0 0 0 6 0 3 1 2 0 1 1 0 1 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 3 0 0 0 3 0 0 0 0 0 0 0 0 0] [ 0 0 0 1 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 2 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 1 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0] [ 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 1 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 3 0 6 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 1 1 1 0 0 0 0 0 0 0 0 13 0 0 0 3 0 0 1 0 2 0 0 1 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 1 0 0 0 0 1 1 2 0 1 0 0 0 0 1 0 0 1 0] [ 0 2 0 0 0 0 0 0 0 0 0 2 0 0 3 0 0 0 0 0 0 0 0 0 0] [ 0 1 0 0 0 0 0 0 0 0 0 4 0 1 0 7 0 0 0 0 3 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 2 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 1 0 0 5 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 1 0 0 1 0 1 1 0 0 0 1 0 0 0 0] [ 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0] [ 0 1 1 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 2 0] [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3]] Classification Report: C:\Users\colin\Anaconda3\lib\site-packages\sklearn\metrics\classification.py:1143: UndefinedMetricWarning: Precision and F-score are ill-defined and being set to 0.0 in labels with no predicted samples. 'precision', 'predicted', average, warn_for) precision recall f1-score support Access 0.50 1.00 0.67 1 Account 0.92 0.78 0.85 74 Action 0.00 0.00 0.00 6 Admin 0.50 0.20 0.29 5 Authorisation 0.00 0.00 0.00 1 Availability 1.00 0.50 0.67 2 Callback 0.00 0.00 0.00 1 Colour 1.00 1.00 1.00 2 Delivery 1.00 0.33 0.50 3 Documents 1.00 0.33 0.50 9 EqualGlass 0.67 1.00 0.80 2 Error 0.33 0.57 0.41 23 Feedback 0.67 1.00 0.80 2 Forward 0.18 0.25 0.21 8 Gables 0.75 0.43 0.55 7 Information 0.37 0.44 0.40 16 Leaver 0.33 1.00 0.50 1 Logo 0.67 0.67 0.67 3 Pricing 0.71 0.62 0.67 8 Project 0.00 0.00 0.00 1 Reminder 0.09 0.20 0.13 5 Report 0.00 0.00 0.00 3 Status 0.33 1.00 0.50 1 Template 0.50 0.40 0.44 5 Weight 1.00 1.00 1.00 3 micro avg 0.58 0.58 0.58 192 macro avg 0.50 0.51 0.46 192 weighted avg 0.65 0.58 0.59 192 Accuracy Score: 0.578125