Closed jonathanpeppers closed 2 years ago
.\train.ps1 -seconds 500
The result, was:
| Trainer MicroAccuracy MacroAccuracy Duration #Iteration | |1 AveragedPerceptronOva 0.8921 0.7356 5.3 1 | |2 SdcaMaximumEntropyMulti 0.8930 0.7602 8.4 2 | |3 LightGbmMulti 0.8871 0.7240 22.4 3 | |4 SymbolicSgdLogisticRegressionOva 0.8615 0.7306 5.7 4 | |5 FastTreeOva 0.8897 0.7488 46.4 5 | |6 LinearSvmOva 0.8900 0.7320 3.4 6 | |7 LbfgsLogisticRegressionOva 0.8855 0.7008 5.8 7 | |8 SgdCalibratedOva 0.8854 0.7034 5.3 8 | |9 FastForestOva 0.8822 0.7411 33.9 9 | |10 LbfgsMaximumEntropyMulti 0.8889 0.7110 5.6 10 | |11 AveragedPerceptronOva 0.8402 0.5691 4.5 11 | |12 SdcaMaximumEntropyMulti 0.8900 0.7382 20.4 12 | |13 LinearSvmOva 0.8512 0.6872 3.4 13 | |14 AveragedPerceptronOva 0.8367 0.5208 3.7 14 | |15 SdcaMaximumEntropyMulti 0.8636 0.6372 3.2 15 | |16 LinearSvmOva 0.8878 0.7318 14.8 16 | |17 AveragedPerceptronOva 0.8562 0.6413 7.9 17 | |18 SdcaMaximumEntropyMulti 0.8350 0.5000 3.2 18 | |19 LinearSvmOva 0.8745 0.7536 7.5 19 | |20 AveragedPerceptronOva 0.8495 0.6112 6.1 20 | |21 SdcaMaximumEntropyMulti 0.8905 0.7305 3.3 21 | |22 LinearSvmOva 0.8619 0.6799 3.4 22 | |23 AveragedPerceptronOva 0.8350 0.5011 4.1 23 | |24 SdcaMaximumEntropyMulti 0.8582 0.7301 3.3 24 | |25 LinearSvmOva 0.8816 0.7206 26.7 25 | |26 AveragedPerceptronOva 0.8496 0.6244 4.1 26 | |27 SdcaMaximumEntropyMulti 0.8948 0.7517 3.1 27 | |28 LinearSvmOva 0.8449 0.6296 3.6 28 | |29 AveragedPerceptronOva 0.8350 0.5000 4.1 29 | |30 SdcaMaximumEntropyMulti 0.8908 0.7352 10.6 30 | |31 LinearSvmOva 0.8891 0.7495 21.9 31 | |32 AveragedPerceptronOva 0.8499 0.5971 5.8 32 | |33 SdcaMaximumEntropyMulti 0.8805 0.7333 3.3 33 | |34 LinearSvmOva 0.8770 0.7153 4.0 34 | |35 AveragedPerceptronOva 0.8452 0.5975 10.9 35 | |36 SdcaMaximumEntropyMulti 0.8818 0.6940 3.2 36 | |37 LinearSvmOva 0.8822 0.7358 4.6 37 | |38 AveragedPerceptronOva 0.8452 0.5813 8.7 38 | |39 SdcaMaximumEntropyMulti 0.8724 0.6735 3.2 39 | |40 LinearSvmOva 0.8875 0.7416 3.7 40 | |41 AveragedPerceptronOva 0.8682 0.6827 13.5 41 | |42 SdcaMaximumEntropyMulti 0.8783 0.7648 3.3 42 | |43 LinearSvmOva 0.8775 0.7618 13.9 43 | |44 AveragedPerceptronOva 0.8350 0.5000 4.3 44 | |45 SdcaMaximumEntropyMulti 0.8880 0.7261 5.3 45 | |48 SdcaMaximumEntropyMulti 0.8893 0.7458 19.5 48 | |50 AveragedPerceptronOva 0.8553 0.6013 4.5 50 | |52 LinearSvmOva 0.8832 0.7224 9.1 52 | |53 AveragedPerceptronOva 0.8671 0.6884 9.2 53 | |54 SdcaMaximumEntropyMulti 0.8702 0.6587 3.1 54 | |55 LinearSvmOva 0.8609 0.6802 3.6 55 | |56 AveragedPerceptronOva 0.8385 0.5673 5.0 56 | |57 SdcaMaximumEntropyMulti 0.8402 0.7580 17.8 57 | |58 LinearSvmOva 0.8842 0.7625 3.8 58 | ===============================================Experiment Results================================================= ------------------------------------------------------------------------------------------------------------------ | Summary | ------------------------------------------------------------------------------------------------------------------ |ML Task: multiclass-classification | |Dataset: C:\src\inclusive-code-reviews-ml\comments\classified.csv | |Label : isnegative | |Total experiment time : 493.19671380000005 Secs | |Total number of models explored: 58 | ------------------------------------------------------------------------------------------------------------------ | Top 5 models explored | ------------------------------------------------------------------------------------------------------------------ | Trainer MicroAccuracy MacroAccuracy Duration #Iteration | |1 SdcaMaximumEntropyMulti 0.8948 0.7517 3.1 1 | |2 SdcaMaximumEntropyMulti 0.8930 0.7602 8.4 2 | |3 AveragedPerceptronOva 0.8921 0.7356 5.3 3 | |4 SdcaMaximumEntropyMulti 0.8908 0.7352 10.6 4 | |5 SdcaMaximumEntropyMulti 0.8905 0.7305 3.3 5 | ------------------------------------------------------------------------------------------------------------------
When creating the model we now get:
************************************************************************************************************* * Metrics for Multi-class Classification model *------------------------------------------------------------------------------------------------------------ * Average MicroAccuracy: 0.889 - Standard deviation: (.008) - Confidence Interval 95%: (.008) * Average MacroAccuracy: 0.704 - Standard deviation: (.017) - Confidence Interval 95%: (.017) * Average LogLoss: .292 - Standard deviation: (.026) - Confidence Interval 95%: (.026) * Average LogLossReduction: .351 - Standard deviation: (.051) - Confidence Interval 95%: (.05) * Average Class 0 Precision: 0.896 - Standard deviation: (.008) - Confidence Interval 95%: (.008) * Average Class 1 Precision: 0.812 - Standard deviation: (.058) - Confidence Interval 95%: (.057) *************************************************************************************************************
Compared to main:
************************************************************************************************************* * Metrics for Multi-class Classification model *------------------------------------------------------------------------------------------------------------ * Average MicroAccuracy: 0.886 - Standard deviation: (.013) - Confidence Interval 95%: (.012) * Average MacroAccuracy: 0.732 - Standard deviation: (.026) - Confidence Interval 95%: (.026) * Average LogLoss: .38 - Standard deviation: (.031) - Confidence Interval 95%: (.031) * Average LogLossReduction: .143 - Standard deviation: (.104) - Confidence Interval 95%: (.102) * Average Class 0 Precision: 0.863 - Standard deviation: (.101) - Confidence Interval 95%: (.099) * Average Class 1 Precision: 0.758 - Standard deviation: (.119) - Confidence Interval 95%: (.117) *************************************************************************************************************
.\train.ps1 -seconds 500The result, was:
When creating the model we now get:
Compared to main: