cognitive-catalyst / WA-Testing-Tool

Scripts that run against Watson Assistant for K fold validation on training set, testing on blind test, and draw precision curves for comparison.
Apache License 2.0
78 stars 61 forks source link

Scale confusion matrix colors by number of examples #240

Closed andrewrfreed closed 6 months ago

andrewrfreed commented 6 months ago

It is useful to see not just accuracy but number of samples in the intent. If intent1 has 100% accuracy over 1 example and intent2 has 98% accuracy over 50 examples they will both currently be roughly the same shade of black. But intent2 is much more significant to the test (50x more samples!) so it should be darker than intent1.