Open rhiever opened 7 years ago
totally a very important section. We could take part of the notebooks from the book for this.
if you're doing spam detection, maybe a FN isn't so bad, so you can use a metric that focuses on maximizing FP and TP but if you're doing cancer detection, a FN is disastrous, so you would use a metric that focuses in minimizing FN (even at the expense of others) etc.
Great point! I think we explained this verbally in the tutorial (or maybe it was a different talk?), but it would be great to add this to the notebooks!!
Currently the sklearn metrics section discusses a whole bunch of metrics but doesn't seem to go into detail on why you would use one particular metric. One point I usually try to make about metrics is that the "correct" metric depends critically on your problem, e.g.,
IMO it's a good idea to give students an intuition behind why we choose certain metrics in ML.