search

haifengl / smile

Statistical Machine Intelligence & Learning Engine
https://haifengl.github.io
Other
5.97k stars 1.13k forks source link

F1, precision & recall for multi-class classification #772

Closed sandervh14 closed 1 month ago

sandervh14 commented 2 months ago

Is your feature request related to a problem? Please describe. I am evaluating a multi-class classification case. The values in the target column and prediction columns can be 0, 1, 2, 3 or 4. The FScore, Precision and Recall calculation results in the following error:

java.lang.IllegalArgumentException: Precision can only be applied to binary classification: 1000

    at smile.validation.metric.Precision.of(Precision.java:54)
    at smile.validation.metric.FScore.of(FScore.java:90)
    at smile.validation.metric.FScore.score(FScore.java:47)

Describe the solution you'd like Could you implement this, please? Calculating precision and recall for multi-class classification should be possible.

Describe alternatives you've considered Not applicable.

Additional context Not applicable.

haifengl commented 2 months ago

Sure, we will add support for multi class. BTW, smile.deep.metric package precision and recall support macro-, micro-, and weighted metrics of multi-class data.

haifengl commented 1 month ago

It is added to master branch now. See tests for examples.

sandervh14 commented 1 month ago

Hi @haifengl,

Thanks for your pointer to smile.deep.metric and for implementing on the master branch! And sorry for replying so late. Due to a higher priority work item added to our backlog, we had postponed the work in which I needed this. Only now I'm getting back to it.

Thanks! Sander

sandervh14 commented 1 month ago

Hi @haifengl,

Could you make a new release? I see on https://github.com/haifengl/smile/blob/master/core/src/main/java/smile/validation/metric/Recall.java that the 3.1.1 release does not yet contain the changes in master. If not possible yet just now, because other issues need fixing before a new version is published, I'll wait. No problem.

Sander

haifengl commented 1 month ago

The new feature has to change the class APIs so that we couldn't include it in a bug release such as 3.1.1. But it will be available in next major release, which we expect soon. Meanwhile, please test it with master branch and let's know if need any additional work. Thanks!