search

jacobgil / confidenceinterval

The long missing library for python confidence intervals
MIT License
131 stars 14 forks source link

Confidence Interval for Macro-F1 is always [0,1] #2

Open vsocrates opened 1 year ago

vsocrates commented 1 year ago

Hi,

Thanks for a great library! How come the macro-f1 has been hardcoded with a CI of [0,1] in this line?

https://github.com/jacobgil/confidenceinterval/blob/7b47bfdf0ce8ec54a2a2d820b7332fd43b15d0d0/confidenceinterval/metrics.py#L527-L528

I believe we should be using the equation from the paper to compute first variance, then computing the CI from that. Thanks!

jacobgil commented 1 year ago

Good catch! Will try updating that in the next few days.

jacobgil commented 1 year ago

Actually that file was an older file and wasn't used by the package. I removed it now. The macro f1 was already implemented here: https://github.com/jacobgil/confidenceinterval/blob/main/confidenceinterval/takahashi_methods.py#L126