search

sousanunes / confidence_intervals

Confidence intervals for classification performance metrics
1 stars 0 forks source link

Thanks! #1

Open GregSilverman opened 4 years ago

GregSilverman commented 4 years ago

Hi, I stumbled across this repo, since I needed a way to compute confidence intervals on F1-score for an NLP ensembling problem.

An initial glance at your derivation looks right. I'll let you know the outcome of my use of this.

GregSilverman commented 4 years ago

Hi, first of all, I have to thank you for doing this.

Glancing at the derivation you present for the expression used in f1_score_confidence_interval everything looks copacetic. I won't have time to absolutely validate this by working it out by hand till after next week, but unless I missed something critical, I believe what you have is correct.

That being said, I think your output parameter names in your example run need to be switched, that is x_upper_bound <-> x_lower_bound (since your example upper bound is really the lower bound of the ci and vice versa)...

Thanks again!