search

yujonglee / eval

Evaluate your LLM apps, RAG pipeline, any generated text, and more!
MIT License
0 stars 0 forks source link

Support num > 2 in runner #94

Closed yujonglee closed 1 year ago

yujonglee commented 1 year ago

We only support num==2 right now.

https://github.com/fastrepl/fastrepl/blob/2cbf1de3559a68a3f585ea64f176dfe0aa8b93c5/fastrepl/utils/kappa.py#L14-L15

https://www.statsmodels.org/stable/generated/statsmodels.stats.inter_rater.fleiss_kappa.html https://www.statsmodels.org/stable/generated/statsmodels.stats.inter_rater.aggregate_raters.html