stanfordnlp / dspy

DSPy: The framework for programming—not prompting—language models
https://dspy.ai
MIT License
19.26k stars 1.47k forks source link

Confusion matrix-based metrics #1660

Open chrico-bu-uab opened 1 month ago

chrico-bu-uab commented 1 month ago

Since the existing frameworks don't allow for confusion matrix-based evaluators or optimization, I created additional evaluator and optimizer classes (Confusion and MCCBootstrapFewShotWithRandomSearch, respectively).

These additions address #556.