NASA-IMPACT / evalem

An evaluation framework for your large model pipelines
0 stars 0 forks source link

Add "score" key to metric result dict #32

Closed NISH1001 closed 1 year ago

NISH1001 commented 1 year ago

Since we're using Jury, the "score" key was nested inside a dictionary. So, now we've added the key to parent dict as well.

Example:

from evalem import BaseMetrics

print(BaseMetrics.AccuracyMetric()(predictions=["a", "b"], references=["a", "a"]))

# previously, it gave: `{'total_items': 2, 'empty_items': 0, 'accuracy': {'score': 0.5}}`

# now, it give: `{'total_items': 2, 'empty_items': 0, 'score': 0.5, 'accuracy': {'score': 0.5}}`