GEM-benchmark / GEM-metrics

Automatic metrics for GEM tasks
https://gem-benchmark.com
MIT License
60 stars 20 forks source link

Fixing aggregation function #83

Closed danieldeutsch closed 2 years ago

danieldeutsch commented 2 years ago

I ran into a case where a metric returned a dict that mapped from a str to an int. No aggregation function matched because it isn't type float. The method was returning None and it was hard to find out why.

This PR enables aggregating over ints and raises an error if no aggregation function is found instead of returning None

sebastianGehrmann commented 2 years ago

Thanks!