search

GEM-benchmark / GEM-metrics

Automatic metrics for GEM tasks
https://gem-benchmark.com
MIT License
60 stars 20 forks source link

Adding Prism and generic ReproReferencedMetric base class for Repro metrics #79

Closed danieldeutsch closed 2 years ago

danieldeutsch commented 2 years ago

In my Repro library I have functionality to run unit tests on-demand. Github Actions supports Docker, so when I merge a metric I first run the metric-specific unit tests with Github Actions on the CPU. Should I also add that here?

tuetschek commented 2 years ago

Sounds like a good idea, if it's not too hard to do 🤔

sebastianGehrmann commented 2 years ago

sgtm as well!