huggingface / model-evaluator

Evaluate Transformers from the Hub 🔥
https://huggingface.co/spaces/autoevaluate/model-evaluator
Apache License 2.0
13 stars 7 forks source link

Refactor model filter #40

Closed lewtun closed 2 years ago

lewtun commented 2 years ago

This PR fixes a bug in the way we determined which models were previously evaluated. With this fix, duplicate evaluations should be prevented for most configurations (we still have an edge case with metrics)