search

eth-easl / modyn

Modyn is a research-platform for training ML models on growing datasets.
MIT License
22 stars 3 forks source link

refactor: Add pydantic interface for metrics #501

Closed robinholzi closed 1 month ago

robinholzi commented 1 month ago

Motivation

Small quality of live improvement that makes it easier to configure evaluation metrics (as discussed here)

github-actions[bot] commented 1 month ago

Line Coverage: -% ( % to main) Branch Coverage: -% ( % to main)

MaxiBoether commented 1 month ago

is that ready for review?

robinholzi commented 1 month ago

Ah yes, I think I was just waiting for the tests. The failing integration tests seems unrelated.

codecov[bot] commented 1 month ago

Codecov Report

Attention: Patch coverage is 96.66667% with 3 lines in your changes missing coverage. Please review.

Project coverage is 82.50%. Comparing base (2ec72d2) to head (8430127).

Files Patch % Lines
modyn/config/schema/pipeline/evaluation/metrics.py 96.07% 2 Missing :warning:
modyn/evaluator/internal/metric_factory.py 83.33% 1 Missing :warning:
Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #501 +/- ## ========================================== + Coverage 82.49% 82.50% +0.01% ========================================== Files 214 214 Lines 9966 9949 -17 ========================================== - Hits 8221 8208 -13 + Misses 1745 1741 -4 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.