eth-easl / modyn

Modyn is a research-platform for training ML models on dynamic datasets.
MIT License
22 stars 3 forks source link

Add statistics for CGLM dataset #516

Closed MaxiBoether closed 1 week ago

MaxiBoether commented 2 weeks ago

We add some statistic collection for the different CGLM variants and helpful plotting scripts.

PR 1/n of porting SIGMOD changes to main. This one should probably not require an indepth review, mostly stuff I needed. Not the prettiest code, but why bother using defaultdicts :D

codecov[bot] commented 2 weeks ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 82.83%. Comparing base (72ffdca) to head (d6d7b72).

Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #516 +/- ## ======================================= Coverage 82.83% 82.83% ======================================= Files 220 220 Lines 10232 10232 ======================================= Hits 8476 8476 Misses 1756 1756 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

github-actions[bot] commented 2 weeks ago

Line Coverage: -% ( % to main) Branch Coverage: -% ( % to main)

XianzheMa commented 1 week ago

I'd say let's merge this. It's in the benchmark/ directory; the scripts there are just for quick utility purpose.

It's better to have raw code that helps than not have it.

Let me update the branch, address CI, and merge it when CI passes.