feat!: Batch evaluations in evaluator

[robinholzi]

Motivation

We want to utilize the batching logic introduced in https://github.com/eth-easl/modyn/pull/554 by grouping requests in the supervisor.

This will break the analytics tool because of the adjusted logging. Once we have a completed run I'll open a followup PR.

[github-actions[bot]]

^{( % to main) ( % to main)}

[MaxiBoether]

Lmk when i should review this

[XianzheMa]

I think it should be called "batching requests" because we do not actually parallelize them?

[robinholzi]

I think it should be called "batching requests" because we do not actually parallelize them?

Ah right, the evaluator doesn't do parallelization (yet)

[codecov[bot]]

Codecov Report

Attention: Patch coverage is 96.22642% with 2 lines in your changes missing coverage. Please review.

Project coverage is 82.93%. Comparing base (c6622d3) to head (8494963).

Files	Patch %	Lines
...yn/supervisor/internal/pipeline_executor/models.py	81.81%	2 Missing :warning:

Additional details and impacted files

```diff @@ Coverage Diff @@ ## main #559 +/- ## ========================================== + Coverage 82.89% 82.93% +0.04% ========================================== Files 226 226 Lines 10419 10450 +31 ========================================== + Hits 8637 8667 +30 - Misses 1782 1783 +1 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

[robinholzi]

I'll merge this now with the rho-loss unit test failing

[XianzheMa]

the failed one is a slightly flaky test. no problem at all