eth-easl / modyn

Modyn is a research-platform for training ML models on growing datasets.
MIT License
25 stars 3 forks source link

refactor: Adjust pipeline orchestration in `PipelineExecutor` #434

Closed robinholzi closed 3 months ago

robinholzi commented 3 months ago

Motivation

The current state of pipeline_executor.py doesn't follow clear sequential or logical order. Also, the setup doesn't allow to automatically profile and log pipeline stage execution. Restructuring the orchestration of a pipeline in a PipelineExecutor here.

Changes

TODO

robinholzi commented 3 months ago

GitHub could not determine anymore that merge conflicts were already resolved. The multi-stage merge conflict resolution and auto rebase likely bifurcated the git history. Getting rid of the commit history via a squash merge in #437