search

eth-easl / modyn

Modyn is a research-platform for training ML models on growing datasets.
MIT License
27 stars 3 forks source link

Make the reason for evaluation failure explicit #424

Closed XianzheMa closed 5 months ago

XianzheMa commented 5 months ago

This is the last irrelevant PR I singled out from https://github.com/eth-easl/modyn/pull/422.

One evaluation request can fail to start due to many reasons. Previously the reasons are not explicitly reflected. This has two downsides:

  1. In unit test when we want to test that "condition A" must be satisfied to start evaluation, we can only assert the evaluation is not started but the failure could be because of "condition B", i.e. we have no way to verify if "condition A" is properly checked.
  2. On supervisor side, we don't know why the evaluation fails.

This PR makes the failure reason explicit.