MLCommons Algorithmic Efficiency is a benchmark and competition measuring neural network training speedups due to algorithmic improvements in both training algorithms and models.
Feature request: allow users to skip eval on train and test
Evaluating on the training and test sets is time-consuming and not necessary for self-reporting results.
We should add a flag that allow the user to skip eval on these datasets, to make scoring faster.
Accordingly, in this scenario we should modify:
goals_reached = (
train_state['validation_goal_reached'] and
train_state['test_goal_reached'])
This would speed up self-evalution even more, by stopping training when validation target is reached, avoiding unnecessary usage of computational resources.
Feature request: allow users to skip eval on train and test
Evaluating on the training and test sets is time-consuming and not necessary for self-reporting results. We should add a flag that allow the user to skip eval on these datasets, to make scoring faster.
Accordingly, in this scenario we should modify:
into:
This would speed up self-evalution even more, by stopping training when validation target is reached, avoiding unnecessary usage of computational resources.