Fix test eval bug for self-tuning ruleset

mlcommons / algorithmic-efficiency

MLCommons Algorithmic Efficiency is a benchmark and competition measuring neural network training speedups due to algorithmic improvements in both training algorithms and models.

https://mlcommons.org/en/groups/research-algorithms/

Apache License 2.0

321 stars 62 forks source link

Fix test eval bug for self-tuning ruleset #700

Closed priyakasimbeg closed 6 months ago

priyakasimbeg commented 6 months ago

The submission_runner module has an obscure check in train_once to set imagenet_v2_data_dir to None for non-imagenet workloads. It only performs the check for the external_tuning ruleset for some reason. As a result the with the self-tuning ruleset other workloads are being passed the default str for imagenet_v2_data_dir as the test_dir and breaking in the test eval.

To fix, this just set the default value for imagenet_v2_data_dir in the flag definition to None.

github-actions[bot] commented 6 months ago

MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅