search

mlcommons / algorithmic-efficiency

MLCommons Algorithmic Efficiency is a benchmark and competition measuring neural network training speedups due to algorithmic improvements in both training algorithms and models.
https://mlcommons.org/en/groups/research-algorithms/
Apache License 2.0
321 stars 62 forks source link

Add `pytorch_num_workers` flag for workloads with PyTorch data loaders #687

Closed runame closed 6 months ago

runame commented 6 months ago

Address #638.

I have added a new flag to set num_workers for workloads with PyTorch data loaders. It defaults to 4 (same as before, with the exception of ImageNet evaluation). I think this flag makes sense because there is no general rule to set num_workers and users might want to tune it to improve performance on their hardware. Actually, the optimal number is probably different for each workload, but I think allowing for a per-workload setting adds too much complexity -- we can revisit this if users request it.

github-actions[bot] commented 6 months ago

MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅

runame commented 6 months ago

Just changed the flag to only apply to eval. I thought it might make sense if people can tune it for training while developing their submission, but you are right that this might lead to errors when self-reporting results.