search

mlcommons / algorithmic-efficiency

MLCommons Algorithmic Efficiency is a benchmark and competition measuring neural network training speedups due to algorithmic improvements in both training algorithms and models.
https://mlcommons.org/en/groups/research-algorithms/
Apache License 2.0
335 stars 69 forks source link

Issue 452: Added the assertion for consistency check and evaluation frequency check #788

Open harneet862 opened 2 months ago

harneet862 commented 2 months ago

For this PR, I added two assertions to validate the timing and evaluation consistency at the end of the training loop:

1) Duration consistency check: The total duration of training (submission time + logging time + evaluation time) is asserted to match the final evaluation result's total_duration, with a tolerance of 10 seconds.

2) Evaluation frequency check: Ensures the number of evaluations does not exceed the allowed evaluations based on the accumulated submission time and the workload's evaluation period.

github-actions[bot] commented 2 months ago

MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅