search

mlcommons / training_policies

Issues related to MLPerf™ training policies, including rules and suggested changes
https://mlcommons.org/en/groups/training
Apache License 2.0
92 stars 66 forks source link

Seed uniqueness in runs should be more explicitly stated #474

Closed sparticlesteve closed 2 years ago

sparticlesteve commented 2 years ago

To remove any ambiguity, it'd help if the rules were super explicit about how random seeds (if used) need to be unique across all runs. I think it's left somewhat implicit now, including via the requirement to run the package checker, but a clear statement which cannot be misinterpreted would help prevent any future debate.

In HPC weak-scaling submissions we are at the point where a submitter can accidentally have multiple common seeds because of the large number of models drawn and usage of a seed generator (e.g. $RANDOM) which can produce repeats.

sparticlesteve commented 2 years ago

this was fixed in https://github.com/mlcommons/training_policies/pull/479