Submission rules, especially compliance principles

[petermattson]

Hi all, We're getting close to submission and need to dust off the submission rules. https://docs.google.com/document/d/1YNGymVFuQdoH7tPFHWk6siSkFdvyHqKi3FuBTXaoxfM/edit#heading=h.k6uj9xqxzwoe It'd be especially good to make sure we're still on the same page regarding role of compliance. Best, Peter

[bitfort]

SWG Notes:

There is general support for the same operating principles:

MLPerf v0.6 is a submission round of the “alpha” version of an agile benchmark. The role of the review process is to ensure fairness of submissions, not to litigate details in an effort to disquality competitors. For example:

• Reviewing submitters should discuss issues with owning submitters before filing objections.
• If an objection is supported by the review committee, the objecting submitter should communicate with the owning submitter to ensure a satisfactory fix.
• The review committee should ignore small differences that do not substantially impact performance (less than 2% cumulative difference for a result), subject to its discretion.

AI(Submitters) Please review the document.

[bitfort]

This was a notice for v0.6; already done.