search

mlcommons / training_policies

Issues related to MLPerf™ training policies, including rules and suggested changes
https://mlcommons.org/en/groups/training
Apache License 2.0
92 stars 66 forks source link

[HPC] Proposal: Improve results website #512

Open nvaprodromou opened 1 year ago

nvaprodromou commented 1 year ago

Introduction:

After collecting feedback from engineers, clients, and press, NVIDIA presented a list of proposals that aim to improve the popularity of the MLPerf HPC benchmark suite. Please see our slide deck for more information on our feedback gathering process and insights.

Proposal: Improve (modernize) results website

Slide 12 in proposals slide deck.

This proposal aims to improve the popularity of the MLPerf HPC benchmark suite by improving on the following aspects:

  1. Simplifies results parsing and understanding [Improves press interest]

Discussion

Pros:

  1. Quick, easy fix
  2. Reduces publics/reporters’ confusion when faced with our results

Cons:

  1. Some reporters have already invested time to understand our current terminology.
    • We believe that reporters will appreciate the clarity and simplicity of the revised terminology.
sparticlesteve commented 1 year ago

I don't know if this really is a quick easy fix, unless you have a simple, specific suggestion.

I believe MLCommons broadly recognizes the need for improved results display. We can rally for this with David, and in meetings, but not sure what else we can do.

TheKanter commented 1 year ago

Hi @nvaprodromou I concur that the results presentation is problematic. I'm actually working with a product manager and we are interviewing folks about how they are used with an eye towards redesigning the results.

This is an area we want to invest in, but it isn't quick :)

nvaprodromou commented 1 year ago

I agree it's not quick and easy - sorry for that. May be the result of copy/paste between filing PRs. Or maybe I grossly misjudged the effort needed. I forget why that statement is there.