Data Perf Beta [May 4th]

mlcommons / mlcube

MLCube® is a project that reduces friction for machine learning by ensuring that models are easily portable and reproducible.

https://mlcommons.github.io/mlcube/

Apache License 2.0

153 stars 32 forks source link

Open bitfort opened 2 years ago

relja128 commented 2 years ago

Containerized the benchmarks from Alpha
Containers have to be runnable at home and on DynaBench 2.0. DB2.0 may not be ready by this point, but containers should be aligned with DB2.0 efforts
Needs to run on top of MLCube
[Nice to have] Users should be able to submit scores and have them show up on DynaBench. This may not be the focus of this taskforce, but the people involved will likely work on this

relja128 commented 2 years ago

We didn't have external participants, Colby will test manually.