search

mlcommons / algorithmic-efficiency

MLCommons Algorithmic Efficiency is a benchmark and competition measuring neural network training speedups due to algorithmic improvements in both training algorithms and models.
https://mlcommons.org/en/groups/research-algorithms/
Apache License 2.0
329 stars 66 forks source link

Add code for self-reporting #527

Open fsschneider opened 1 year ago

fsschneider commented 1 year ago

Description

We need to check whether the user experience of self-reporting is clear and easy. This might require additional code or scripts to make self-reporting as easy as possible. It is worth thinking about exactly how we expect people to do this, particularly in the external tuning ruleset, but really in both.

See https://github.com/mlcommons/algorithmic-efficiency/pull/476#discussion_r1318012347

priyakasimbeg commented 8 months ago

Added self-reporting instructions in getting started and script to package logs in https://github.com/mlcommons/algorithmic-efficiency/pull/657.

I suppose one remaining question is what medium we want to use to download logs?

priyakasimbeg commented 7 months ago

Perhaps we can use the MLCommons Benchmarking Infra submission tool to allow submitters to upload the logs to as well.