search

mlcommons / cm4mlops

A collection of portable, reusable and cross-platform automation recipes (CM scripts) to make it easier to build and benchmark AI systems across diverse models, data sets, software and hardware
http://docs.mlcommons.org/cm4mlops/
Apache License 2.0
13 stars 20 forks source link

Support default SUT folder name - submission generation #536

Closed anandhu-eng closed 5 hours ago

anandhu-eng commented 1 week ago

Currently, the SUT folder name for generating an MLPerf Inference Submission is constructed based on keys such as hardware name, framework version, etc., which are first obtained from cm-sut-info.json. If these values are not present, they are then derived from the default SUT folder name. By default, it would be preferable to support using the exact SUT folder name provided by the user.

Hi @arjunsuresh, as discussed, the priority order would be:

  1. cm-sut-info.json
  2. User-provided folder name
anandhu-eng commented 1 week ago

We might have to update the documentation stating that if a user provides cm-sut-info in SUT root folder, it would overwrite the current SUT folder name with the text combination generated from cm-sut-info.json