-
Hi, can we update run results and benchmark results to be clear
- [x] that accuracy is the primary metric,
- [x] time to train and throughput are optional. A
- [x] additionally, per discussion,…
-
**Command:** cm run script --tags=get,ml-model,sdxl,_fp16,_rclone -j
**OS Version:** Ubuntu 22.04 with kernel 6.5.0
**CUDA Version:** 12.0
**Pytorch version:** 2.2.1
**Error Message:**
* cm ru…
-
The MLCommons Task Force on Automation and Reproducibility has increasing number of requests to extend [MLCommons Modular C++ inference library (MIL)](https://github.com/mlcommons/ck/blob/master/cm-ml…
-
Currently only 'https' is supported in CM pull repo and this causes problems to users using `ssh` access like reported [here](https://discord.com/channels/864995525526290502/1167509983810814042/124323…
-
Currently (on 2024-05-24) the latest version of `docker` python package is 7.0.0.
However, acouple of days ago new `requests` library was released, `2.32.0`.
`docker` package [is not compatible](htt…
-
Following below link to download dlrmv2 model, its always failed to download it.
https://github.com/mlcommons/inference/tree/master/recommendation/dlrm_v2/pytorch
command as below:
$ wget https://…
-
Not long after we [deployed](https://github.com/IQSS/dataverse.harvard.edu/issues/294) our [Croissant implementation](https://github.com/gdcc/exporter-croissant) to [Harvard Dataverse](https://dataver…
-
The submission checker has two undefined variables
https://github.com/mlcommons/inference/blob/2a78b2fdc0407b70681771af94d68577f472db77/tools/submission/submission_checker.py#L1964-L1965
-
We have already enabled TEST01 for SDXL - wasn't mandatory for v4.0 (because the proposal came late), but mandatory for v4.1. https://github.com/mlcommons/inference/pull/1574
NVIDIA has checked int…
-
https://cloud.mlcommons.org/index.php/s/LCdW5RM6wgGWbxC
https://cloud.mlcommons.org/index.php/s/DjnCSGyNBkWA4Ro
Pages Not Found error for SDXL model links.