-
The command used to run mlperf inference for resnet50 model on ubuntu with rocm is below:
cm run script --tags=run-mlperf,inference \
--model=resnet50 \
--implementation=reference \
--…
-
To Be Discussed:
I want to clarify for everyone that the official rules for calculating MLPerf scores is timestamp of run_stop minus timestamp of run_start. This has caused some confusion for the "…
-
o, I have now 4 solid test scenarios thanks to everyone's help here. The have all been tested in cpu mode. I am now switching to nvidia and the docker container doesn't seem to build.
I will be t…
-
We propose to add some basic unit test framework (likely pytest) and tests to the inference repo. Ideally, it should test:
- All configuration (mlperf.conf, user.conf) is valid and working (i.e. wi…
-
https://github.com/mlcommons/training_policies/blob/master/training_rules.adoc#14-appendix-benchmark-specific-rules
Here, it is stated that feature caching is not allowed. What is the definition of…
-
(python3-venv) aarch64_sh ~> cm run script --tags=run-mlperf,inference,_find-performance,_full,_r4.1 --model=dlrm_v2-99 --implementation=reference --framework=pytorch --category=datacenter…
-
I'm trying to reproduce the Intel results for gptj-99 and trying to setup the docker by doing a `./build_gpt-j_int4_container.sh`.
However, the build seems to fail due to a bunch of errors
```
…
-
For an Open submission, we have tried setting TTFT/TPOT latencies in `user.conf` e.g.:
mixtral-8x7b.Server.ttft_latency = 3000
mixtral-8x7b.Server.tpot_latency = 300
However, in the resulting…
-
Hi ,
When I'm trying to download tokenizer model from gs://mlperf-llm-public2/vocab/c4_en_301_5Mexp2_spm.model using such command:
`./google-cloud-sdk/bin/gsutil cp -R gs://mlperf-llm-public2/voca…
-
Test written with hard coded paths to python environments that need to moved to a central location or build as part of the test.