mlcommons / inference_results_v3.0

This repository contains the results and code for the MLPerf™ Inference v3.0 benchmark.
https://mlcommons.org/en/inference-datacenter-30/
Apache License 2.0
18 stars 15 forks source link

exporting MLPERF_SCRATCH_PATH to something different than the default causes errors in running benchmarks #17

Open JonShelley opened 11 months ago

JonShelley commented 11 months ago

When I try to follow the instructions in the H100 benchmarking guide I noticed that the environment variable MLPERF_SCRATCH_PATH was not propagating through the files. This causes the data to be put in the wrong location and the scripts to fail.

When I make the following modifications to Makefile.const and the Makefile.data then it appears to work for the bert offline case

sed -i s/"bash code\/"/"MLPERF_SCRATCH_PATH=\$(MLPERF_SCRATCH_PATH) bash code\/"/g Makefile.data sed -i s#"MLPERF_SCRATCH_PATH ?= \/home\/mlperf_inference_data"#"MLPERF_SCRATCH_PATH \?\= $MLPERF_SCRATCH_PATH"#g Makefile.const

Also Makefile.const refers to MLPERF_SCRATCH_PATH before it is checked with by the following

Please run export MLPERF_SCRATCH_PATH=<path> to set your scratch space path.

ifneq ($(wildcard /home/mlperf_inference_data),) MLPERF_SCRATCH_PATH ?= /mnt/resource_nvme/H103/mlperf_inference_data endif

So I have to move it above the DATA_DIR section for it to work.