gfursin
commented
1 year ago Thank you for reporting this issue, @ValouBambou! We didn't check these CM scripts on Arch Linux. We will try to do it now and will keep in touch to check it again ...
Closed ValouBambou closed 1 year ago
gfursin
commented
1 year ago Thank you for reporting this issue, @ValouBambou! We didn't check these CM scripts on Arch Linux. We will try to do it now and will keep in touch to check it again ...
arjunsuresh
commented
1 year ago Can you please try cm rm cache --tags=get,sys-utils-cm, cm pull repo mlcommons@ck and rerun the commands? archlinux installation should now be supported.
ValouBambou
commented
1 year ago Thanks ! The commands are running fine now.
ValouBambou
commented
1 year ago Ok nvm I got another issue with the command cm run script "get dataset object-detection open-images original _validation _500" which runs into an error with a missing module. This module seems to have been installed according to the pip log but in the wrong place since I used a venv. The complete log is:
* Searching for collective script(s) with tags "get,dataset,object-detection,open-images,original" and variations "validation,500"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,dataset,object-detection,open-images,original
- Number of cached script outputs found: 0
- Found script::get-dataset-openimages,0a9d49b644cf4142 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-dataset-openimages
Prepared variations: _validation,_500
- Creating new "cache" script artifact in the CM local repository ...
- Tags: tmp,get,dataset,object-detection,open-images,original,openimages,_validation,_500,script-artifact-0a9d49b644cf4142
- Changing to /home/val/CM/repos/local/cache/1afee6231c5e4add
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "get,sys-utils-cm"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,sys-utils-cm
- Number of cached script outputs found: 1
- Found script::get-sys-utils-cm,bc90993277e84b8e in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-sys-utils-cm
- Found cached script output: /home/val/CM/repos/local/cache/86882f03222a41fc
- Checking dynamic dependencies on other CM scripts:
- Processing env after dependencies ...
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,sys-utils-cm": 0.10 sec.
* Searching for collective script(s) with tags "get,python3"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,python3
- Number of cached script outputs found: 1
- Found script::get-python3,d0b5dd74373f4a62 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-python3
- Found cached script output: /home/val/CM/repos/local/cache/4331568fb9804fc5
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,python,python3,get-python,get-python3": 0.07 sec.
* Searching for collective script(s) with tags "mlperf,inference,source"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,mlperf,inference,source
- Number of cached script outputs found: 1
- Found script::get-mlperf-inference-src,4b57186581024797 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-mlperf-inference-src
Prepared variations: _default,_short-history
- Found cached script output: /home/val/CM/repos/local/cache/77b0a2bc77f444dd
- Checking dynamic dependencies on other CM scripts:
- Processing env after dependencies ...
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,src,source,inference,inference-src,inference-source,mlperf,mlcommons": 0.06 sec.
* Searching for collective script(s) with tags "get,generic-python-lib" and variations "boto3"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,generic-python-lib
- Number of cached script outputs found: 0
- Found script::get-generic-python-lib,94b62a682bc44791 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-generic-python-lib
Prepared variations: _boto3
- Creating new "cache" script artifact in the CM local repository ...
- Tags: tmp,get,generic-python-lib,generic,_boto3,deps-python-version-3.10.8,deps-python-non-virtual,script-artifact-94b62a682bc44791
- Changing to /home/val/CM/repos/local/cache/2dbd41f008544a60
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,cpu"
- Number of scripts found: 1
- Found script::detect-cpu,586c8a43320142f7 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-cpu
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,os"
- Number of scripts found: 1
- Found script::detect-os,863735b7db8c44fc in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-os
- Running preprocess ...
- running time of script "detect-os,detect,os,info": 0.10 sec.
- Processing env after dependencies ...
- Running preprocess ...
- running time of script "detect,cpu,detect-cpu,info": 0.19 sec.
* Searching for collective script(s) with tags "get,python3"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,python3
- Number of cached script outputs found: 1
- Found script::get-python3,d0b5dd74373f4a62 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-python3
- Found cached script output: /home/val/CM/repos/local/cache/4331568fb9804fc5
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,python,python3,get-python,get-python3": 0.08 sec.
- Processing env after dependencies ...
- Running preprocess ...
- Checking prehook dependencies on other CM scripts:
* Searching for collective script(s) with tags "install,generic-python-lib" and variations "boto3"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,install,generic-python-lib
- Number of cached script outputs found: 0
- Found script::install-generic-python-lib,f4f502b7b5d545c8 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/install-generic-python-lib
Prepared variations: _boto3
- Creating new "cache" script artifact in the CM local repository ...
- Tags: tmp,install,generic-python-lib,generic,_boto3,deps-python-version-3.10.8,deps-python-non-virtual,script-artifact-f4f502b7b5d545c8
- Changing to /home/val/CM/repos/local/cache/530335bb765c4f8e
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,cpu"
- Number of scripts found: 1
- Found script::detect-cpu,586c8a43320142f7 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-cpu
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,os"
- Number of scripts found: 1
- Found script::detect-os,863735b7db8c44fc in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-os
- Running preprocess ...
- running time of script "detect-os,detect,os,info": 0.09 sec.
- Processing env after dependencies ...
- Running preprocess ...
- running time of script "detect,cpu,detect-cpu,info": 0.18 sec.
* Searching for collective script(s) with tags "get,python"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,python
- Number of cached script outputs found: 1
- Found script::get-python3,d0b5dd74373f4a62 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-python3
- Found cached script output: /home/val/CM/repos/local/cache/4331568fb9804fc5
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,python,python3,get-python,get-python3": 0.08 sec.
- Processing env after dependencies ...
Defaulting to user installation because normal site-packages is not writeable
Collecting boto3
Downloading boto3-1.26.9-py3-none-any.whl (132 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 132.5/132.5 kB 3.1 MB/s eta 0:00:00
Collecting botocore<1.30.0,>=1.29.9
Downloading botocore-1.29.9-py3-none-any.whl (9.9 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 9.9/9.9 MB 2.2 MB/s eta 0:00:00
Collecting s3transfer<0.7.0,>=0.6.0
Downloading s3transfer-0.6.0-py3-none-any.whl (79 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 79.6/79.6 kB 10.1 MB/s eta 0:00:00
Collecting jmespath<2.0.0,>=0.7.1
Downloading jmespath-1.0.1-py3-none-any.whl (20 kB)
Collecting python-dateutil<3.0.0,>=2.1
Downloading python_dateutil-2.8.2-py2.py3-none-any.whl (247 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 247.7/247.7 kB 10.1 MB/s eta 0:00:00
Requirement already satisfied: urllib3<1.27,>=1.25.4 in /usr/lib/python3.10/site-packages (from botocore<1.30.0,>=1.29.9->boto3) (1.26.12)
Requirement already satisfied: six>=1.5 in /usr/lib/python3.10/site-packages (from python-dateutil<3.0.0,>=2.1->botocore<1.30.0,>=1.29.9->boto3) (1.16.0)
Installing collected packages: python-dateutil, jmespath, botocore, s3transfer, boto3
Successfully installed boto3-1.26.9 botocore-1.29.9 jmespath-1.0.1 python-dateutil-2.8.2 s3transfer-0.6.0
- Removing tmp tag in the script cached output 530335bb765c4f8e ...
- running time of script "install,generic,generic-python-lib": 8.20 sec.
Detected version: 1.26.9
- Removing tmp tag in the script cached output 2dbd41f008544a60 ...
- running time of script "get,generic,generic-python-lib": 8.92 sec.
* Searching for collective script(s) with tags "get,generic-python-lib" and variations "tqdm"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,generic-python-lib
- Number of cached script outputs found: 1
- Found script::get-generic-python-lib,94b62a682bc44791 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-generic-python-lib
Prepared variations: _tqdm
- Creating new "cache" script artifact in the CM local repository ...
- Tags: tmp,get,generic-python-lib,generic,_tqdm,deps-python-version-3.10.8,deps-python-non-virtual,script-artifact-94b62a682bc44791
- Changing to /home/val/CM/repos/local/cache/368ad3f2acb94751
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,cpu"
- Number of scripts found: 1
- Found script::detect-cpu,586c8a43320142f7 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-cpu
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,os"
- Number of scripts found: 1
- Found script::detect-os,863735b7db8c44fc in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-os
- Running preprocess ...
- running time of script "detect-os,detect,os,info": 0.09 sec.
- Processing env after dependencies ...
- Running preprocess ...
- running time of script "detect,cpu,detect-cpu,info": 0.18 sec.
* Searching for collective script(s) with tags "get,python3"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,python3
- Number of cached script outputs found: 1
- Found script::get-python3,d0b5dd74373f4a62 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-python3
- Found cached script output: /home/val/CM/repos/local/cache/4331568fb9804fc5
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,python,python3,get-python,get-python3": 0.07 sec.
- Processing env after dependencies ...
- Running preprocess ...
- Checking prehook dependencies on other CM scripts:
* Searching for collective script(s) with tags "install,generic-python-lib" and variations "tqdm"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,install,generic-python-lib
- Number of cached script outputs found: 1
- Found script::install-generic-python-lib,f4f502b7b5d545c8 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/install-generic-python-lib
Prepared variations: _tqdm
- Creating new "cache" script artifact in the CM local repository ...
- Tags: tmp,install,generic-python-lib,generic,_tqdm,deps-python-version-3.10.8,deps-python-non-virtual,script-artifact-f4f502b7b5d545c8
- Changing to /home/val/CM/repos/local/cache/56300d7327844051
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,cpu"
- Number of scripts found: 1
- Found script::detect-cpu,586c8a43320142f7 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-cpu
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,os"
- Number of scripts found: 1
- Found script::detect-os,863735b7db8c44fc in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-os
- Running preprocess ...
- running time of script "detect-os,detect,os,info": 0.09 sec.
- Processing env after dependencies ...
- Running preprocess ...
- running time of script "detect,cpu,detect-cpu,info": 0.18 sec.
* Searching for collective script(s) with tags "get,python"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,python
- Number of cached script outputs found: 1
- Found script::get-python3,d0b5dd74373f4a62 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-python3
- Found cached script output: /home/val/CM/repos/local/cache/4331568fb9804fc5
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,python,python3,get-python,get-python3": 0.08 sec.
- Processing env after dependencies ...
Defaulting to user installation because normal site-packages is not writeable
Collecting tqdm
Using cached tqdm-4.64.1-py2.py3-none-any.whl (78 kB)
Installing collected packages: tqdm
Successfully installed tqdm-4.64.1
- Removing tmp tag in the script cached output 56300d7327844051 ...
- running time of script "install,generic,generic-python-lib": 1.36 sec.
Detected version: 4.64.1
- Removing tmp tag in the script cached output 368ad3f2acb94751 ...
- running time of script "get,generic,generic-python-lib": 2.03 sec.
* Searching for collective script(s) with tags "get,generic-python-lib" and variations "numpy"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,generic-python-lib
- Number of cached script outputs found: 2
- Found script::get-generic-python-lib,94b62a682bc44791 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-generic-python-lib
Prepared variations: _numpy
- Creating new "cache" script artifact in the CM local repository ...
- Tags: tmp,get,generic-python-lib,generic,_numpy,deps-python-version-3.10.8,deps-python-non-virtual,script-artifact-94b62a682bc44791
- Changing to /home/val/CM/repos/local/cache/5bdefffc0ba7428b
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,cpu"
- Number of scripts found: 1
- Found script::detect-cpu,586c8a43320142f7 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-cpu
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,os"
- Number of scripts found: 1
- Found script::detect-os,863735b7db8c44fc in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-os
- Running preprocess ...
- running time of script "detect-os,detect,os,info": 0.09 sec.
- Processing env after dependencies ...
- Running preprocess ...
- running time of script "detect,cpu,detect-cpu,info": 0.18 sec.
* Searching for collective script(s) with tags "get,python3"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,python3
- Number of cached script outputs found: 1
- Found script::get-python3,d0b5dd74373f4a62 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-python3
- Found cached script output: /home/val/CM/repos/local/cache/4331568fb9804fc5
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,python,python3,get-python,get-python3": 0.08 sec.
- Processing env after dependencies ...
- Running preprocess ...
Detected version: 1.23.4
- Checking prehook dependencies on other CM scripts:
Detected version: 1.23.4
- Removing tmp tag in the script cached output 5bdefffc0ba7428b ...
- running time of script "get,generic,generic-python-lib": 0.65 sec.
* Searching for collective script(s) with tags "get,generic-python-lib" and variations "opencv-python"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,generic-python-lib
- Number of cached script outputs found: 3
- Found script::get-generic-python-lib,94b62a682bc44791 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-generic-python-lib
Prepared variations: _opencv-python
- Creating new "cache" script artifact in the CM local repository ...
- Tags: tmp,get,generic-python-lib,generic,_opencv-python,deps-python-version-3.10.8,deps-python-non-virtual,script-artifact-94b62a682bc44791
- Changing to /home/val/CM/repos/local/cache/e129c842b01741b1
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,cpu"
- Number of scripts found: 1
- Found script::detect-cpu,586c8a43320142f7 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-cpu
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,os"
- Number of scripts found: 1
- Found script::detect-os,863735b7db8c44fc in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-os
- Running preprocess ...
- running time of script "detect-os,detect,os,info": 0.10 sec.
- Processing env after dependencies ...
- Running preprocess ...
- running time of script "detect,cpu,detect-cpu,info": 0.19 sec.
* Searching for collective script(s) with tags "get,python3"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,python3
- Number of cached script outputs found: 1
- Found script::get-python3,d0b5dd74373f4a62 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-python3
- Found cached script output: /home/val/CM/repos/local/cache/4331568fb9804fc5
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,python,python3,get-python,get-python3": 0.08 sec.
- Processing env after dependencies ...
- Running preprocess ...
- Checking prehook dependencies on other CM scripts:
* Searching for collective script(s) with tags "install,generic-python-lib" and variations "opencv-python"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,install,generic-python-lib
- Number of cached script outputs found: 2
- Found script::install-generic-python-lib,f4f502b7b5d545c8 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/install-generic-python-lib
Prepared variations: _opencv-python
- Creating new "cache" script artifact in the CM local repository ...
- Tags: tmp,install,generic-python-lib,generic,_opencv-python,deps-python-version-3.10.8,deps-python-non-virtual,script-artifact-f4f502b7b5d545c8
- Changing to /home/val/CM/repos/local/cache/fcd68f9c20c146e0
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,cpu"
- Number of scripts found: 1
- Found script::detect-cpu,586c8a43320142f7 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-cpu
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,os"
- Number of scripts found: 1
- Found script::detect-os,863735b7db8c44fc in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-os
- Running preprocess ...
- running time of script "detect-os,detect,os,info": 0.09 sec.
- Processing env after dependencies ...
- Running preprocess ...
- running time of script "detect,cpu,detect-cpu,info": 0.19 sec.
* Searching for collective script(s) with tags "get,python"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,python
- Number of cached script outputs found: 1
- Found script::get-python3,d0b5dd74373f4a62 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-python3
- Found cached script output: /home/val/CM/repos/local/cache/4331568fb9804fc5
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,python,python3,get-python,get-python3": 0.08 sec.
- Processing env after dependencies ...
Defaulting to user installation because normal site-packages is not writeable
Collecting opencv-python
Downloading opencv_python-4.6.0.66-cp36-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (60.9 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 60.9/60.9 MB 10.1 MB/s eta 0:00:00
Requirement already satisfied: numpy>=1.21.2 in /home/val/.local/lib/python3.10/site-packages (from opencv-python) (1.23.4)
Installing collected packages: opencv-python
Successfully installed opencv-python-4.6.0.66
- Removing tmp tag in the script cached output fcd68f9c20c146e0 ...
- running time of script "install,generic,generic-python-lib": 7.76 sec.
Detected version: 4.6.0.66
- Removing tmp tag in the script cached output e129c842b01741b1 ...
- running time of script "get,generic,generic-python-lib": 8.44 sec.
* Searching for collective script(s) with tags "get,generic-python-lib" and variations "pandas"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,generic-python-lib
- Number of cached script outputs found: 4
- Found script::get-generic-python-lib,94b62a682bc44791 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-generic-python-lib
Prepared variations: _pandas
- Creating new "cache" script artifact in the CM local repository ...
- Tags: tmp,get,generic-python-lib,generic,_pandas,deps-python-version-3.10.8,deps-python-non-virtual,script-artifact-94b62a682bc44791
- Changing to /home/val/CM/repos/local/cache/8c96b4eed2564412
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,cpu"
- Number of scripts found: 1
- Found script::detect-cpu,586c8a43320142f7 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-cpu
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,os"
- Number of scripts found: 1
- Found script::detect-os,863735b7db8c44fc in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-os
- Running preprocess ...
- running time of script "detect-os,detect,os,info": 0.09 sec.
- Processing env after dependencies ...
- Running preprocess ...
- running time of script "detect,cpu,detect-cpu,info": 0.18 sec.
* Searching for collective script(s) with tags "get,python3"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,python3
- Number of cached script outputs found: 1
- Found script::get-python3,d0b5dd74373f4a62 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-python3
- Found cached script output: /home/val/CM/repos/local/cache/4331568fb9804fc5
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,python,python3,get-python,get-python3": 0.07 sec.
- Processing env after dependencies ...
- Running preprocess ...
- Checking prehook dependencies on other CM scripts:
* Searching for collective script(s) with tags "install,generic-python-lib" and variations "pandas"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,install,generic-python-lib
- Number of cached script outputs found: 3
- Found script::install-generic-python-lib,f4f502b7b5d545c8 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/install-generic-python-lib
Prepared variations: _pandas
- Creating new "cache" script artifact in the CM local repository ...
- Tags: tmp,install,generic-python-lib,generic,_pandas,deps-python-version-3.10.8,deps-python-non-virtual,script-artifact-f4f502b7b5d545c8
- Changing to /home/val/CM/repos/local/cache/5458ebbf02c64655
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,cpu"
- Number of scripts found: 1
- Found script::detect-cpu,586c8a43320142f7 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-cpu
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,os"
- Number of scripts found: 1
- Found script::detect-os,863735b7db8c44fc in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-os
- Running preprocess ...
- running time of script "detect-os,detect,os,info": 0.09 sec.
- Processing env after dependencies ...
- Running preprocess ...
- running time of script "detect,cpu,detect-cpu,info": 0.18 sec.
* Searching for collective script(s) with tags "get,python"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,python
- Number of cached script outputs found: 1
- Found script::get-python3,d0b5dd74373f4a62 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-python3
- Found cached script output: /home/val/CM/repos/local/cache/4331568fb9804fc5
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,python,python3,get-python,get-python3": 0.08 sec.
- Processing env after dependencies ...
Defaulting to user installation because normal site-packages is not writeable
Collecting pandas
Downloading pandas-1.5.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (12.1 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 12.1/12.1 MB 11.1 MB/s eta 0:00:00
Collecting pytz>=2020.1
Downloading pytz-2022.6-py2.py3-none-any.whl (498 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 498.1/498.1 kB 8.1 MB/s eta 0:00:00
Requirement already satisfied: numpy>=1.21.0 in /home/val/.local/lib/python3.10/site-packages (from pandas) (1.23.4)
Requirement already satisfied: python-dateutil>=2.8.1 in /home/val/.local/lib/python3.10/site-packages (from pandas) (2.8.2)
Requirement already satisfied: six>=1.5 in /usr/lib/python3.10/site-packages (from python-dateutil>=2.8.1->pandas) (1.16.0)
Installing collected packages: pytz, pandas
Successfully installed pandas-1.5.1 pytz-2022.6
- Removing tmp tag in the script cached output 5458ebbf02c64655 ...
- running time of script "install,generic,generic-python-lib": 5.59 sec.
Detected version: 1.5.1
- Removing tmp tag in the script cached output 8c96b4eed2564412 ...
- running time of script "get,generic,generic-python-lib": 6.27 sec.
- Processing env after dependencies ...
Requirement already satisfied: requests in /home/val/Coding/mlcommon/venv-cm/lib/python3.10/site-packages (from -r /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-dataset-openimages/requirements.txt (line 1)) (2.28.1)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/val/Coding/mlcommon/venv-cm/lib/python3.10/site-packages (from requests->-r /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-dataset-openimages/requirements.txt (line 1)) (1.26.12)
Requirement already satisfied: charset-normalizer<3,>=2 in /home/val/Coding/mlcommon/venv-cm/lib/python3.10/site-packages (from requests->-r /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-dataset-openimages/requirements.txt (line 1)) (2.1.1)
Requirement already satisfied: certifi>=2017.4.17 in /home/val/Coding/mlcommon/venv-cm/lib/python3.10/site-packages (from requests->-r /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-dataset-openimages/requirements.txt (line 1)) (2022.9.24)
Requirement already satisfied: idna<4,>=2.5 in /home/val/Coding/mlcommon/venv-cm/lib/python3.10/site-packages (from requests->-r /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-dataset-openimages/requirements.txt (line 1)) (3.4)
[notice] A new release of pip available: 22.2.2 -> 22.3.1
[notice] To update, run: pip install --upgrade pip
./openimages_mlperf.sh -d /home/val/CM/repos/local/cache/1afee6231c5e4add/install -m 500
Traceback (most recent call last):
File "/home/val/CM/repos/local/cache/77b0a2bc77f444dd/inference/vision/classification_and_detection/tools/openimages.py", line 46, in <module>
import boto3
ModuleNotFoundError: No module named 'boto3'
CM error: Portable CM script failed (return code = 256)
Note that it is often a portability problem of the third-party tool or native script that is wrapped and unified by this CM script.
The CM concept is to collaboratively fix such issues inside portable CM scripts to make existing tools and native script more portable, interoperable, deterministic and reproducible.
Please help the community by reporting the full log with the command line here:
* https://github.com/mlcommons/ck/issues
* https://bit.ly/mlperf-edu-wgI checked and indeed some modules downloaded at the same time are installed on the system's python and not inside the venv. That's at leat the case for boto3, botocore, s3tranfer, jmespath that I can import from another python shell. Strangely, I can't do the same for python_dateutil. Hope this can help.
gfursin
commented
1 year ago We have a minor issue with python venv on some systems like yours. I think I know how to fix it and I will try to release the update within an hour. You will need to clean the whole CM cache and reinstall virtual env. I will get back to you shortly! Thank you for your feedback again!!!
gfursin
commented
1 year ago Hi @ValouBambou . I think we have fixed the issue. Do you think you can update your installation, clean cache and rerun MLPerf via CM as follows, please?:
python3 -m pip install cmind -U
cm pull repo mlcommons@ck --checkout=master
cm rm cache -f
cm run script "get sys-utils-cm" --quiet
cm run script "install python-venv" --version=3.10.8 --name=mlperf
cm run script --tags=run,mlperf,inference,generate-run-cmds,_submission,_short,_dashboard \
--adr.python.name=mlperf \
--adr.python.version_min=3.8 \
--adr.compiler.tags=gcc \
--adr.openimages-preprocessed.tags=_500 \
--submitter="OctoML" \
--hw_name=default \
--model=retinanet \
--backend=onnxruntime \
--device=cpu \
--scenario=Offline \
--test_query_count=10 \
--rerunThank you!!!
ValouBambou
commented
1 year ago This time the benchmark runs but after 15 minutes it crashed with a missing module again. This time its pandas and it has been installed on the system's python but not in the venv.
The log of the last command:
* Searching for collective script(s) with tags "run,mlperf,inference,generate-run-cmds" and variations "submission,short,dashboard"
- Number of scripts found: 1
- Found script::run-mlperf-inference-app,4a5d5b13fd7e4ac8 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/run-mlperf-inference-app
Prepared variations: _submission,_short,_dashboard,_all-modes
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,os"
- Number of scripts found: 1
- Found script::detect-os,863735b7db8c44fc in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-os
- Running preprocess ...
- running time of script "detect-os,detect,os,info": 0.10 sec.
* Searching for collective script(s) with tags "detect,cpu"
- Number of scripts found: 1
- Found script::detect-cpu,586c8a43320142f7 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-cpu
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,os"
- Number of scripts found: 1
- Found script::detect-os,863735b7db8c44fc in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-os
- Running preprocess ...
- running time of script "detect-os,detect,os,info": 0.09 sec.
- Processing env after dependencies ...
- Running preprocess ...
- running time of script "detect,cpu,detect-cpu,info": 0.19 sec.
* Searching for collective script(s) with tags "get,python"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,python
- Number of cached script outputs found: 2
- Found script::get-python3,d0b5dd74373f4a62 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-python3
- Requested version: >= 3.8
- Found cached script output: /home/val/CM/repos/local/cache/01ee27882c904360
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,python,python3,get-python,get-python3": 0.09 sec.
* Searching for collective script(s) with tags "get,mlcommons,inference,src"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,mlcommons,inference,src
- Number of cached script outputs found: 0
- Found script::get-mlperf-inference-src,4b57186581024797 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-mlperf-inference-src
Prepared variations: _default,_short-history
- Creating new "cache" script artifact in the CM local repository ...
- Tags: tmp,get,mlcommons,inference,src,source,inference-src,inference-source,mlperf,_default,_short-history,script-artifact-4b57186581024797
- Changing to /home/val/CM/repos/local/cache/200bfc700b804564
- Version is not specified - use either default_version from meta or min/max/usable: master
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,os"
- Number of scripts found: 1
- Found script::detect-os,863735b7db8c44fc in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-os
- Running preprocess ...
- running time of script "detect-os,detect,os,info": 0.10 sec.
* Searching for collective script(s) with tags "get,python3"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,python3
- Number of cached script outputs found: 2
- Found script::get-python3,d0b5dd74373f4a62 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-python3
- Requested version: >= 3.8
- Found cached script output: /home/val/CM/repos/local/cache/01ee27882c904360
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,python,python3,get-python,get-python3": 0.07 sec.
- Processing env after dependencies ...
- Running preprocess ...
# potential PIP version string (if needed): ==master
******************************************************
Cloning Mlcommons from https://github.com/mlcommons/inference.git with branch master --depth 10 --recurse-submodules...
Cloning into 'inference'...
remote: Enumerating objects: 519, done.
remote: Counting objects: 100% (519/519), done.
remote: Compressing objects: 100% (423/423), done.
remote: Total 519 (delta 103), reused 297 (delta 60), pack-reused 0
Receiving objects: 100% (519/519), 1.64 MiB | 3.74 MiB/s, done.
Resolving deltas: 100% (103/103), done.
Submodule 'language/bert/DeepLearningExamples' (https://github.com/NVIDIA/DeepLearningExamples.git) registered for path 'language/bert/DeepLearningExamples'
Submodule 'third_party/pybind' (https://github.com/pybind/pybind11.git) registered for path 'third_party/pybind'
Submodule 'tools/submission/power-dev' (https://github.com/mlcommons/power-dev) registered for path 'tools/submission/power-dev'
Submodule 'vision/medical_imaging/3d-unet-brats19/nnUnet' (https://github.com/MIC-DKFZ/nnUNet.git) registered for path 'vision/medical_imaging/3d-unet-brats19/nnUnet'
Cloning into '/home/val/CM/repos/local/cache/200bfc700b804564/inference/language/bert/DeepLearningExamples'...
remote: Enumerating objects: 31659, done.
remote: Counting objects: 100% (588/588), done.
remote: Compressing objects: 100% (347/347), done.
remote: Total 31659 (delta 246), reused 518 (delta 228), pack-reused 31071
Receiving objects: 100% (31659/31659), 98.13 MiB | 10.55 MiB/s, done.
Resolving deltas: 100% (22474/22474), done.
Cloning into '/home/val/CM/repos/local/cache/200bfc700b804564/inference/third_party/pybind'...
remote: Enumerating objects: 17856, done.
remote: Counting objects: 100% (55/55), done.
remote: Compressing objects: 100% (53/53), done.
remote: Total 17856 (delta 8), reused 38 (delta 1), pack-reused 17801
Receiving objects: 100% (17856/17856), 8.01 MiB | 2.03 MiB/s, done.
Resolving deltas: 100% (12318/12318), done.
Cloning into '/home/val/CM/repos/local/cache/200bfc700b804564/inference/tools/submission/power-dev'...
remote: Enumerating objects: 1266, done.
remote: Counting objects: 100% (160/160), done.
remote: Compressing objects: 100% (70/70), done.
remote: Total 1266 (delta 106), reused 116 (delta 89), pack-reused 1106
Receiving objects: 100% (1266/1266), 582.53 KiB | 5.88 MiB/s, done.
Resolving deltas: 100% (801/801), done.
Cloning into '/home/val/CM/repos/local/cache/200bfc700b804564/inference/vision/medical_imaging/3d-unet-brats19/nnUnet'...
remote: Enumerating objects: 6002, done.
remote: Counting objects: 100% (9/9), done.
remote: Compressing objects: 100% (9/9), done.
remote: Total 6002 (delta 0), reused 9 (delta 0), pack-reused 5993
Receiving objects: 100% (6002/6002), 1.50 MiB | 7.74 MiB/s, done.
Resolving deltas: 100% (4723/4723), done.
Submodule path 'language/bert/DeepLearningExamples': checked out 'b03375bd6c2c5233130e61a3be49e26d1a20ac7c'
Submodule 'PyTorch/SpeechRecognition/Jasper/external/tensorrt-inference-server' (https://github.com/NVIDIA/tensorrt-inference-server.git) registered for path 'language/bert/DeepLearningExamples/PyTorch/SpeechRecognition/Jasper/external/tensorrt-inference-server'
Submodule 'PyTorch/Translation/Transformer/cutlass' (https://github.com/NVIDIA/cutlass.git) registered for path 'language/bert/DeepLearningExamples/PyTorch/Translation/Transformer/cutlass'
Cloning into '/home/val/CM/repos/local/cache/200bfc700b804564/inference/language/bert/DeepLearningExamples/PyTorch/SpeechRecognition/Jasper/external/tensorrt-inference-server'...
remote: Enumerating objects: 34200, done.
remote: Counting objects: 100% (29/29), done.
remote: Compressing objects: 100% (23/23), done.
remote: Total 34200 (delta 8), reused 16 (delta 5), pack-reused 34171
Receiving objects: 100% (34200/34200), 24.36 MiB | 9.58 MiB/s, done.
Resolving deltas: 100% (24607/24607), done.
Cloning into '/home/val/CM/repos/local/cache/200bfc700b804564/inference/language/bert/DeepLearningExamples/PyTorch/Translation/Transformer/cutlass'...
remote: Enumerating objects: 16745, done.
remote: Counting objects: 100% (811/811), done.
remote: Compressing objects: 100% (433/433), done.
remote: Total 16745 (delta 401), reused 697 (delta 338), pack-reused 15934
Receiving objects: 100% (16745/16745), 23.58 MiB | 10.30 MiB/s, done.
Resolving deltas: 100% (12093/12093), done.
Submodule path 'language/bert/DeepLearningExamples/PyTorch/SpeechRecognition/Jasper/external/tensorrt-inference-server': checked out '71f0771cb8cb2a2eb1c6a9433f9a56dd1f206c96'
Submodule path 'language/bert/DeepLearningExamples/PyTorch/Translation/Transformer/cutlass': checked out 'ed2ed4d667ce95e1371bd62db32b6a114e774336'
Submodule 'tools/external/googletest' (https://github.com/google/googletest.git) registered for path 'language/bert/DeepLearningExamples/PyTorch/Translation/Transformer/cutlass/tools/external/googletest'
Cloning into '/home/val/CM/repos/local/cache/200bfc700b804564/inference/language/bert/DeepLearningExamples/PyTorch/Translation/Transformer/cutlass/tools/external/googletest'...
remote: Enumerating objects: 25054, done.
remote: Counting objects: 100% (40/40), done.
remote: Compressing objects: 100% (26/26), done.
remote: Total 25054 (delta 20), reused 27 (delta 12), pack-reused 25014
Receiving objects: 100% (25054/25054), 10.89 MiB | 10.59 MiB/s, done.
Resolving deltas: 100% (18551/18551), done.
Submodule path 'language/bert/DeepLearningExamples/PyTorch/Translation/Transformer/cutlass/tools/external/googletest': checked out '9077ec7efe5b652468ab051e93c67589d5cb8f85'
Submodule path 'third_party/pybind': checked out '25abf7efba0b2990f5a6dfb0a31bc65c0f2f4d17'
Submodule 'tools/clang' (https://github.com/wjakob/clang-cindex-python3) registered for path 'third_party/pybind/tools/clang'
Cloning into '/home/val/CM/repos/local/cache/200bfc700b804564/inference/third_party/pybind/tools/clang'...
remote: Enumerating objects: 355, done.
remote: Counting objects: 100% (14/14), done.
remote: Compressing objects: 100% (10/10), done.
remote: Total 355 (delta 6), reused 8 (delta 4), pack-reused 341
Receiving objects: 100% (355/355), 145.66 KiB | 3.03 MiB/s, done.
Resolving deltas: 100% (147/147), done.
remote: Enumerating objects: 9, done.
remote: Counting objects: 100% (3/3), done.
remote: Total 9 (delta 3), reused 3 (delta 3), pack-reused 6
Unpacking objects: 100% (9/9), 11.12 KiB | 367.00 KiB/s, done.
From https://github.com/wjakob/clang-cindex-python3
* branch 6a00cbc4a9b8e68b71caf7f774b3f9c753ae84d5 -> FETCH_HEAD
Submodule path 'third_party/pybind/tools/clang': checked out '6a00cbc4a9b8e68b71caf7f774b3f9c753ae84d5'
remote: Enumerating objects: 7, done.
remote: Counting objects: 100% (7/7), done.
remote: Compressing objects: 100% (1/1), done.
remote: Total 4 (delta 3), reused 3 (delta 3), pack-reused 0
Unpacking objects: 100% (4/4), 397 bytes | 198.00 KiB/s, done.
From https://github.com/mlcommons/power-dev
* branch e2baedb555f5360811dd9d2d14ed41766db76dfb -> FETCH_HEAD
Submodule path 'tools/submission/power-dev': checked out 'e2baedb555f5360811dd9d2d14ed41766db76dfb'
Submodule path 'vision/medical_imaging/3d-unet-brats19/nnUnet': checked out 'b38c69b345b2f60cd0d053039669e8f988b0c0af'
- Removing tmp tag in the script cached output 200bfc700b804564 ...
- running time of script "get,src,source,inference,inference-src,inference-source,mlperf,mlcommons": 31.82 sec.
- Processing env after dependencies ...
- Running preprocess ...
Using MLCommons Inference source from /home/val/CM/repos/local/cache/200bfc700b804564/inference
* Searching for collective script(s) with tags "app,mlperf,inference,generic" and variations "python,retinanet,onnxruntime,cpu,test"
- Number of scripts found: 1
- Found script::app-mlperf-inference,d775cac873ee4231 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference
Prepared variations: _python,_retinanet,_onnxruntime,_cpu,_test
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,os"
- Number of scripts found: 1
- Found script::detect-os,863735b7db8c44fc in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-os
- Running preprocess ...
- running time of script "detect-os,detect,os,info": 0.11 sec.
* Searching for collective script(s) with tags "detect,cpu"
- Number of scripts found: 1
- Found script::detect-cpu,586c8a43320142f7 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-cpu
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,os"
- Number of scripts found: 1
- Found script::detect-os,863735b7db8c44fc in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-os
- Running preprocess ...
- running time of script "detect-os,detect,os,info": 0.10 sec.
- Processing env after dependencies ...
- Running preprocess ...
- running time of script "detect,cpu,detect-cpu,info": 0.20 sec.
* Searching for collective script(s) with tags "get,sys-utils-cm"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,sys-utils-cm
- Number of cached script outputs found: 0
- Found script::get-sys-utils-cm,bc90993277e84b8e in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-sys-utils-cm
- Creating new "cache" script artifact in the CM local repository ...
- Tags: tmp,get,sys-utils-cm,script-artifact-bc90993277e84b8e
- Changing to /home/val/CM/repos/local/cache/e97b5ea0489546fd
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,os"
- Number of scripts found: 1
- Found script::detect-os,863735b7db8c44fc in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-os
- Running preprocess ...
- running time of script "detect-os,detect,os,info": 0.11 sec.
- Processing env after dependencies ...
- Running preprocess ...
***********************************************************************
This script will attempt to install minimal system dependencies for CM.
Note that you may be asked for your SUDO password ...
***********************************************************************
- Removing tmp tag in the script cached output e97b5ea0489546fd ...
- running time of script "get,sys-utils-cm": 0.24 sec.
* Searching for collective script(s) with tags "get,python"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,python
- Number of cached script outputs found: 2
- Found script::get-python3,d0b5dd74373f4a62 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-python3
- Requested version: >= 3.8
- Found cached script output: /home/val/CM/repos/local/cache/01ee27882c904360
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,python,python3,get-python,get-python3": 0.08 sec.
* Searching for collective script(s) with tags "get,generic-python-lib" and variations "onnxruntime"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,generic-python-lib
- Number of cached script outputs found: 0
- Found script::get-generic-python-lib,94b62a682bc44791 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-generic-python-lib
Prepared variations: _onnxruntime
- Creating new "cache" script artifact in the CM local repository ...
- Tags: tmp,get,generic-python-lib,generic,_onnxruntime,deps-python-version-3.10.8,deps-python-virtual,deps-python-name-mlperf,script-artifact-94b62a682bc44791
- Changing to /home/val/CM/repos/local/cache/3698977e5b3e4807
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,cpu"
- Number of scripts found: 1
- Found script::detect-cpu,586c8a43320142f7 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-cpu
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,os"
- Number of scripts found: 1
- Found script::detect-os,863735b7db8c44fc in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-os
- Running preprocess ...
- running time of script "detect-os,detect,os,info": 0.10 sec.
- Processing env after dependencies ...
- Running preprocess ...
- running time of script "detect,cpu,detect-cpu,info": 0.19 sec.
* Searching for collective script(s) with tags "get,python3"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,python3
- Number of cached script outputs found: 2
- Found script::get-python3,d0b5dd74373f4a62 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-python3
- Requested version: >= 3.8
- Found cached script output: /home/val/CM/repos/local/cache/01ee27882c904360
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,python,python3,get-python,get-python3": 0.08 sec.
- Processing env after dependencies ...
- Running preprocess ...
- Checking prehook dependencies on other CM scripts:
* Searching for collective script(s) with tags "install,generic-python-lib" and variations "onnxruntime"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,install,generic-python-lib
- Number of cached script outputs found: 0
- Found script::install-generic-python-lib,f4f502b7b5d545c8 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/install-generic-python-lib
Prepared variations: _onnxruntime
- Creating new "cache" script artifact in the CM local repository ...
- Tags: tmp,install,generic-python-lib,generic,_onnxruntime,deps-python-version-3.10.8,deps-python-virtual,deps-python-name-mlperf,script-artifact-f4f502b7b5d545c8
- Changing to /home/val/CM/repos/local/cache/def8d54e29fe42ff
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,cpu"
- Number of scripts found: 1
- Found script::detect-cpu,586c8a43320142f7 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-cpu
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,os"
- Number of scripts found: 1
- Found script::detect-os,863735b7db8c44fc in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-os
- Running preprocess ...
- running time of script "detect-os,detect,os,info": 0.10 sec.
- Processing env after dependencies ...
- Running preprocess ...
- running time of script "detect,cpu,detect-cpu,info": 0.19 sec.
* Searching for collective script(s) with tags "get,python"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,python
- Number of cached script outputs found: 2
- Found script::get-python3,d0b5dd74373f4a62 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-python3
- Requested version: >= 3.8
- Found cached script output: /home/val/CM/repos/local/cache/01ee27882c904360
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,python,python3,get-python,get-python3": 0.08 sec.
- Processing env after dependencies ...
Collecting onnxruntime
Downloading onnxruntime-1.13.1-cp310-cp310-manylinux_2_27_x86_64.whl (4.5 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 4.5/4.5 MB 10.9 MB/s eta 0:00:00
Collecting protobuf
Downloading protobuf-4.21.9-cp37-abi3-manylinux2014_x86_64.whl (408 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 408.4/408.4 kB 12.2 MB/s eta 0:00:00
Collecting sympy
Downloading sympy-1.11.1-py3-none-any.whl (6.5 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 6.5/6.5 MB 11.4 MB/s eta 0:00:00
Collecting coloredlogs
Downloading coloredlogs-15.0.1-py2.py3-none-any.whl (46 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 46.0/46.0 kB 11.0 MB/s eta 0:00:00
Collecting flatbuffers
Using cached flatbuffers-22.10.26-py2.py3-none-any.whl (26 kB)
Collecting packaging
Using cached packaging-21.3-py3-none-any.whl (40 kB)
Collecting numpy>=1.21.6
Using cached numpy-1.23.4-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (17.1 MB)
Collecting humanfriendly>=9.1
Downloading humanfriendly-10.0-py2.py3-none-any.whl (86 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 86.8/86.8 kB 7.3 MB/s eta 0:00:00
Collecting pyparsing!=3.0.5,>=2.0.2
Using cached pyparsing-3.0.9-py3-none-any.whl (98 kB)
Collecting mpmath>=0.19
Downloading mpmath-1.2.1-py3-none-any.whl (532 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 532.6/532.6 kB 11.3 MB/s eta 0:00:00
Installing collected packages: mpmath, flatbuffers, sympy, pyparsing, protobuf, numpy, humanfriendly, packaging, coloredlogs, onnxruntime
Successfully installed coloredlogs-15.0.1 flatbuffers-22.10.26 humanfriendly-10.0 mpmath-1.2.1 numpy-1.23.4 onnxruntime-1.13.1 packaging-21.3 protobuf-4.21.9 pyparsing-3.0.9 sympy-1.11.1
[notice] A new release of pip available: 22.2.2 -> 22.3.1
[notice] To update, run: pip install --upgrade pip
- Removing tmp tag in the script cached output def8d54e29fe42ff ...
- running time of script "install,generic,generic-python-lib": 11.42 sec.
Detected version: 1.13.1
- Removing tmp tag in the script cached output 3698977e5b3e4807 ...
- running time of script "get,generic,generic-python-lib": 12.06 sec.
* Searching for collective script(s) with tags "get,loadgen"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,loadgen
- Number of cached script outputs found: 0
- Found script::get-mlperf-inference-loadgen,64c3d98d0ba04950 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-mlperf-inference-loadgen
- Creating new "cache" script artifact in the CM local repository ...
- Tags: tmp,get,loadgen,inference,inference-loadgen,mlperf,mlcommons,deps-python-version-3.10.8,deps-python-virtual,deps-python-name-mlperf,script-artifact-64c3d98d0ba04950
- Changing to /home/val/CM/repos/local/cache/760e1dcded444891
- Version is not specified - use either default_version from meta or min/max/usable: master
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,os"
- Number of scripts found: 1
- Found script::detect-os,863735b7db8c44fc in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-os
- Running preprocess ...
- running time of script "detect-os,detect,os,info": 0.10 sec.
* Searching for collective script(s) with tags "get,python3"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,python3
- Number of cached script outputs found: 2
- Found script::get-python3,d0b5dd74373f4a62 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-python3
- Requested version: >= 3.8
- Found cached script output: /home/val/CM/repos/local/cache/01ee27882c904360
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,python,python3,get-python,get-python3": 0.07 sec.
* Searching for collective script(s) with tags "get,mlcommons,inference,src"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,mlcommons,inference,src
- Number of cached script outputs found: 1
- Found script::get-mlperf-inference-src,4b57186581024797 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-mlperf-inference-src
Prepared variations: _default,_short-history
- Requested version: == master
- Found cached script output: /home/val/CM/repos/local/cache/200bfc700b804564
- Checking dynamic dependencies on other CM scripts:
- Processing env after dependencies ...
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,src,source,inference,inference-src,inference-source,mlperf,mlcommons": 0.06 sec.
* Searching for collective script(s) with tags "get,compiler,gcc"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,compiler,gcc
- Number of cached script outputs found: 0
- Found script::get-gcc,dbf4ab5cbed74372 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-gcc
- Creating new "cache" script artifact in the CM local repository ...
- Tags: tmp,get,compiler,gcc,c-compiler,cpp-compiler,get-gcc,script-artifact-dbf4ab5cbed74372
- Changing to /home/val/CM/repos/local/cache/aa24bd324cbe4ef9
- Running preprocess ...
* /usr/bin/gcc
/usr/bin/gcc --version
gcc (GCC) 12.2.0
Copyright (C) 2022 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Detected version: 12.2.0
# Found artifact in /usr/bin/gcc
/usr/bin/gcc --version
gcc (GCC) 12.2.0
Copyright (C) 2022 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Detected version: 12.2.0
* Searching for collective script(s) with tags "get,compiler-flags"
- Number of scripts found: 1
- Found script::get-compiler-flags,31be8b74a69742f8 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-compiler-flags
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,cpu"
- Number of scripts found: 1
- Found script::detect-cpu,586c8a43320142f7 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-cpu
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,os"
- Number of scripts found: 1
- Found script::detect-os,863735b7db8c44fc in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-os
- Running preprocess ...
- running time of script "detect-os,detect,os,info": 0.10 sec.
- Processing env after dependencies ...
- Running preprocess ...
- running time of script "detect,cpu,detect-cpu,info": 0.19 sec.
- Processing env after dependencies ...
- Running preprocess ...
- running time of script "get,compiler-flags": 0.27 sec.
- Removing tmp tag in the script cached output aa24bd324cbe4ef9 ...
- running time of script "get,gcc,compiler,c-compiler,cpp-compiler,get-gcc": 0.41 sec.
* Searching for collective script(s) with tags "get,cmake"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,cmake
- Number of cached script outputs found: 0
- Found script::get-cmake,52bf974d791b4fc8 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-cmake
- Creating new "cache" script artifact in the CM local repository ...
- Tags: tmp,get,cmake,get-cmake,script-artifact-52bf974d791b4fc8
- Changing to /home/val/CM/repos/local/cache/ab50a4a4895047a5
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,cpu"
- Number of scripts found: 1
- Found script::detect-cpu,586c8a43320142f7 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-cpu
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,os"
- Number of scripts found: 1
- Found script::detect-os,863735b7db8c44fc in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-os
- Running preprocess ...
- running time of script "detect-os,detect,os,info": 0.10 sec.
- Processing env after dependencies ...
- Running preprocess ...
- running time of script "detect,cpu,detect-cpu,info": 0.19 sec.
- Processing env after dependencies ...
- Running preprocess ...
- Checking prehook dependencies on other CM scripts:
* Searching for collective script(s) with tags "install,cmake,prebuilt"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,install,cmake,prebuilt
- Number of cached script outputs found: 0
- Found script::install-cmake-prebuilt,5a39ef05992b4103 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/install-cmake-prebuilt
- Creating new "cache" script artifact in the CM local repository ...
- Tags: tmp,install,cmake,prebuilt,prebuilt-cmake,install-prebuilt-cmake,script-artifact-5a39ef05992b4103
- Changing to /home/val/CM/repos/local/cache/c40185b00eb044b4
- Version is not specified - use either default_version from meta or min/max/usable: 3.21.1
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,os"
- Number of scripts found: 1
- Found script::detect-os,863735b7db8c44fc in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-os
- Running preprocess ...
- running time of script "detect-os,detect,os,info": 0.09 sec.
- Processing env after dependencies ...
- Running preprocess ...
# Requested version: 3.21.1
# Prepared package URL: https://github.com/Kitware/CMake/releases/download/v3.21.1/cmake-3.21.1-linux-x86_64.tar.gz
Downloading from https://github.com/Kitware/CMake/releases/download/v3.21.1/cmake-3.21.1-linux-x86_64.tar.gz ...
Downloaded: 100%
# potential PIP version string (if needed): ==3.21.1
Unarchiving cmake-3.21.1-linux-x86_64.tar.gz ...
- Removing tmp tag in the script cached output c40185b00eb044b4 ...
- running time of script "install,prebuilt,cmake,prebuilt-cmake,install-prebuilt-cmake": 5.43 sec.
Detected version: 3.21.1
- Removing tmp tag in the script cached output ab50a4a4895047a5 ...
- running time of script "get,cmake,get-cmake": 5.74 sec.
- Processing env after dependencies ...
- Running preprocess ...
# potential PIP version string (if needed): ==master
******************************************************
-- The C compiler identification is GNU 12.2.0
-- The CXX compiler identification is GNU 12.2.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
mlperf_loadgen v1.1
-- Using C++ compiler flags: -O2 -O3 -W -Wall
-- Using C++ standard: 14
-- Using static linker flags:
-- Using shared linker flags: -O2
-- Using output path: /home/val/CM/repos/local/cache/760e1dcded444891/build
-- Found PythonInterp: /home/val/CM/repos/local/cache/a4149a33cd464596/venv/bin/python (found version "3.10.8")
-- Using Python interpreter: /home/val/CM/repos/local/cache/a4149a33cd464596/venv/bin/python
-- Configuring done
-- Generating done
-- Build files have been written to: /home/val/CM/repos/local/cache/760e1dcded444891/build
******************************************************
[ 15%] Building CXX object CMakeFiles/mlperf_loadgen.dir/early_stopping.cc.o
[ 15%] Building CXX object CMakeFiles/mlperf_loadgen.dir/bindings/c_api.cc.o
[ 30%] Building CXX object CMakeFiles/mlperf_loadgen.dir/loadgen.cc.o
[ 30%] Building CXX object CMakeFiles/mlperf_loadgen.dir/issue_query_controller.cc.o
[ 38%] Building CXX object CMakeFiles/mlperf_loadgen.dir/logging.cc.o
[ 46%] Building CXX object CMakeFiles/mlperf_loadgen.dir/test_settings_internal.cc.o
[ 53%] Building CXX object CMakeFiles/mlperf_loadgen.dir/utils.cc.o
[ 61%] Building CXX object CMakeFiles/mlperf_loadgen.dir/results.cc.o
[ 69%] Building CXX object CMakeFiles/mlperf_loadgen.dir/version.cc.o
[ 76%] Building CXX object CMakeFiles/mlperf_loadgen.dir/version_generated.cc.o
[ 84%] Linking CXX static library libmlperf_loadgen.a
[ 84%] Built target mlperf_loadgen
[ 92%] Building CXX object CMakeFiles/benchmark.dir/benchmark/repro.cpp.o
/home/val/CM/repos/local/cache/200bfc700b804564/inference/loadgen/benchmark/repro.cpp: In member function ‘virtual void QSL::LoadSamplesToRam(const std::vector<long unsigned int>&)’:
/home/val/CM/repos/local/cache/200bfc700b804564/inference/loadgen/benchmark/repro.cpp:37:52: warning: unused parameter ‘samples’ [-Wunused-parameter]
37 | const std::vector<mlperf::QuerySampleIndex>& samples) override {}
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~
/home/val/CM/repos/local/cache/200bfc700b804564/inference/loadgen/benchmark/repro.cpp: In member function ‘virtual void QSL::UnloadSamplesFromRam(const std::vector<long unsigned int>&)’:
/home/val/CM/repos/local/cache/200bfc700b804564/inference/loadgen/benchmark/repro.cpp:39:52: warning: unused parameter ‘samples’ [-Wunused-parameter]
39 | const std::vector<mlperf::QuerySampleIndex>& samples) override {}
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~
/home/val/CM/repos/local/cache/200bfc700b804564/inference/loadgen/benchmark/repro.cpp: In member function ‘virtual void BasicSUT::IssueQuery(const std::vector<mlperf::QuerySample>&)’:
/home/val/CM/repos/local/cache/200bfc700b804564/inference/loadgen/benchmark/repro.cpp:55:11: warning: comparison of integer expressions of different signedness: ‘int’ and ‘std::vector<mlperf::QuerySampleResponse>::size_type’ {aka ‘long unsigned int’} [-Wsign-compare]
55 | if (n > mResponses.size()) {
| ~~^~~~~~~~~~~~~~~~~~~
/home/val/CM/repos/local/cache/200bfc700b804564/inference/loadgen/benchmark/repro.cpp: In member function ‘void QueueSUT::CompleteThread(int)’:
/home/val/CM/repos/local/cache/200bfc700b804564/inference/loadgen/benchmark/repro.cpp:125:27: warning: comparison of integer expressions of different signedness: ‘int’ and ‘size_t’ {aka ‘long unsigned int’} [-Wsign-compare]
125 | for (int i = 0; i < actualSize; i++) {
| ~~^~~~~~~~~~~~
/home/val/CM/repos/local/cache/200bfc700b804564/inference/loadgen/benchmark/repro.cpp: In member function ‘virtual void MultiBasicSUT::IssueQuery(const std::vector<mlperf::QuerySample>&)’:
/home/val/CM/repos/local/cache/200bfc700b804564/inference/loadgen/benchmark/repro.cpp:171:11: warning: comparison of integer expressions of different signedness: ‘int’ and ‘std::vector<mlperf::QuerySampleResponse>::size_type’ {aka ‘long unsigned int’} [-Wsign-compare]
171 | if (n > reponses.size()) {
| ~~^~~~~~~~~~~~~~~~~
[100%] Linking CXX executable benchmark
[100%] Built target benchmark
Install the project...
-- Install configuration: ""
-- Installing: /home/val/CM/repos/local/cache/760e1dcded444891/install/lib/libmlperf_loadgen.a
-- Installing: /home/val/CM/repos/local/cache/760e1dcded444891/install/include
-- Installing: /home/val/CM/repos/local/cache/760e1dcded444891/install/include/benchmark
-- Installing: /home/val/CM/repos/local/cache/760e1dcded444891/install/include/results.h
-- Installing: /home/val/CM/repos/local/cache/760e1dcded444891/install/include/tests
-- Installing: /home/val/CM/repos/local/cache/760e1dcded444891/install/include/tests/loadgen_test.h
-- Installing: /home/val/CM/repos/local/cache/760e1dcded444891/install/include/query_sample_library.h
-- Installing: /home/val/CM/repos/local/cache/760e1dcded444891/install/include/logging.h
-- Installing: /home/val/CM/repos/local/cache/760e1dcded444891/install/include/early_stopping.h
-- Installing: /home/val/CM/repos/local/cache/760e1dcded444891/install/include/test_settings_internal.h
-- Installing: /home/val/CM/repos/local/cache/760e1dcded444891/install/include/query_sample.h
-- Installing: /home/val/CM/repos/local/cache/760e1dcded444891/install/include/system_under_test.h
-- Installing: /home/val/CM/repos/local/cache/760e1dcded444891/install/include/demos
-- Installing: /home/val/CM/repos/local/cache/760e1dcded444891/install/include/demos/lon
-- Installing: /home/val/CM/repos/local/cache/760e1dcded444891/install/include/test_settings.h
-- Installing: /home/val/CM/repos/local/cache/760e1dcded444891/install/include/tools
-- Installing: /home/val/CM/repos/local/cache/760e1dcded444891/install/include/bindings
-- Installing: /home/val/CM/repos/local/cache/760e1dcded444891/install/include/bindings/c_api.h
-- Installing: /home/val/CM/repos/local/cache/760e1dcded444891/install/include/version.h
-- Installing: /home/val/CM/repos/local/cache/760e1dcded444891/install/include/docs
-- Installing: /home/val/CM/repos/local/cache/760e1dcded444891/install/include/docs/src
-- Installing: /home/val/CM/repos/local/cache/760e1dcded444891/install/include/utils.h
-- Installing: /home/val/CM/repos/local/cache/760e1dcded444891/install/include/loadgen.h
-- Installing: /home/val/CM/repos/local/cache/760e1dcded444891/install/include/issue_query_controller.h
Collecting wheel
Using cached wheel-0.38.4-py3-none-any.whl (36 kB)
Installing collected packages: wheel
Successfully installed wheel-0.38.4
[notice] A new release of pip available: 22.2.2 -> 22.3.1
[notice] To update, run: pip install --upgrade pip
running bdist_wheel
running build
running build_ext
building 'mlperf_loadgen' extension
creating build
creating build/temp.linux-x86_64-cpython-310
creating build/temp.linux-x86_64-cpython-310/bindings
creating build/temp.linux-x86_64-cpython-310/generated
gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -ffat-lto-objects -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -std=c++14 -O3 -fPIC -DMAJOR_VERSION=2 -DMINOR_VERSION=1 -I. -I../third_party/pybind/include -I/home/val/CM/repos/local/cache/a4149a33cd464596/venv/include -I/usr/include/python3.10 -c bindings/python_api.cc -o build/temp.linux-x86_64-cpython-310/bindings/python_api.o
In file included from ../third_party/pybind/include/pybind11/cast.h:16,
from ../third_party/pybind/include/pybind11/attr.h:13,
from ../third_party/pybind/include/pybind11/pybind11.h:44,
from ../third_party/pybind/include/pybind11/functional.h:12,
from bindings/python_api.cc:26:
../third_party/pybind/include/pybind11/detail/internals.h: In function ‘pybind11::detail::internals& pybind11::detail::get_internals()’:
../third_party/pybind/include/pybind11/detail/internals.h:200:27: warning: ‘void PyEval_InitThreads()’ is deprecated [-Wdeprecated-declarations]
200 | PyEval_InitThreads();
| ~~~~~~~~~~~~~~~~~~^~
In file included from /usr/include/python3.10/Python.h:130,
from ../third_party/pybind/include/pybind11/detail/common.h:112,
from ../third_party/pybind/include/pybind11/pytypes.h:12,
from ../third_party/pybind/include/pybind11/cast.h:13:
/usr/include/python3.10/ceval.h:122:37: note: declared here
122 | Py_DEPRECATED(3.9) PyAPI_FUNC(void) PyEval_InitThreads(void);
| ^~~~~~~~~~~~~~~~~~
gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -ffat-lto-objects -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -std=c++14 -O3 -fPIC -DMAJOR_VERSION=2 -DMINOR_VERSION=1 -I. -I../third_party/pybind/include -I/home/val/CM/repos/local/cache/a4149a33cd464596/venv/include -I/usr/include/python3.10 -c early_stopping.cc -o build/temp.linux-x86_64-cpython-310/early_stopping.o
gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -ffat-lto-objects -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -std=c++14 -O3 -fPIC -DMAJOR_VERSION=2 -DMINOR_VERSION=1 -I. -I../third_party/pybind/include -I/home/val/CM/repos/local/cache/a4149a33cd464596/venv/include -I/usr/include/python3.10 -c generated/version_generated.cc -o build/temp.linux-x86_64-cpython-310/generated/version_generated.o
gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -ffat-lto-objects -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -std=c++14 -O3 -fPIC -DMAJOR_VERSION=2 -DMINOR_VERSION=1 -I. -I../third_party/pybind/include -I/home/val/CM/repos/local/cache/a4149a33cd464596/venv/include -I/usr/include/python3.10 -c issue_query_controller.cc -o build/temp.linux-x86_64-cpython-310/issue_query_controller.o
gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -ffat-lto-objects -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -std=c++14 -O3 -fPIC -DMAJOR_VERSION=2 -DMINOR_VERSION=1 -I. -I../third_party/pybind/include -I/home/val/CM/repos/local/cache/a4149a33cd464596/venv/include -I/usr/include/python3.10 -c loadgen.cc -o build/temp.linux-x86_64-cpython-310/loadgen.o
gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -ffat-lto-objects -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -std=c++14 -O3 -fPIC -DMAJOR_VERSION=2 -DMINOR_VERSION=1 -I. -I../third_party/pybind/include -I/home/val/CM/repos/local/cache/a4149a33cd464596/venv/include -I/usr/include/python3.10 -c logging.cc -o build/temp.linux-x86_64-cpython-310/logging.o
gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -ffat-lto-objects -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -std=c++14 -O3 -fPIC -DMAJOR_VERSION=2 -DMINOR_VERSION=1 -I. -I../third_party/pybind/include -I/home/val/CM/repos/local/cache/a4149a33cd464596/venv/include -I/usr/include/python3.10 -c results.cc -o build/temp.linux-x86_64-cpython-310/results.o
gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -ffat-lto-objects -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -std=c++14 -O3 -fPIC -DMAJOR_VERSION=2 -DMINOR_VERSION=1 -I. -I../third_party/pybind/include -I/home/val/CM/repos/local/cache/a4149a33cd464596/venv/include -I/usr/include/python3.10 -c test_settings_internal.cc -o build/temp.linux-x86_64-cpython-310/test_settings_internal.o
gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -ffat-lto-objects -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -std=c++14 -O3 -fPIC -DMAJOR_VERSION=2 -DMINOR_VERSION=1 -I. -I../third_party/pybind/include -I/home/val/CM/repos/local/cache/a4149a33cd464596/venv/include -I/usr/include/python3.10 -c utils.cc -o build/temp.linux-x86_64-cpython-310/utils.o
gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -ffat-lto-objects -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -std=c++14 -O3 -fPIC -DMAJOR_VERSION=2 -DMINOR_VERSION=1 -I. -I../third_party/pybind/include -I/home/val/CM/repos/local/cache/a4149a33cd464596/venv/include -I/usr/include/python3.10 -c version.cc -o build/temp.linux-x86_64-cpython-310/version.o
creating build/lib.linux-x86_64-cpython-310
g++ -shared -Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now -flto=auto -Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now -flto=auto -O2 -std=c++14 -O3 build/temp.linux-x86_64-cpython-310/bindings/python_api.o build/temp.linux-x86_64-cpython-310/early_stopping.o build/temp.linux-x86_64-cpython-310/generated/version_generated.o build/temp.linux-x86_64-cpython-310/issue_query_controller.o build/temp.linux-x86_64-cpython-310/loadgen.o build/temp.linux-x86_64-cpython-310/logging.o build/temp.linux-x86_64-cpython-310/results.o build/temp.linux-x86_64-cpython-310/test_settings_internal.o build/temp.linux-x86_64-cpython-310/utils.o build/temp.linux-x86_64-cpython-310/version.o -L/usr/lib -o build/lib.linux-x86_64-cpython-310/mlperf_loadgen.cpython-310-x86_64-linux-gnu.so
/home/val/CM/repos/local/cache/a4149a33cd464596/venv/lib/python3.10/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
warnings.warn(
installing to build/bdist.linux-x86_64/wheel
running install
running install_lib
creating build/bdist.linux-x86_64
creating build/bdist.linux-x86_64/wheel
copying build/lib.linux-x86_64-cpython-310/mlperf_loadgen.cpython-310-x86_64-linux-gnu.so -> build/bdist.linux-x86_64/wheel
running install_egg_info
running egg_info
creating mlperf_loadgen.egg-info
writing mlperf_loadgen.egg-info/PKG-INFO
writing dependency_links to mlperf_loadgen.egg-info/dependency_links.txt
writing top-level names to mlperf_loadgen.egg-info/top_level.txt
writing manifest file 'mlperf_loadgen.egg-info/SOURCES.txt'
reading manifest file 'mlperf_loadgen.egg-info/SOURCES.txt'
writing manifest file 'mlperf_loadgen.egg-info/SOURCES.txt'
Copying mlperf_loadgen.egg-info to build/bdist.linux-x86_64/wheel/mlperf_loadgen-2.1-py3.10.egg-info
running install_scripts
creating build/bdist.linux-x86_64/wheel/mlperf_loadgen-2.1.dist-info/WHEEL
creating 'dist/mlperf_loadgen-2.1-cp310-cp310-linux_x86_64.whl' and adding 'build/bdist.linux-x86_64/wheel' to it
adding 'mlperf_loadgen.cpython-310-x86_64-linux-gnu.so'
adding 'mlperf_loadgen-2.1.dist-info/METADATA'
adding 'mlperf_loadgen-2.1.dist-info/WHEEL'
adding 'mlperf_loadgen-2.1.dist-info/top_level.txt'
adding 'mlperf_loadgen-2.1.dist-info/RECORD'
removing build/bdist.linux-x86_64/wheel
Processing ./dist/mlperf_loadgen-2.1-cp310-cp310-linux_x86_64.whl
Installing collected packages: mlperf-loadgen
Successfully installed mlperf-loadgen-2.1
[notice] A new release of pip available: 22.2.2 -> 22.3.1
[notice] To update, run: pip install --upgrade pip
******************************************************
Loadgen is built and installed to /home/val/CM/repos/local/cache/760e1dcded444891/install ...
- Removing tmp tag in the script cached output 760e1dcded444891 ...
- running time of script "get,loadgen,inference,inference-loadgen,mlperf,mlcommons": 71.80 sec.
* Searching for collective script(s) with tags "get,mlcommons,inference,src"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,mlcommons,inference,src
- Number of cached script outputs found: 1
- Found script::get-mlperf-inference-src,4b57186581024797 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-mlperf-inference-src
Prepared variations: _default,_short-history
- Found cached script output: /home/val/CM/repos/local/cache/200bfc700b804564
- Checking dynamic dependencies on other CM scripts:
- Processing env after dependencies ...
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,src,source,inference,inference-src,inference-source,mlperf,mlcommons": 0.07 sec.
* Searching for collective script(s) with tags "get,sut,configs"
- Number of scripts found: 1
- Found script::get-mlperf-inference-sut-configs,c2fbf72009e2445b in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-mlperf-inference-sut-configs
- Running preprocess ...
- running time of script "get,mlperf,sut,configs,sut-configs": 0.08 sec.
* Searching for collective script(s) with tags "get,dataset,object-detection,open-images,openimages,preprocessed" and variations "validation,NCHW,500"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,dataset,object-detection,open-images,openimages,preprocessed
- Number of cached script outputs found: 0
- Found script::get-preprocessed-dataset-openimages,9842f1be8cba4c7b in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-preprocessed-dataset-openimages
Prepared variations: _validation,_NCHW,_500
- Creating new "cache" script artifact in the CM local repository ...
- Tags: tmp,get,dataset,object-detection,open-images,openimages,preprocessed,_validation,_NCHW,_500,script-artifact-9842f1be8cba4c7b
- Changing to /home/val/CM/repos/local/cache/57ab2b01ae0b4674
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "get,python3"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,python3
- Number of cached script outputs found: 2
- Found script::get-python3,d0b5dd74373f4a62 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-python3
- Requested version: >= 3.8
- Found cached script output: /home/val/CM/repos/local/cache/01ee27882c904360
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,python,python3,get-python,get-python3": 0.08 sec.
* Searching for collective script(s) with tags "get,dataset,object-detection,openimages,original" and variations "validation"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,dataset,object-detection,openimages,original
- Number of cached script outputs found: 0
- Found script::get-dataset-openimages,0a9d49b644cf4142 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-dataset-openimages
Prepared variations: _validation
- Creating new "cache" script artifact in the CM local repository ...
- Tags: tmp,get,dataset,object-detection,openimages,original,open-images,_validation,script-artifact-0a9d49b644cf4142
- Changing to /home/val/CM/repos/local/cache/2e696c40fb444966
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "get,sys-utils-cm"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,sys-utils-cm
- Number of cached script outputs found: 1
- Found script::get-sys-utils-cm,bc90993277e84b8e in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-sys-utils-cm
- Found cached script output: /home/val/CM/repos/local/cache/e97b5ea0489546fd
- Checking dynamic dependencies on other CM scripts:
- Processing env after dependencies ...
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,sys-utils-cm": 0.10 sec.
* Searching for collective script(s) with tags "get,python3"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,python3
- Number of cached script outputs found: 2
- Found script::get-python3,d0b5dd74373f4a62 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-python3
- Requested version: >= 3.8
- Found cached script output: /home/val/CM/repos/local/cache/01ee27882c904360
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,python,python3,get-python,get-python3": 0.07 sec.
* Searching for collective script(s) with tags "mlperf,inference,source"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,mlperf,inference,source
- Number of cached script outputs found: 1
- Found script::get-mlperf-inference-src,4b57186581024797 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-mlperf-inference-src
Prepared variations: _default,_short-history
- Found cached script output: /home/val/CM/repos/local/cache/200bfc700b804564
- Checking dynamic dependencies on other CM scripts:
- Processing env after dependencies ...
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,src,source,inference,inference-src,inference-source,mlperf,mlcommons": 0.06 sec.
* Searching for collective script(s) with tags "get,generic-python-lib" and variations "boto3"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,generic-python-lib
- Number of cached script outputs found: 1
- Found script::get-generic-python-lib,94b62a682bc44791 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-generic-python-lib
Prepared variations: _boto3
- Creating new "cache" script artifact in the CM local repository ...
- Tags: tmp,get,generic-python-lib,generic,_boto3,deps-python-version-3.10.8,deps-python-virtual,deps-python-name-mlperf,script-artifact-94b62a682bc44791
- Changing to /home/val/CM/repos/local/cache/3406fea4cdad4090
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,cpu"
- Number of scripts found: 1
- Found script::detect-cpu,586c8a43320142f7 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-cpu
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,os"
- Number of scripts found: 1
- Found script::detect-os,863735b7db8c44fc in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-os
- Running preprocess ...
- running time of script "detect-os,detect,os,info": 0.09 sec.
- Processing env after dependencies ...
- Running preprocess ...
- running time of script "detect,cpu,detect-cpu,info": 0.18 sec.
* Searching for collective script(s) with tags "get,python3"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,python3
- Number of cached script outputs found: 2
- Found script::get-python3,d0b5dd74373f4a62 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-python3
- Requested version: >= 3.8
- Found cached script output: /home/val/CM/repos/local/cache/01ee27882c904360
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,python,python3,get-python,get-python3": 0.08 sec.
- Processing env after dependencies ...
- Running preprocess ...
- Checking prehook dependencies on other CM scripts:
* Searching for collective script(s) with tags "install,generic-python-lib" and variations "boto3"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,install,generic-python-lib
- Number of cached script outputs found: 1
- Found script::install-generic-python-lib,f4f502b7b5d545c8 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/install-generic-python-lib
Prepared variations: _boto3
- Creating new "cache" script artifact in the CM local repository ...
- Tags: tmp,install,generic-python-lib,generic,_boto3,deps-python-version-3.10.8,deps-python-virtual,deps-python-name-mlperf,script-artifact-f4f502b7b5d545c8
- Changing to /home/val/CM/repos/local/cache/cc3c4ddf5ca4466a
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,cpu"
- Number of scripts found: 1
- Found script::detect-cpu,586c8a43320142f7 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-cpu
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,os"
- Number of scripts found: 1
- Found script::detect-os,863735b7db8c44fc in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-os
- Running preprocess ...
- running time of script "detect-os,detect,os,info": 0.10 sec.
- Processing env after dependencies ...
- Running preprocess ...
- running time of script "detect,cpu,detect-cpu,info": 0.19 sec.
* Searching for collective script(s) with tags "get,python"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,python
- Number of cached script outputs found: 2
- Found script::get-python3,d0b5dd74373f4a62 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-python3
- Requested version: >= 3.8
- Found cached script output: /home/val/CM/repos/local/cache/01ee27882c904360
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,python,python3,get-python,get-python3": 0.08 sec.
- Processing env after dependencies ...
Collecting boto3
Using cached boto3-1.26.9-py3-none-any.whl (132 kB)
Collecting botocore<1.30.0,>=1.29.9
Using cached botocore-1.29.9-py3-none-any.whl (9.9 MB)
Collecting s3transfer<0.7.0,>=0.6.0
Using cached s3transfer-0.6.0-py3-none-any.whl (79 kB)
Collecting jmespath<2.0.0,>=0.7.1
Using cached jmespath-1.0.1-py3-none-any.whl (20 kB)
Collecting urllib3<1.27,>=1.25.4
Using cached urllib3-1.26.12-py2.py3-none-any.whl (140 kB)
Collecting python-dateutil<3.0.0,>=2.1
Using cached python_dateutil-2.8.2-py2.py3-none-any.whl (247 kB)
Collecting six>=1.5
Using cached six-1.16.0-py2.py3-none-any.whl (11 kB)
Installing collected packages: urllib3, six, jmespath, python-dateutil, botocore, s3transfer, boto3
Successfully installed boto3-1.26.9 botocore-1.29.9 jmespath-1.0.1 python-dateutil-2.8.2 s3transfer-0.6.0 six-1.16.0 urllib3-1.26.12
[notice] A new release of pip available: 22.2.2 -> 22.3.1
[notice] To update, run: pip install --upgrade pip
- Removing tmp tag in the script cached output cc3c4ddf5ca4466a ...
- running time of script "install,generic,generic-python-lib": 3.04 sec.
Detected version: 1.26.9
- Removing tmp tag in the script cached output 3406fea4cdad4090 ...
- running time of script "get,generic,generic-python-lib": 3.66 sec.
* Searching for collective script(s) with tags "get,generic-python-lib" and variations "tqdm"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,generic-python-lib
- Number of cached script outputs found: 2
- Found script::get-generic-python-lib,94b62a682bc44791 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-generic-python-lib
Prepared variations: _tqdm
- Creating new "cache" script artifact in the CM local repository ...
- Tags: tmp,get,generic-python-lib,generic,_tqdm,deps-python-version-3.10.8,deps-python-virtual,deps-python-name-mlperf,script-artifact-94b62a682bc44791
- Changing to /home/val/CM/repos/local/cache/f399055efa7b448e
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,cpu"
- Number of scripts found: 1
- Found script::detect-cpu,586c8a43320142f7 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-cpu
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,os"
- Number of scripts found: 1
- Found script::detect-os,863735b7db8c44fc in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-os
- Running preprocess ...
- running time of script "detect-os,detect,os,info": 0.10 sec.
- Processing env after dependencies ...
- Running preprocess ...
- running time of script "detect,cpu,detect-cpu,info": 0.19 sec.
* Searching for collective script(s) with tags "get,python3"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,python3
- Number of cached script outputs found: 2
- Found script::get-python3,d0b5dd74373f4a62 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-python3
- Requested version: >= 3.8
- Found cached script output: /home/val/CM/repos/local/cache/01ee27882c904360
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,python,python3,get-python,get-python3": 0.08 sec.
- Processing env after dependencies ...
- Running preprocess ...
- Checking prehook dependencies on other CM scripts:
* Searching for collective script(s) with tags "install,generic-python-lib" and variations "tqdm"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,install,generic-python-lib
- Number of cached script outputs found: 2
- Found script::install-generic-python-lib,f4f502b7b5d545c8 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/install-generic-python-lib
Prepared variations: _tqdm
- Creating new "cache" script artifact in the CM local repository ...
- Tags: tmp,install,generic-python-lib,generic,_tqdm,deps-python-version-3.10.8,deps-python-virtual,deps-python-name-mlperf,script-artifact-f4f502b7b5d545c8
- Changing to /home/val/CM/repos/local/cache/2a2c80023c6f458d
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,cpu"
- Number of scripts found: 1
- Found script::detect-cpu,586c8a43320142f7 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-cpu
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,os"
- Number of scripts found: 1
- Found script::detect-os,863735b7db8c44fc in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-os
- Running preprocess ...
- running time of script "detect-os,detect,os,info": 0.09 sec.
- Processing env after dependencies ...
- Running preprocess ...
- running time of script "detect,cpu,detect-cpu,info": 0.18 sec.
* Searching for collective script(s) with tags "get,python"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,python
- Number of cached script outputs found: 2
- Found script::get-python3,d0b5dd74373f4a62 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-python3
- Requested version: >= 3.8
- Found cached script output: /home/val/CM/repos/local/cache/01ee27882c904360
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,python,python3,get-python,get-python3": 0.07 sec.
- Processing env after dependencies ...
Collecting tqdm
Using cached tqdm-4.64.1-py2.py3-none-any.whl (78 kB)
Installing collected packages: tqdm
Successfully installed tqdm-4.64.1
[notice] A new release of pip available: 22.2.2 -> 22.3.1
[notice] To update, run: pip install --upgrade pip
- Removing tmp tag in the script cached output 2a2c80023c6f458d ...
- running time of script "install,generic,generic-python-lib": 1.13 sec.
Detected version: 4.64.1
- Removing tmp tag in the script cached output f399055efa7b448e ...
- running time of script "get,generic,generic-python-lib": 1.77 sec.
* Searching for collective script(s) with tags "get,generic-python-lib" and variations "numpy"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,generic-python-lib
- Number of cached script outputs found: 3
- Found script::get-generic-python-lib,94b62a682bc44791 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-generic-python-lib
Prepared variations: _numpy
- Creating new "cache" script artifact in the CM local repository ...
- Tags: tmp,get,generic-python-lib,generic,_numpy,deps-python-version-3.10.8,deps-python-virtual,deps-python-name-mlperf,script-artifact-94b62a682bc44791
- Changing to /home/val/CM/repos/local/cache/a8cc1a6bd3e44e6b
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,cpu"
- Number of scripts found: 1
- Found script::detect-cpu,586c8a43320142f7 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-cpu
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,os"
- Number of scripts found: 1
- Found script::detect-os,863735b7db8c44fc in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-os
- Running preprocess ...
- running time of script "detect-os,detect,os,info": 0.09 sec.
- Processing env after dependencies ...
- Running preprocess ...
- running time of script "detect,cpu,detect-cpu,info": 0.19 sec.
* Searching for collective script(s) with tags "get,python3"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,python3
- Number of cached script outputs found: 2
- Found script::get-python3,d0b5dd74373f4a62 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-python3
- Requested version: >= 3.8
- Found cached script output: /home/val/CM/repos/local/cache/01ee27882c904360
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,python,python3,get-python,get-python3": 0.08 sec.
- Processing env after dependencies ...
- Running preprocess ...
Detected version: 1.23.4
- Checking prehook dependencies on other CM scripts:
Detected version: 1.23.4
- Removing tmp tag in the script cached output a8cc1a6bd3e44e6b ...
- running time of script "get,generic,generic-python-lib": 0.65 sec.
* Searching for collective script(s) with tags "get,generic-python-lib" and variations "opencv-python"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,generic-python-lib
- Number of cached script outputs found: 4
- Found script::get-generic-python-lib,94b62a682bc44791 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-generic-python-lib
Prepared variations: _opencv-python
- Creating new "cache" script artifact in the CM local repository ...
- Tags: tmp,get,generic-python-lib,generic,_opencv-python,deps-python-version-3.10.8,deps-python-virtual,deps-python-name-mlperf,script-artifact-94b62a682bc44791
- Changing to /home/val/CM/repos/local/cache/ec84afc3e3144d97
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,cpu"
- Number of scripts found: 1
- Found script::detect-cpu,586c8a43320142f7 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-cpu
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,os"
- Number of scripts found: 1
- Found script::detect-os,863735b7db8c44fc in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-os
- Running preprocess ...
- running time of script "detect-os,detect,os,info": 0.10 sec.
- Processing env after dependencies ...
- Running preprocess ...
- running time of script "detect,cpu,detect-cpu,info": 0.20 sec.
* Searching for collective script(s) with tags "get,python3"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,python3
- Number of cached script outputs found: 2
- Found script::get-python3,d0b5dd74373f4a62 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-python3
- Requested version: >= 3.8
- Found cached script output: /home/val/CM/repos/local/cache/01ee27882c904360
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,python,python3,get-python,get-python3": 0.08 sec.
- Processing env after dependencies ...
- Running preprocess ...
- Checking prehook dependencies on other CM scripts:
* Searching for collective script(s) with tags "install,generic-python-lib" and variations "opencv-python"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,install,generic-python-lib
- Number of cached script outputs found: 3
- Found script::install-generic-python-lib,f4f502b7b5d545c8 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/install-generic-python-lib
Prepared variations: _opencv-python
- Creating new "cache" script artifact in the CM local repository ...
- Tags: tmp,install,generic-python-lib,generic,_opencv-python,deps-python-version-3.10.8,deps-python-virtual,deps-python-name-mlperf,script-artifact-f4f502b7b5d545c8
- Changing to /home/val/CM/repos/local/cache/95304a56e5194df9
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,cpu"
- Number of scripts found: 1
- Found script::detect-cpu,586c8a43320142f7 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-cpu
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,os"
- Number of scripts found: 1
- Found script::detect-os,863735b7db8c44fc in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-os
- Running preprocess ...
- running time of script "detect-os,detect,os,info": 0.09 sec.
- Processing env after dependencies ...
- Running preprocess ...
- running time of script "detect,cpu,detect-cpu,info": 0.19 sec.
* Searching for collective script(s) with tags "get,python"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,python
- Number of cached script outputs found: 2
- Found script::get-python3,d0b5dd74373f4a62 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-python3
- Requested version: >= 3.8
- Found cached script output: /home/val/CM/repos/local/cache/01ee27882c904360
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,python,python3,get-python,get-python3": 0.08 sec.
- Processing env after dependencies ...
Collecting opencv-python
Using cached opencv_python-4.6.0.66-cp36-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (60.9 MB)
Requirement already satisfied: numpy>=1.19.3 in /home/val/CM/repos/local/cache/a4149a33cd464596/venv/lib/python3.10/site-packages (from opencv-python) (1.23.4)
Installing collected packages: opencv-python
Successfully installed opencv-python-4.6.0.66
[notice] A new release of pip available: 22.2.2 -> 22.3.1
[notice] To update, run: pip install --upgrade pip
- Removing tmp tag in the script cached output 95304a56e5194df9 ...
- running time of script "install,generic,generic-python-lib": 2.21 sec.
Detected version: 4.6.0.66
- Removing tmp tag in the script cached output ec84afc3e3144d97 ...
- running time of script "get,generic,generic-python-lib": 2.86 sec.
* Searching for collective script(s) with tags "get,generic-python-lib" and variations "pandas"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,generic-python-lib
- Number of cached script outputs found: 5
- Found script::get-generic-python-lib,94b62a682bc44791 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-generic-python-lib
Prepared variations: _pandas
- Creating new "cache" script artifact in the CM local repository ...
- Tags: tmp,get,generic-python-lib,generic,_pandas,deps-python-version-3.10.8,deps-python-virtual,deps-python-name-mlperf,script-artifact-94b62a682bc44791
- Changing to /home/val/CM/repos/local/cache/05a92a7971744135
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,cpu"
- Number of scripts found: 1
- Found script::detect-cpu,586c8a43320142f7 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-cpu
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,os"
- Number of scripts found: 1
- Found script::detect-os,863735b7db8c44fc in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-os
- Running preprocess ...
- running time of script "detect-os,detect,os,info": 0.10 sec.
- Processing env after dependencies ...
- Running preprocess ...
- running time of script "detect,cpu,detect-cpu,info": 0.19 sec.
* Searching for collective script(s) with tags "get,python3"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,python3
- Number of cached script outputs found: 2
- Found script::get-python3,d0b5dd74373f4a62 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-python3
- Requested version: >= 3.8
- Found cached script output: /home/val/CM/repos/local/cache/01ee27882c904360
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,python,python3,get-python,get-python3": 0.08 sec.
- Processing env after dependencies ...
- Running preprocess ...
- Checking prehook dependencies on other CM scripts:
* Searching for collective script(s) with tags "install,generic-python-lib" and variations "pandas"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,install,generic-python-lib
- Number of cached script outputs found: 4
- Found script::install-generic-python-lib,f4f502b7b5d545c8 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/install-generic-python-lib
Prepared variations: _pandas
- Creating new "cache" script artifact in the CM local repository ...
- Tags: tmp,install,generic-python-lib,generic,_pandas,deps-python-version-3.10.8,deps-python-virtual,deps-python-name-mlperf,script-artifact-f4f502b7b5d545c8
- Changing to /home/val/CM/repos/local/cache/b06c5f8c7a1a4c49
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,cpu"
- Number of scripts found: 1
- Found script::detect-cpu,586c8a43320142f7 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-cpu
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,os"
- Number of scripts found: 1
- Found script::detect-os,863735b7db8c44fc in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-os
- Running preprocess ...
- running time of script "detect-os,detect,os,info": 0.10 sec.
- Processing env after dependencies ...
- Running preprocess ...
- running time of script "detect,cpu,detect-cpu,info": 0.20 sec.
* Searching for collective script(s) with tags "get,python"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,python
- Number of cached script outputs found: 2
- Found script::get-python3,d0b5dd74373f4a62 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-python3
- Requested version: >= 3.8
- Found cached script output: /home/val/CM/repos/local/cache/01ee27882c904360
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,python,python3,get-python,get-python3": 0.08 sec.
- Processing env after dependencies ...
Collecting pandas
Using cached pandas-1.5.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (12.1 MB)
Requirement already satisfied: python-dateutil>=2.8.1 in /home/val/CM/repos/local/cache/a4149a33cd464596/venv/lib/python3.10/site-packages (from pandas) (2.8.2)
Requirement already satisfied: numpy>=1.21.0 in /home/val/CM/repos/local/cache/a4149a33cd464596/venv/lib/python3.10/site-packages (from pandas) (1.23.4)
Collecting pytz>=2020.1
Using cached pytz-2022.6-py2.py3-none-any.whl (498 kB)
Requirement already satisfied: six>=1.5 in /home/val/CM/repos/local/cache/a4149a33cd464596/venv/lib/python3.10/site-packages (from python-dateutil>=2.8.1->pandas) (1.16.0)
Installing collected packages: pytz, pandas
Successfully installed pandas-1.5.1 pytz-2022.6
[notice] A new release of pip available: 22.2.2 -> 22.3.1
[notice] To update, run: pip install --upgrade pip
- Removing tmp tag in the script cached output b06c5f8c7a1a4c49 ...
- running time of script "install,generic,generic-python-lib": 4.08 sec.
Detected version: 1.5.1
- Removing tmp tag in the script cached output 05a92a7971744135 ...
- running time of script "get,generic,generic-python-lib": 4.72 sec.
- Processing env after dependencies ...
Collecting requests
Using cached requests-2.28.1-py3-none-any.whl (62 kB)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/val/CM/repos/local/cache/a4149a33cd464596/venv/lib/python3.10/site-packages (from requests->-r /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-dataset-openimages/requirements.txt (line 1)) (1.26.12)
Collecting certifi>=2017.4.17
Using cached certifi-2022.9.24-py3-none-any.whl (161 kB)
Collecting charset-normalizer<3,>=2
Using cached charset_normalizer-2.1.1-py3-none-any.whl (39 kB)
Collecting idna<4,>=2.5
Using cached idna-3.4-py3-none-any.whl (61 kB)
Installing collected packages: idna, charset-normalizer, certifi, requests
Successfully installed certifi-2022.9.24 charset-normalizer-2.1.1 idna-3.4 requests-2.28.1
[notice] A new release of pip available: 22.2.2 -> 22.3.1
[notice] To update, run: pip install --upgrade pip
./openimages_mlperf.sh -d /home/val/CM/repos/local/cache/2e696c40fb444966/install -m 500
Downloading images: 100%|███████████████████████████████████████████████████████| 500/500 [00:36<00:00, 13.75it/s]
Converting annotations to COCO format...
- Removing tmp tag in the script cached output 2e696c40fb444966 ...
- running time of script "get,dataset,openimages,open-images,object-detection,original": 95.18 sec.
* Searching for collective script(s) with tags "mlperf,mlcommons,inference,source,src"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,mlperf,mlcommons,inference,source,src
- Number of cached script outputs found: 1
- Found script::get-mlperf-inference-src,4b57186581024797 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-mlperf-inference-src
Prepared variations: _default,_short-history
- Found cached script output: /home/val/CM/repos/local/cache/200bfc700b804564
- Checking dynamic dependencies on other CM scripts:
- Processing env after dependencies ...
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,src,source,inference,inference-src,inference-source,mlperf,mlcommons": 0.07 sec.
* Searching for collective script(s) with tags "get,generic-python-lib" and variations "pycocotools"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,generic-python-lib
- Number of cached script outputs found: 6
- Found script::get-generic-python-lib,94b62a682bc44791 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-generic-python-lib
Prepared variations: _pycocotools
- Creating new "cache" script artifact in the CM local repository ...
- Tags: tmp,get,generic-python-lib,generic,_pycocotools,deps-python-version-3.10.8,deps-python-virtual,deps-python-name-mlperf,script-artifact-94b62a682bc44791
- Changing to /home/val/CM/repos/local/cache/05c48b7351184399
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,cpu"
- Number of scripts found: 1
- Found script::detect-cpu,586c8a43320142f7 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-cpu
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,os"
- Number of scripts found: 1
- Found script::detect-os,863735b7db8c44fc in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-os
- Running preprocess ...
- running time of script "detect-os,detect,os,info": 0.10 sec.
- Processing env after dependencies ...
- Running preprocess ...
- running time of script "detect,cpu,detect-cpu,info": 0.20 sec.
* Searching for collective script(s) with tags "get,python3"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,python3
- Number of cached script outputs found: 2
- Found script::get-python3,d0b5dd74373f4a62 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-python3
- Requested version: >= 3.8
- Found cached script output: /home/val/CM/repos/local/cache/01ee27882c904360
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,python,python3,get-python,get-python3": 0.08 sec.
- Processing env after dependencies ...
- Running preprocess ...
- Checking prehook dependencies on other CM scripts:
* Searching for collective script(s) with tags "install,generic-python-lib" and variations "pycocotools"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,install,generic-python-lib
- Number of cached script outputs found: 5
- Found script::install-generic-python-lib,f4f502b7b5d545c8 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/install-generic-python-lib
Prepared variations: _pycocotools
- Creating new "cache" script artifact in the CM local repository ...
- Tags: tmp,install,generic-python-lib,generic,_pycocotools,deps-python-version-3.10.8,deps-python-virtual,deps-python-name-mlperf,script-artifact-f4f502b7b5d545c8
- Changing to /home/val/CM/repos/local/cache/a276c55152894f68
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,cpu"
- Number of scripts found: 1
- Found script::detect-cpu,586c8a43320142f7 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-cpu
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,os"
- Number of scripts found: 1
- Found script::detect-os,863735b7db8c44fc in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-os
- Running preprocess ...
- running time of script "detect-os,detect,os,info": 0.10 sec.
- Processing env after dependencies ...
- Running preprocess ...
- running time of script "detect,cpu,detect-cpu,info": 0.20 sec.
* Searching for collective script(s) with tags "get,python"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,python
- Number of cached script outputs found: 2
- Found script::get-python3,d0b5dd74373f4a62 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-python3
- Requested version: >= 3.8
- Found cached script output: /home/val/CM/repos/local/cache/01ee27882c904360
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,python,python3,get-python,get-python3": 0.08 sec.
- Processing env after dependencies ...
Collecting pycocotools
Downloading pycocotools-2.0.6.tar.gz (24 kB)
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing metadata (pyproject.toml) ... done
Collecting matplotlib>=2.1.0
Downloading matplotlib-3.6.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.8 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 11.8/11.8 MB 11.3 MB/s eta 0:00:00
Requirement already satisfied: numpy in /home/val/CM/repos/local/cache/a4149a33cd464596/venv/lib/python3.10/site-packages (from pycocotools) (1.23.4)
Requirement already satisfied: python-dateutil>=2.7 in /home/val/CM/repos/local/cache/a4149a33cd464596/venv/lib/python3.10/site-packages (from matplotlib>=2.1.0->pycocotools) (2.8.2)
Collecting pillow>=6.2.0
Downloading Pillow-9.3.0-cp310-cp310-manylinux_2_28_x86_64.whl (3.3 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.3/3.3 MB 11.3 MB/s eta 0:00:00
Requirement already satisfied: pyparsing>=2.2.1 in /home/val/CM/repos/local/cache/a4149a33cd464596/venv/lib/python3.10/site-packages (from matplotlib>=2.1.0->pycocotools) (3.0.9)
Collecting cycler>=0.10
Downloading cycler-0.11.0-py3-none-any.whl (6.4 kB)
Collecting fonttools>=4.22.0
Downloading fonttools-4.38.0-py3-none-any.whl (965 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 965.4/965.4 kB 10.7 MB/s eta 0:00:00
Collecting contourpy>=1.0.1
Downloading contourpy-1.0.6-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (296 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 296.1/296.1 kB 13.1 MB/s eta 0:00:00
Requirement already satisfied: packaging>=20.0 in /home/val/CM/repos/local/cache/a4149a33cd464596/venv/lib/python3.10/site-packages (from matplotlib>=2.1.0->pycocotools) (21.3)
Collecting kiwisolver>=1.0.1
Downloading kiwisolver-1.4.4-cp310-cp310-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (1.6 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.6/1.6 MB 11.3 MB/s eta 0:00:00
Requirement already satisfied: six>=1.5 in /home/val/CM/repos/local/cache/a4149a33cd464596/venv/lib/python3.10/site-packages (from python-dateutil>=2.7->matplotlib>=2.1.0->pycocotools) (1.16.0)
Building wheels for collected packages: pycocotools
Building wheel for pycocotools (pyproject.toml) ... done
Created wheel for pycocotools: filename=pycocotools-2.0.6-cp310-cp310-linux_x86_64.whl size=377500 sha256=7c30ecda05f4e57a7b11352b1a711d326dd2ec262c7ba3627c0395f4ca35ead5
Stored in directory: /home/val/.cache/pip/wheels/58/e6/f9/f87c8f8be098b51b616871315318329cae12cdb618f4caac93
Successfully built pycocotools
Installing collected packages: pillow, kiwisolver, fonttools, cycler, contourpy, matplotlib, pycocotools
Successfully installed contourpy-1.0.6 cycler-0.11.0 fonttools-4.38.0 kiwisolver-1.4.4 matplotlib-3.6.2 pillow-9.3.0 pycocotools-2.0.6
[notice] A new release of pip available: 22.2.2 -> 22.3.1
[notice] To update, run: pip install --upgrade pip
- Removing tmp tag in the script cached output a276c55152894f68 ...
- running time of script "install,generic,generic-python-lib": 18.14 sec.
Detected version: 2.0.6
- Removing tmp tag in the script cached output 05c48b7351184399 ...
- running time of script "get,generic,generic-python-lib": 18.80 sec.
- Processing env after dependencies ...
- Running preprocess ...
Using MLCommons Inference source from '/home/val/CM/repos/local/cache/200bfc700b804564/inference'
Requirement already satisfied: opencv-python in /home/val/CM/repos/local/cache/a4149a33cd464596/venv/lib/python3.10/site-packages (from -r /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-preprocessed-dataset-openimages/requirements.txt (line 1)) (4.6.0.66)
Requirement already satisfied: Pillow in /home/val/CM/repos/local/cache/a4149a33cd464596/venv/lib/python3.10/site-packages (from -r /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-preprocessed-dataset-openimages/requirements.txt (line 2)) (9.3.0)
Requirement already satisfied: numpy>=1.14.5 in /home/val/CM/repos/local/cache/a4149a33cd464596/venv/lib/python3.10/site-packages (from opencv-python->-r /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-preprocessed-dataset-openimages/requirements.txt (line 1)) (1.23.4)
[notice] A new release of pip available: 22.2.2 -> 22.3.1
[notice] To update, run: pip install --upgrade pip
INFO:coco:loaded 495 images, cache=True, took=31.9sec
/home/val/CM/repos/local/cache/200bfc700b804564/inference/vision/classification_and_detection/python/openimages.py:109: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray.
self.label_list = np.array(self.label_list)
- Removing tmp tag in the script cached output 57ab2b01ae0b4674 ...
- running time of script "get,dataset,openimages,open-images,object-detection,preprocessed": 147.00 sec.
* Searching for collective script(s) with tags "get,generic-python-lib" and variations "pycocotools"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,generic-python-lib
- Number of cached script outputs found: 7
- Found script::get-generic-python-lib,94b62a682bc44791 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-generic-python-lib
Prepared variations: _pycocotools
- Found cached script output: /home/val/CM/repos/local/cache/05c48b7351184399
- Checking dynamic dependencies on other CM scripts:
- Processing env after dependencies ...
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,generic,generic-python-lib": 0.08 sec.
* Searching for collective script(s) with tags "get,ml-model,object-detection,resnext50,fp32" and variations "onnx"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,ml-model,object-detection,resnext50,fp32
- Number of cached script outputs found: 0
- Found script::get-ml-model-retinanet,427bc5665e4541c2 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-ml-model-retinanet
Prepared variations: _onnx
- Creating new "cache" script artifact in the CM local repository ...
- Tags: tmp,get,ml-model,object-detection,resnext50,fp32,retinanet,_onnx,script-artifact-427bc5665e4541c2
- Changing to /home/val/CM/repos/local/cache/b30a624ab3954fe3
- Running preprocess ...
Downloading from https://zenodo.org/record/6617879/files/resnext50_32x4d_fpn.onnx
Downloaded: 100%
- Removing tmp tag in the script cached output b30a624ab3954fe3 ...
- running time of script "get,ml-model,resnext50,retinanet,object-detection,fp32": 13.51 sec.
- Processing env after dependencies ...
- Running preprocess ...
Using MLCommons Inference source from '/home/val/CM/repos/local/cache/200bfc700b804564/inference'
Output Dir: '/home/val/CM/repos/local/cache/200bfc700b804564/inference/vision/classification_and_detection/test_results/onnxruntime-cpu/retinanet/offline/performance/run_1'
retinanet.Offline.target_qps = 1
retinanet.Offline.max_query_count = 10
retinanet.Offline.min_query_count = 10
Requirement already satisfied: pycocotools in /home/val/CM/repos/local/cache/a4149a33cd464596/venv/lib/python3.10/site-packages (from -r /home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference/requirements.txt (line 1)) (2.0.6)
Requirement already satisfied: numpy in /home/val/CM/repos/local/cache/a4149a33cd464596/venv/lib/python3.10/site-packages (from -r /home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference/requirements.txt (line 2)) (1.23.4)
Requirement already satisfied: opencv-python in /home/val/CM/repos/local/cache/a4149a33cd464596/venv/lib/python3.10/site-packages (from -r /home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference/requirements.txt (line 3)) (4.6.0.66)
Requirement already satisfied: protobuf in /home/val/CM/repos/local/cache/a4149a33cd464596/venv/lib/python3.10/site-packages (from -r /home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference/requirements.txt (line 4)) (4.21.9)
Collecting future
Downloading future-0.18.2.tar.gz (829 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 829.2/829.2 kB 8.3 MB/s eta 0:00:00
Preparing metadata (setup.py) ... done
Requirement already satisfied: matplotlib>=2.1.0 in /home/val/CM/repos/local/cache/a4149a33cd464596/venv/lib/python3.10/site-packages (from pycocotools->-r /home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference/requirements.txt (line 1)) (3.6.2)
Requirement already satisfied: kiwisolver>=1.0.1 in /home/val/CM/repos/local/cache/a4149a33cd464596/venv/lib/python3.10/site-packages (from matplotlib>=2.1.0->pycocotools->-r /home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference/requirements.txt (line 1)) (1.4.4)
Requirement already satisfied: packaging>=20.0 in /home/val/CM/repos/local/cache/a4149a33cd464596/venv/lib/python3.10/site-packages (from matplotlib>=2.1.0->pycocotools->-r /home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference/requirements.txt (line 1)) (21.3)
Requirement already satisfied: fonttools>=4.22.0 in /home/val/CM/repos/local/cache/a4149a33cd464596/venv/lib/python3.10/site-packages (from matplotlib>=2.1.0->pycocotools->-r /home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference/requirements.txt (line 1)) (4.38.0)
Requirement already satisfied: pillow>=6.2.0 in /home/val/CM/repos/local/cache/a4149a33cd464596/venv/lib/python3.10/site-packages (from matplotlib>=2.1.0->pycocotools->-r /home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference/requirements.txt (line 1)) (9.3.0)
Requirement already satisfied: cycler>=0.10 in /home/val/CM/repos/local/cache/a4149a33cd464596/venv/lib/python3.10/site-packages (from matplotlib>=2.1.0->pycocotools->-r /home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference/requirements.txt (line 1)) (0.11.0)
Requirement already satisfied: contourpy>=1.0.1 in /home/val/CM/repos/local/cache/a4149a33cd464596/venv/lib/python3.10/site-packages (from matplotlib>=2.1.0->pycocotools->-r /home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference/requirements.txt (line 1)) (1.0.6)
Requirement already satisfied: python-dateutil>=2.7 in /home/val/CM/repos/local/cache/a4149a33cd464596/venv/lib/python3.10/site-packages (from matplotlib>=2.1.0->pycocotools->-r /home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference/requirements.txt (line 1)) (2.8.2)
Requirement already satisfied: pyparsing>=2.2.1 in /home/val/CM/repos/local/cache/a4149a33cd464596/venv/lib/python3.10/site-packages (from matplotlib>=2.1.0->pycocotools->-r /home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference/requirements.txt (line 1)) (3.0.9)
Requirement already satisfied: six>=1.5 in /home/val/CM/repos/local/cache/a4149a33cd464596/venv/lib/python3.10/site-packages (from python-dateutil>=2.7->matplotlib>=2.1.0->pycocotools->-r /home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference/requirements.txt (line 1)) (1.16.0)
Building wheels for collected packages: future
Building wheel for future (setup.py) ... done
Created wheel for future: filename=future-0.18.2-py3-none-any.whl size=491059 sha256=83b910f8fd1f16690c1e5a9a9b9f8507052097c2041894b7f3c4658efb2dc57e
Stored in directory: /home/val/.cache/pip/wheels/22/73/06/557dc4f4ef68179b9d763930d6eec26b88ed7c389b19588a1c
Successfully built future
Installing collected packages: future
Successfully installed future-0.18.2
[notice] A new release of pip available: 22.2.2 -> 22.3.1
[notice] To update, run: pip install --upgrade pip
cd '/home/val/CM/repos/local/cache/200bfc700b804564/inference/vision/classification_and_detection' && OUTPUT_DIR='/home/val/CM/repos/local/cache/200bfc700b804564/inference/vision/classification_and_detection/test_results/onnxruntime-cpu/retinanet/offline/performance/run_1' ./run_local.sh onnxruntime retinanet cpu --scenario Offline --mlperf_conf '/home/val/CM/repos/local/cache/200bfc700b804564/inference/mlperf.conf' --threads 8 --count 10 --user_conf '/home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference/tmp/e9de1c9686be41009f0b7ea96d489582.conf' --cache_dir /home/val/CM/repos/local/cache/57ab2b01ae0b4674
python3 python/main.py --profile retinanet-onnxruntime --mlperf_conf ../../mlperf.conf --model "/home/val/CM/repos/local/cache/b30a624ab3954fe3/resnext50_32x4d_fpn.onnx" --dataset-path /home/val/CM/repos/local/cache/2e696c40fb444966/install --output "/home/val/CM/repos/local/cache/200bfc700b804564/inference/vision/classification_and_detection/test_results/onnxruntime-cpu/retinanet/offline/performance/run_1" --scenario Offline --mlperf_conf /home/val/CM/repos/local/cache/200bfc700b804564/inference/mlperf.conf --threads 8 --count 10 --user_conf /home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference/tmp/e9de1c9686be41009f0b7ea96d489582.conf --cache_dir /home/val/CM/repos/local/cache/57ab2b01ae0b4674
INFO:main:Namespace(dataset='openimages-800-retinanet-onnx', dataset_path='/home/val/CM/repos/local/cache/2e696c40fb444966/install', dataset_list=None, data_format=None, profile='retinanet-onnxruntime', scenario='Offline', max_batchsize=1, model='/home/val/CM/repos/local/cache/b30a624ab3954fe3/resnext50_32x4d_fpn.onnx', output='/home/val/CM/repos/local/cache/200bfc700b804564/inference/vision/classification_and_detection/test_results/onnxruntime-cpu/retinanet/offline/performance/run_1', inputs=['images'], outputs=['boxes', 'labels', 'scores'], backend='onnxruntime', model_name='retinanet', threads=8, qps=None, cache=0, cache_dir='/home/val/CM/repos/local/cache/57ab2b01ae0b4674', accuracy=False, find_peak_performance=False, debug=False, mlperf_conf='/home/val/CM/repos/local/cache/200bfc700b804564/inference/mlperf.conf', user_conf='/home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference/tmp/e9de1c9686be41009f0b7ea96d489582.conf', audit_conf='audit.config', time=None, count=10, performance_sample_count=None, max_latency=None, samples_per_query=8)
INFO:coco:loaded 10 images, cache=0, took=0.0sec
/home/val/CM/repos/local/cache/200bfc700b804564/inference/vision/classification_and_detection/python/openimages.py:109: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray.
self.label_list = np.array(self.label_list)
INFO:main:starting TestScenario.Offline
TestScenario.Offline qps=0.58, mean=68.6325, time=1134.697, queries=660, tiles=50.0:69.8027,80.0:72.2939,90.0:73.5601,95.0:74.8389,99.0:77.4116,99.9:82.9094
================================================
MLPerf Results Summary
================================================
SUT name : PySUT
Scenario : Offline
Mode : PerformanceOnly
Samples per second: 0.581664
Result is : VALID
Min duration satisfied : Yes
Min queries satisfied : Yes
Early stopping satisfied: Yes
================================================
Additional Stats
================================================
Min latency (ns) : 13488169553
Max latency (ns) : 1134675820648
Mean latency (ns) : 581733984405
50.00 percentile latency (ns) : 589646330737
90.00 percentile latency (ns) : 1029191863383
95.00 percentile latency (ns) : 1085167691023
97.00 percentile latency (ns) : 1106253892515
99.00 percentile latency (ns) : 1131001295931
99.90 percentile latency (ns) : 1134675820648
================================================
Test Parameters Used
================================================
samples_per_query : 660
target_qps : 1
target_latency (ns): 0
max_async_queries : 1
min_duration (ms): 600000
max_duration (ms): 0
min_query_count : 1
max_query_count : 10
qsl_rng_seed : 14284205019438841327
sample_index_rng_seed : 4163916728725999944
schedule_rng_seed : 299063814864929621
accuracy_log_rng_seed : 0
accuracy_log_probability : 0
accuracy_log_sampling_target : 0
print_timestamps : 0
performance_issue_unique : 0
performance_issue_same : 0
performance_issue_same_index : 0
performance_sample_count : 64
No warnings encountered during test.
No errors encountered during test.
- running time of script "app,vision,language,mlcommons,mlperf,inference,reference,generic,ref": 1391.93 sec.
* Searching for collective script(s) with tags "app,mlperf,inference,generic" and variations "python,retinanet,onnxruntime,cpu,test"
- Number of scripts found: 1
- Found script::app-mlperf-inference,d775cac873ee4231 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference
Prepared variations: _python,_retinanet,_onnxruntime,_cpu,_test
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,os"
- Number of scripts found: 1
- Found script::detect-os,863735b7db8c44fc in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-os
- Running preprocess ...
- running time of script "detect-os,detect,os,info": 0.09 sec.
* Searching for collective script(s) with tags "detect,cpu"
- Number of scripts found: 1
- Found script::detect-cpu,586c8a43320142f7 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-cpu
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,os"
- Number of scripts found: 1
- Found script::detect-os,863735b7db8c44fc in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-os
- Running preprocess ...
- running time of script "detect-os,detect,os,info": 0.09 sec.
- Processing env after dependencies ...
- Running preprocess ...
- running time of script "detect,cpu,detect-cpu,info": 0.17 sec.
* Searching for collective script(s) with tags "get,sys-utils-cm"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,sys-utils-cm
- Number of cached script outputs found: 1
- Found script::get-sys-utils-cm,bc90993277e84b8e in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-sys-utils-cm
- Found cached script output: /home/val/CM/repos/local/cache/e97b5ea0489546fd
- Checking dynamic dependencies on other CM scripts:
- Processing env after dependencies ...
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,sys-utils-cm": 0.10 sec.
* Searching for collective script(s) with tags "get,python"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,python
- Number of cached script outputs found: 2
- Found script::get-python3,d0b5dd74373f4a62 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-python3
- Requested version: >= 3.8
- Found cached script output: /home/val/CM/repos/local/cache/01ee27882c904360
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,python,python3,get-python,get-python3": 0.07 sec.
* Searching for collective script(s) with tags "get,generic-python-lib" and variations "onnxruntime"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,generic-python-lib
- Number of cached script outputs found: 7
- Found script::get-generic-python-lib,94b62a682bc44791 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-generic-python-lib
Prepared variations: _onnxruntime
- Found cached script output: /home/val/CM/repos/local/cache/3698977e5b3e4807
- Checking dynamic dependencies on other CM scripts:
- Processing env after dependencies ...
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,generic,generic-python-lib": 0.07 sec.
* Searching for collective script(s) with tags "get,loadgen"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,loadgen
- Number of cached script outputs found: 1
- Found script::get-mlperf-inference-loadgen,64c3d98d0ba04950 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-mlperf-inference-loadgen
- Found cached script output: /home/val/CM/repos/local/cache/760e1dcded444891
- Checking dynamic dependencies on other CM scripts:
- Processing env after dependencies ...
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,loadgen,inference,inference-loadgen,mlperf,mlcommons": 0.12 sec.
* Searching for collective script(s) with tags "get,mlcommons,inference,src"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,mlcommons,inference,src
- Number of cached script outputs found: 1
- Found script::get-mlperf-inference-src,4b57186581024797 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-mlperf-inference-src
Prepared variations: _default,_short-history
- Found cached script output: /home/val/CM/repos/local/cache/200bfc700b804564
- Checking dynamic dependencies on other CM scripts:
- Processing env after dependencies ...
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,src,source,inference,inference-src,inference-source,mlperf,mlcommons": 0.06 sec.
* Searching for collective script(s) with tags "get,sut,configs"
- Number of scripts found: 1
- Found script::get-mlperf-inference-sut-configs,c2fbf72009e2445b in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-mlperf-inference-sut-configs
- Running preprocess ...
- running time of script "get,mlperf,sut,configs,sut-configs": 0.07 sec.
* Searching for collective script(s) with tags "get,dataset,object-detection,open-images,openimages,preprocessed" and variations "validation,NCHW,500"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,dataset,object-detection,open-images,openimages,preprocessed
- Number of cached script outputs found: 1
- Found script::get-preprocessed-dataset-openimages,9842f1be8cba4c7b in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-preprocessed-dataset-openimages
Prepared variations: _validation,_NCHW,_500
- Found cached script output: /home/val/CM/repos/local/cache/57ab2b01ae0b4674
- Checking dynamic dependencies on other CM scripts:
- Processing env after dependencies ...
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,dataset,openimages,open-images,object-detection,preprocessed": 0.08 sec.
* Searching for collective script(s) with tags "get,generic-python-lib" and variations "pycocotools"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,generic-python-lib
- Number of cached script outputs found: 7
- Found script::get-generic-python-lib,94b62a682bc44791 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-generic-python-lib
Prepared variations: _pycocotools
- Found cached script output: /home/val/CM/repos/local/cache/05c48b7351184399
- Checking dynamic dependencies on other CM scripts:
- Processing env after dependencies ...
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,generic,generic-python-lib": 0.07 sec.
* Searching for collective script(s) with tags "get,ml-model,object-detection,resnext50,fp32" and variations "onnx"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,ml-model,object-detection,resnext50,fp32
- Number of cached script outputs found: 1
- Found script::get-ml-model-retinanet,427bc5665e4541c2 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-ml-model-retinanet
Prepared variations: _onnx
- Found cached script output: /home/val/CM/repos/local/cache/b30a624ab3954fe3
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,ml-model,resnext50,retinanet,object-detection,fp32": 0.06 sec.
- Processing env after dependencies ...
- Running preprocess ...
Using MLCommons Inference source from '/home/val/CM/repos/local/cache/200bfc700b804564/inference'
Output Dir: '/home/val/CM/repos/local/cache/200bfc700b804564/inference/vision/classification_and_detection/test_results/onnxruntime-cpu/retinanet/offline/accuracy'
retinanet.Offline.target_qps = 1
retinanet.Offline.max_query_count = 10
retinanet.Offline.min_query_count = 10
Requirement already satisfied: pycocotools in /home/val/CM/repos/local/cache/a4149a33cd464596/venv/lib/python3.10/site-packages (from -r /home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference/requirements.txt (line 1)) (2.0.6)
Requirement already satisfied: numpy in /home/val/CM/repos/local/cache/a4149a33cd464596/venv/lib/python3.10/site-packages (from -r /home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference/requirements.txt (line 2)) (1.23.4)
Requirement already satisfied: opencv-python in /home/val/CM/repos/local/cache/a4149a33cd464596/venv/lib/python3.10/site-packages (from -r /home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference/requirements.txt (line 3)) (4.6.0.66)
Requirement already satisfied: protobuf in /home/val/CM/repos/local/cache/a4149a33cd464596/venv/lib/python3.10/site-packages (from -r /home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference/requirements.txt (line 4)) (4.21.9)
Requirement already satisfied: future in /home/val/CM/repos/local/cache/a4149a33cd464596/venv/lib/python3.10/site-packages (from -r /home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference/requirements.txt (line 5)) (0.18.2)
Requirement already satisfied: matplotlib>=2.1.0 in /home/val/CM/repos/local/cache/a4149a33cd464596/venv/lib/python3.10/site-packages (from pycocotools->-r /home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference/requirements.txt (line 1)) (3.6.2)
Requirement already satisfied: fonttools>=4.22.0 in /home/val/CM/repos/local/cache/a4149a33cd464596/venv/lib/python3.10/site-packages (from matplotlib>=2.1.0->pycocotools->-r /home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference/requirements.txt (line 1)) (4.38.0)
Requirement already satisfied: pillow>=6.2.0 in /home/val/CM/repos/local/cache/a4149a33cd464596/venv/lib/python3.10/site-packages (from matplotlib>=2.1.0->pycocotools->-r /home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference/requirements.txt (line 1)) (9.3.0)
Requirement already satisfied: cycler>=0.10 in /home/val/CM/repos/local/cache/a4149a33cd464596/venv/lib/python3.10/site-packages (from matplotlib>=2.1.0->pycocotools->-r /home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference/requirements.txt (line 1)) (0.11.0)
Requirement already satisfied: python-dateutil>=2.7 in /home/val/CM/repos/local/cache/a4149a33cd464596/venv/lib/python3.10/site-packages (from matplotlib>=2.1.0->pycocotools->-r /home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference/requirements.txt (line 1)) (2.8.2)
Requirement already satisfied: packaging>=20.0 in /home/val/CM/repos/local/cache/a4149a33cd464596/venv/lib/python3.10/site-packages (from matplotlib>=2.1.0->pycocotools->-r /home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference/requirements.txt (line 1)) (21.3)
Requirement already satisfied: kiwisolver>=1.0.1 in /home/val/CM/repos/local/cache/a4149a33cd464596/venv/lib/python3.10/site-packages (from matplotlib>=2.1.0->pycocotools->-r /home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference/requirements.txt (line 1)) (1.4.4)
Requirement already satisfied: contourpy>=1.0.1 in /home/val/CM/repos/local/cache/a4149a33cd464596/venv/lib/python3.10/site-packages (from matplotlib>=2.1.0->pycocotools->-r /home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference/requirements.txt (line 1)) (1.0.6)
Requirement already satisfied: pyparsing>=2.2.1 in /home/val/CM/repos/local/cache/a4149a33cd464596/venv/lib/python3.10/site-packages (from matplotlib>=2.1.0->pycocotools->-r /home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference/requirements.txt (line 1)) (3.0.9)
Requirement already satisfied: six>=1.5 in /home/val/CM/repos/local/cache/a4149a33cd464596/venv/lib/python3.10/site-packages (from python-dateutil>=2.7->matplotlib>=2.1.0->pycocotools->-r /home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference/requirements.txt (line 1)) (1.16.0)
[notice] A new release of pip available: 22.2.2 -> 22.3.1
[notice] To update, run: pip install --upgrade pip
cd '/home/val/CM/repos/local/cache/200bfc700b804564/inference/vision/classification_and_detection' && OUTPUT_DIR='/home/val/CM/repos/local/cache/200bfc700b804564/inference/vision/classification_and_detection/test_results/onnxruntime-cpu/retinanet/offline/accuracy' ./run_local.sh onnxruntime retinanet cpu --scenario Offline --mlperf_conf '/home/val/CM/repos/local/cache/200bfc700b804564/inference/mlperf.conf' --threads 8 --count 10 --user_conf '/home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference/tmp/ab431ecdf6c24116a4d8b9ed9b3c987a.conf' --accuracy --cache_dir /home/val/CM/repos/local/cache/57ab2b01ae0b4674
python3 python/main.py --profile retinanet-onnxruntime --mlperf_conf ../../mlperf.conf --model "/home/val/CM/repos/local/cache/b30a624ab3954fe3/resnext50_32x4d_fpn.onnx" --dataset-path /home/val/CM/repos/local/cache/2e696c40fb444966/install --output "/home/val/CM/repos/local/cache/200bfc700b804564/inference/vision/classification_and_detection/test_results/onnxruntime-cpu/retinanet/offline/accuracy" --scenario Offline --mlperf_conf /home/val/CM/repos/local/cache/200bfc700b804564/inference/mlperf.conf --threads 8 --count 10 --user_conf /home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference/tmp/ab431ecdf6c24116a4d8b9ed9b3c987a.conf --accuracy --cache_dir /home/val/CM/repos/local/cache/57ab2b01ae0b4674
INFO:main:Namespace(dataset='openimages-800-retinanet-onnx', dataset_path='/home/val/CM/repos/local/cache/2e696c40fb444966/install', dataset_list=None, data_format=None, profile='retinanet-onnxruntime', scenario='Offline', max_batchsize=1, model='/home/val/CM/repos/local/cache/b30a624ab3954fe3/resnext50_32x4d_fpn.onnx', output='/home/val/CM/repos/local/cache/200bfc700b804564/inference/vision/classification_and_detection/test_results/onnxruntime-cpu/retinanet/offline/accuracy', inputs=['images'], outputs=['boxes', 'labels', 'scores'], backend='onnxruntime', model_name='retinanet', threads=8, qps=None, cache=0, cache_dir='/home/val/CM/repos/local/cache/57ab2b01ae0b4674', accuracy=True, find_peak_performance=False, debug=False, mlperf_conf='/home/val/CM/repos/local/cache/200bfc700b804564/inference/mlperf.conf', user_conf='/home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference/tmp/ab431ecdf6c24116a4d8b9ed9b3c987a.conf', audit_conf='audit.config', time=None, count=10, performance_sample_count=None, max_latency=None, samples_per_query=8)
INFO:coco:loaded 10 images, cache=0, took=0.0sec
/home/val/CM/repos/local/cache/200bfc700b804564/inference/vision/classification_and_detection/python/openimages.py:109: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray.
self.label_list = np.array(self.label_list)
INFO:main:starting TestScenario.Offline
loading annotations into memory...
Done (t=0.04s)
creating index...
index created!
Loading and preparing results...
Converting ndarray to lists...
(1008, 7)
0/1008
DONE (t=0.01s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=0.08s).
Accumulating evaluation results...
DONE (t=0.07s).
Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.548
Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.787
Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.714
Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = -1.000
Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.304
Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.631
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.433
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.648
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.663
Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = -1.000
Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.343
Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.731
TestScenario.Offline qps=0.50, mean=15.1211, time=19.869, acc=48.413%, mAP=54.814%, queries=10, tiles=50.0:14.4207,80.0:16.0521,90.0:19.4148,95.0:19.4598,99.0:19.4958,99.9:19.5039
No warnings encountered during test.
No errors encountered during test.
* Searching for collective script(s) with tags "run,accuracy,mlperf" and variations "openimages"
- Number of scripts found: 1
- Found script::process-mlperf-accuracy,6e809013816b42ea in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/process-mlperf-accuracy
Prepared variations: _openimages
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "get,python"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,python
- Number of cached script outputs found: 2
- Found script::get-python3,d0b5dd74373f4a62 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-python3
- Requested version: >= 3.8
- Found cached script output: /home/val/CM/repos/local/cache/01ee27882c904360
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,python,python3,get-python,get-python3": 0.07 sec.
* Searching for collective script(s) with tags "get,dataset,original,object-detection,open-images"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,dataset,original,object-detection,open-images
- Number of cached script outputs found: 1
- Found script::get-dataset-openimages,0a9d49b644cf4142 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-dataset-openimages
Prepared variations: _validation
- Found cached script output: /home/val/CM/repos/local/cache/2e696c40fb444966
- Checking dynamic dependencies on other CM scripts:
- Processing env after dependencies ...
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,dataset,openimages,open-images,object-detection,original": 0.06 sec.
* Searching for collective script(s) with tags "get,mlcommons,inference,src"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,mlcommons,inference,src
- Number of cached script outputs found: 1
- Found script::get-mlperf-inference-src,4b57186581024797 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-mlperf-inference-src
Prepared variations: _default,_short-history
- Found cached script output: /home/val/CM/repos/local/cache/200bfc700b804564
- Checking dynamic dependencies on other CM scripts:
- Processing env after dependencies ...
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,src,source,inference,inference-src,inference-source,mlperf,mlcommons": 0.06 sec.
* Searching for collective script(s) with tags "get,generic-python-lib" and variations "pycocotools"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,generic-python-lib
- Number of cached script outputs found: 7
- Found script::get-generic-python-lib,94b62a682bc44791 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-generic-python-lib
Prepared variations: _pycocotools
- Found cached script output: /home/val/CM/repos/local/cache/05c48b7351184399
- Checking dynamic dependencies on other CM scripts:
- Processing env after dependencies ...
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,generic,generic-python-lib": 0.07 sec.
- Processing env after dependencies ...
- Running preprocess ...
python3 '/home/val/CM/repos/local/cache/200bfc700b804564/inference/vision/classification_and_detection/tools/accuracy-openimages.py' --mlperf-accuracy-file '/home/val/CM/repos/local/cache/200bfc700b804564/inference/vision/classification_and_detection/test_results/onnxruntime-cpu/retinanet/offline/accuracy/mlperf_log_accuracy.json' --openimages-dir '/home/val/CM/repos/local/cache/2e696c40fb444966/install' > '/home/val/CM/repos/local/cache/200bfc700b804564/inference/vision/classification_and_detection/test_results/onnxruntime-cpu/retinanet/offline/accuracy/accuracy.txt'
loading annotations into memory...
Done (t=0.01s)
creating index...
index created!
Loading and preparing results...
DONE (t=0.01s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=0.06s).
Accumulating evaluation results...
DONE (t=0.07s).
Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.548
Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.787
Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.714
Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = -1.000
Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.304
Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.631
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.433
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.648
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.663
Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = -1.000
Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.343
Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.731
mAP=54.814%
- running time of script "run,mlperf,mlcommons,accuracy,mlc,process-accuracy": 0.71 sec.
- running time of script "app,vision,language,mlcommons,mlperf,inference,reference,generic,ref": 36.82 sec.
* Searching for collective script(s) with tags "get,sut,description"
- Number of scripts found: 1
- Found script::get-mlperf-inference-sut-description,e49a3f758b2d4e7b in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-mlperf-inference-sut-description
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,os"
- Number of scripts found: 1
- Found script::detect-os,863735b7db8c44fc in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-os
- Running preprocess ...
- running time of script "detect-os,detect,os,info": 0.12 sec.
* Searching for collective script(s) with tags "detect,cpu"
- Number of scripts found: 1
- Found script::detect-cpu,586c8a43320142f7 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-cpu
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,os"
- Number of scripts found: 1
- Found script::detect-os,863735b7db8c44fc in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-os
- Running preprocess ...
- running time of script "detect-os,detect,os,info": 0.11 sec.
- Processing env after dependencies ...
- Running preprocess ...
- running time of script "detect,cpu,detect-cpu,info": 0.22 sec.
* Searching for collective script(s) with tags "get,python3"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,python3
- Number of cached script outputs found: 2
- Found script::get-python3,d0b5dd74373f4a62 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-python3
- Requested version: >= 3.8
- Found cached script output: /home/val/CM/repos/local/cache/01ee27882c904360
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,python,python3,get-python,get-python3": 0.07 sec.
* Searching for collective script(s) with tags "get,compiler,gcc"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,compiler,gcc
- Number of cached script outputs found: 1
- Found script::get-gcc,dbf4ab5cbed74372 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-gcc
- Found cached script output: /home/val/CM/repos/local/cache/aa24bd324cbe4ef9
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
* Searching for collective script(s) with tags "get,compiler-flags"
- Number of scripts found: 1
- Found script::get-compiler-flags,31be8b74a69742f8 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-compiler-flags
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,cpu"
- Number of scripts found: 1
- Found script::detect-cpu,586c8a43320142f7 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-cpu
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,os"
- Number of scripts found: 1
- Found script::detect-os,863735b7db8c44fc in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-os
- Running preprocess ...
- running time of script "detect-os,detect,os,info": 0.10 sec.
- Processing env after dependencies ...
- Running preprocess ...
- running time of script "detect,cpu,detect-cpu,info": 0.19 sec.
- Processing env after dependencies ...
- Running preprocess ...
- running time of script "get,compiler-flags": 0.27 sec.
- running time of script "get,gcc,compiler,c-compiler,cpp-compiler,get-gcc": 0.35 sec.
- Processing env after dependencies ...
- Running preprocess ...
Generating SUT description file for default-onnxruntime-1.13.1
- running time of script "get,mlperf,sut,description,system-under-test,system-description": 0.88 sec.
* Searching for collective script(s) with tags "generate,mlperf,inference,submission"
- Number of scripts found: 1
- Found script::generate-mlperf-inference-submission,5f8ab2d0b5874d53 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/generate-mlperf-inference-submission
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "get,python"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,python
- Number of cached script outputs found: 2
- Found script::get-python3,d0b5dd74373f4a62 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-python3
- Requested version: >= 3.8
- Found cached script output: /home/val/CM/repos/local/cache/01ee27882c904360
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,python,python3,get-python,get-python3": 0.07 sec.
* Searching for collective script(s) with tags "mlcommons,inference,src"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,mlcommons,inference,src
- Number of cached script outputs found: 1
- Found script::get-mlperf-inference-src,4b57186581024797 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-mlperf-inference-src
Prepared variations: _default,_short-history
- Found cached script output: /home/val/CM/repos/local/cache/200bfc700b804564
- Checking dynamic dependencies on other CM scripts:
- Processing env after dependencies ...
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,src,source,inference,inference-src,inference-source,mlperf,mlcommons": 0.06 sec.
* Searching for collective script(s) with tags "get,sut,system-description"
- Number of scripts found: 1
- Found script::get-mlperf-inference-sut-description,e49a3f758b2d4e7b in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-mlperf-inference-sut-description
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,os"
- Number of scripts found: 1
- Found script::detect-os,863735b7db8c44fc in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-os
- Running preprocess ...
- running time of script "detect-os,detect,os,info": 0.09 sec.
* Searching for collective script(s) with tags "detect,cpu"
- Number of scripts found: 1
- Found script::detect-cpu,586c8a43320142f7 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-cpu
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,os"
- Number of scripts found: 1
- Found script::detect-os,863735b7db8c44fc in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-os
- Running preprocess ...
- running time of script "detect-os,detect,os,info": 0.10 sec.
- Processing env after dependencies ...
- Running preprocess ...
- running time of script "detect,cpu,detect-cpu,info": 0.18 sec.
* Searching for collective script(s) with tags "get,python3"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,python3
- Number of cached script outputs found: 2
- Found script::get-python3,d0b5dd74373f4a62 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-python3
- Requested version: >= 3.8
- Found cached script output: /home/val/CM/repos/local/cache/01ee27882c904360
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,python,python3,get-python,get-python3": 0.07 sec.
* Searching for collective script(s) with tags "get,compiler,gcc"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,compiler,gcc
- Number of cached script outputs found: 1
- Found script::get-gcc,dbf4ab5cbed74372 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-gcc
- Found cached script output: /home/val/CM/repos/local/cache/aa24bd324cbe4ef9
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
* Searching for collective script(s) with tags "get,compiler-flags"
- Number of scripts found: 1
- Found script::get-compiler-flags,31be8b74a69742f8 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-compiler-flags
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,cpu"
- Number of scripts found: 1
- Found script::detect-cpu,586c8a43320142f7 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-cpu
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,os"
- Number of scripts found: 1
- Found script::detect-os,863735b7db8c44fc in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-os
- Running preprocess ...
- running time of script "detect-os,detect,os,info": 0.12 sec.
- Processing env after dependencies ...
- Running preprocess ...
- running time of script "detect,cpu,detect-cpu,info": 0.23 sec.
- Processing env after dependencies ...
- Running preprocess ...
- running time of script "get,compiler-flags": 0.32 sec.
- running time of script "get,gcc,compiler,c-compiler,cpp-compiler,get-gcc": 0.40 sec.
- Processing env after dependencies ...
- Running preprocess ...
Generating SUT description file for default-onnxruntime-1.13.1
- running time of script "get,mlperf,sut,description,system-under-test,system-description": 0.84 sec.
- Processing env after dependencies ...
- Running preprocess ...
* MLPerf inference submission dir: /home/val/mlperf_submission
* MLPerf inference results dir: /home/val/CM/repos/local/cache/200bfc700b804564/inference/vision/classification_and_detection/test_results
* MLPerf inference division: open
* MLPerf inference submitter: OctoML
* Target: cpu
* Framework: onnxruntime
* MLPerf inference model: retinanet
* mlperf_log_detail.txt
* mlperf_log_accuracy.json
* mlperf_log_summary.txt
* mlperf_log_detail.txt
* mlperf_log_accuracy.json
* mlperf_log_summary.txt
* accuracy.txt
* Searching for collective script(s) with tags "accuracy,truncate,mlc"
- Number of scripts found: 1
- Found script::run-mlperf-accuracy-log-truncator,9d5ec20434084d14 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/run-mlperf-accuracy-log-truncator
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "get,python"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,python
- Number of cached script outputs found: 2
- Found script::get-python3,d0b5dd74373f4a62 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-python3
- Requested version: >= 3.8
- Found cached script output: /home/val/CM/repos/local/cache/01ee27882c904360
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,python,python3,get-python,get-python3": 0.10 sec.
* Searching for collective script(s) with tags "get,mlcommons,inference,src"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,mlcommons,inference,src
- Number of cached script outputs found: 1
- Found script::get-mlperf-inference-src,4b57186581024797 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-mlperf-inference-src
Prepared variations: _default,_short-history
- Found cached script output: /home/val/CM/repos/local/cache/200bfc700b804564
- Checking dynamic dependencies on other CM scripts:
- Processing env after dependencies ...
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,src,source,inference,inference-src,inference-source,mlperf,mlcommons": 0.08 sec.
- Processing env after dependencies ...
- Running preprocess ...
python3 '/home/val/CM/repos/local/cache/200bfc700b804564/inference/tools/submission/truncate_accuracy_log.py' --input /home/val/mlperf_submission --submitter OctoML --backup /home/val/mlperf_submission_logs
INFO:main:open/OctoML/results/onnxruntime-cpu/retinanet/offline/accuracy/mlperf_log_accuracy.json truncated
ERROR:main:no submission in open/OctoML/compliance
INFO:main:Make sure you keep a backup of /home/val/mlperf_submission_logs in case mlperf wants to see the original accuracy logs
- running time of script "run,mlc,mlcommons,mlperf,inference,mlperf-inference,truncation,truncator,truncate,accuracy,accuracy-log,accuracy-log-trancation,accuracy-log-truncator,mlc-accuracy-log-trancation,mlc-accuracy-log-truncator": 0.34 sec.
* Searching for collective script(s) with tags "submission,checker,mlc" and variations "short-run"
- Number of scripts found: 1
- Found script::run-mlperf-inference-submission-checker,15d03ec2c1af4297 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/run-mlperf-inference-submission-checker
Prepared variations: _short-run
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "get,python"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,python
- Number of cached script outputs found: 2
- Found script::get-python3,d0b5dd74373f4a62 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-python3
- Requested version: >= 3.8
- Found cached script output: /home/val/CM/repos/local/cache/01ee27882c904360
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,python,python3,get-python,get-python3": 0.07 sec.
* Searching for collective script(s) with tags "get,mlcommons,inference,src"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,mlcommons,inference,src
- Number of cached script outputs found: 1
- Found script::get-mlperf-inference-src,4b57186581024797 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-mlperf-inference-src
Prepared variations: _default,_short-history
- Found cached script output: /home/val/CM/repos/local/cache/200bfc700b804564
- Checking dynamic dependencies on other CM scripts:
- Processing env after dependencies ...
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,src,source,inference,inference-src,inference-source,mlperf,mlcommons": 0.08 sec.
- Processing env after dependencies ...
- Running preprocess ...
=================================================
python3 /home/val/CM/repos/local/cache/200bfc700b804564/inference/tools/submission/submission_checker1.py --input /home/val/mlperf_submission --submitter OctoML
=================================================
[2022-11-15 18:22:06,398 log_parser.py:50 INFO] Sucessfully loaded MLPerf log from open/OctoML/results/onnxruntime-cpu/retinanet/offline/accuracy/mlperf_log_detail.txt.
[2022-11-15 18:22:06,398 log_parser.py:50 INFO] Sucessfully loaded MLPerf log from open/OctoML/results/onnxruntime-cpu/retinanet/offline/performance/run_1/mlperf_log_detail.txt.
[2022-11-15 18:22:06,399 log_parser.py:50 INFO] Sucessfully loaded MLPerf log from open/OctoML/results/onnxruntime-cpu/retinanet/offline/performance/run_1/mlperf_log_detail.txt.
[2022-11-15 18:22:06,399 submission_checker1.py:1516 INFO] Target latency: None, Latency: 1131001295931, Scenario: Offline
[2022-11-15 18:22:06,399 submission_checker1.py:2455 INFO] ---
[2022-11-15 18:22:06,399 submission_checker1.py:2459 INFO] Results open/OctoML/results/onnxruntime-cpu/retinanet/offline 0.581664
[2022-11-15 18:22:06,399 submission_checker1.py:2461 INFO] ---
[2022-11-15 18:22:06,399 submission_checker1.py:2467 INFO] ---
[2022-11-15 18:22:06,399 submission_checker1.py:2468 INFO] Results=1, NoResults=0
[2022-11-15 18:22:06,399 submission_checker1.py:2474 INFO] SUMMARY: submission looks OK
Traceback (most recent call last):
File "/home/val/Coding/mlcommon/venv-cm/bin/cm", line 8, in <module>
sys.exit(run())
File "/home/val/Coding/mlcommon/venv-cm/lib/python3.10/site-packages/cmind/cli.py", line 35, in run
r = cm.access(argv, out='con')
File "/home/val/Coding/mlcommon/venv-cm/lib/python3.10/site-packages/cmind/core.py", line 486, in access
r = action_addr(i)
File "/home/val/CM/repos/mlcommons@ck/cm-mlops/automation/script/module.py", line 1309, in run
r = self._run_deps(post_deps, clean_env_keys_post_deps, env, state, const, const_state, add_deps_recursive, recursion_spaces,
File "/home/val/CM/repos/mlcommons@ck/cm-mlops/automation/script/module.py", line 1847, in _run_deps
r = self.cmind.access(ii)
File "/home/val/Coding/mlcommon/venv-cm/lib/python3.10/site-packages/cmind/core.py", line 486, in access
r = action_addr(i)
File "/home/val/CM/repos/mlcommons@ck/cm-mlops/automation/script/module.py", line 1309, in run
r = self._run_deps(post_deps, clean_env_keys_post_deps, env, state, const, const_state, add_deps_recursive, recursion_spaces,
File "/home/val/CM/repos/mlcommons@ck/cm-mlops/automation/script/module.py", line 1847, in _run_deps
r = self.cmind.access(ii)
File "/home/val/Coding/mlcommon/venv-cm/lib/python3.10/site-packages/cmind/core.py", line 486, in access
r = action_addr(i)
File "/home/val/CM/repos/mlcommons@ck/cm-mlops/automation/script/module.py", line 1291, in run
r = prepare_and_run_script_with_postprocessing(run_script_input)
File "/home/val/CM/repos/mlcommons@ck/cm-mlops/automation/script/module.py", line 3019, in prepare_and_run_script_with_postprocessing
rr = run_postprocess(customize_code, customize_common_input, recursion_spaces, env, state, const,
File "/home/val/CM/repos/mlcommons@ck/cm-mlops/automation/script/module.py", line 3066, in run_postprocess
r = customize_code.postprocess(ii)
File "/home/val/CM/repos/mlcommons@ck/cm-mlops/script/run-mlperf-inference-submission-checker/customize.py", line 49, in postprocess
import pandas
ModuleNotFoundError: No module named 'pandas'Oh, you are nearly there ;) . I saw this issue before too and I thought I've fixed it for virtual environments. Let me check it! And thank you very much for reporting it - very helpful!
gfursin
commented
1 year ago @ValouBambou - I believe I fixed the issue.
Can you please update your mlcommons@ck repo and rerun the cm command (just remove --rerun to avoid rerunning experiments and only testing the submission checker with pandas):
cm pull repo mlcommons@ckThanks!
gfursin
commented
1 year ago Embarrassing: I forgot some debug info ;) - sorry about that. Can you please try to pull mlcommons@ck repo and try again:
cm pull repo mlcommons@ck
cm run script ...It still fail with the same error :/
Log:
Searching for collective script(s) with tags "run,mlperf,inference,generate-run-cmds" and variations "submission,short,dashboard"
- Number of scripts found: 1
- Found script::run-mlperf-inference-app,4a5d5b13fd7e4ac8 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/run-mlperf-inference-app
Prepared variations: _submission,_short,_dashboard,_all-modes
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,os"
- Number of scripts found: 1
- Found script::detect-os,863735b7db8c44fc in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-os
- Running preprocess ...
- running time of script "detect-os,detect,os,info": 0.11 sec.
* Searching for collective script(s) with tags "detect,cpu"
- Number of scripts found: 1
- Found script::detect-cpu,586c8a43320142f7 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-cpu
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,os"
- Number of scripts found: 1
- Found script::detect-os,863735b7db8c44fc in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-os
- Running preprocess ...
- running time of script "detect-os,detect,os,info": 0.09 sec.
- Processing env after dependencies ...
- Running preprocess ...
- running time of script "detect,cpu,detect-cpu,info": 0.19 sec.
* Searching for collective script(s) with tags "get,python"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,python
- Number of cached script outputs found: 2
- Found script::get-python3,d0b5dd74373f4a62 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-python3
- Requested version: >= 3.8
- Found cached script output: /home/val/CM/repos/local/cache/01ee27882c904360
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,python,python3,get-python,get-python3": 0.09 sec.
* Searching for collective script(s) with tags "get,mlcommons,inference,src"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,mlcommons,inference,src
- Number of cached script outputs found: 1
- Found script::get-mlperf-inference-src,4b57186581024797 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-mlperf-inference-src
Prepared variations: _default,_short-history
- Found cached script output: /home/val/CM/repos/local/cache/200bfc700b804564
- Checking dynamic dependencies on other CM scripts:
- Processing env after dependencies ...
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,src,source,inference,inference-src,inference-source,mlperf,mlcommons": 0.07 sec.
- Processing env after dependencies ...
- Running preprocess ...
Using MLCommons Inference source from /home/val/CM/repos/local/cache/200bfc700b804564/inference
* Searching for collective script(s) with tags "app,mlperf,inference,generic" and variations "python,retinanet,onnxruntime,cpu,test"
- Number of scripts found: 1
- Found script::app-mlperf-inference,d775cac873ee4231 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference
Prepared variations: _python,_retinanet,_onnxruntime,_cpu,_test
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,os"
- Number of scripts found: 1
- Found script::detect-os,863735b7db8c44fc in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-os
- Running preprocess ...
- running time of script "detect-os,detect,os,info": 0.10 sec.
* Searching for collective script(s) with tags "detect,cpu"
- Number of scripts found: 1
- Found script::detect-cpu,586c8a43320142f7 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-cpu
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,os"
- Number of scripts found: 1
- Found script::detect-os,863735b7db8c44fc in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-os
- Running preprocess ...
- running time of script "detect-os,detect,os,info": 0.09 sec.
- Processing env after dependencies ...
- Running preprocess ...
- running time of script "detect,cpu,detect-cpu,info": 0.20 sec.
* Searching for collective script(s) with tags "get,sys-utils-cm"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,sys-utils-cm
- Number of cached script outputs found: 1
- Found script::get-sys-utils-cm,bc90993277e84b8e in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-sys-utils-cm
- Found cached script output: /home/val/CM/repos/local/cache/e97b5ea0489546fd
- Checking dynamic dependencies on other CM scripts:
- Processing env after dependencies ...
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,sys-utils-cm": 0.10 sec.
* Searching for collective script(s) with tags "get,python"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,python
- Number of cached script outputs found: 2
- Found script::get-python3,d0b5dd74373f4a62 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-python3
- Requested version: >= 3.8
- Found cached script output: /home/val/CM/repos/local/cache/01ee27882c904360
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,python,python3,get-python,get-python3": 0.08 sec.
* Searching for collective script(s) with tags "get,generic-python-lib" and variations "onnxruntime"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,generic-python-lib
- Number of cached script outputs found: 7
- Found script::get-generic-python-lib,94b62a682bc44791 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-generic-python-lib
Prepared variations: _onnxruntime
- Found cached script output: /home/val/CM/repos/local/cache/3698977e5b3e4807
- Checking dynamic dependencies on other CM scripts:
- Processing env after dependencies ...
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,generic,generic-python-lib": 0.08 sec.
* Searching for collective script(s) with tags "get,loadgen"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,loadgen
- Number of cached script outputs found: 1
- Found script::get-mlperf-inference-loadgen,64c3d98d0ba04950 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-mlperf-inference-loadgen
- Found cached script output: /home/val/CM/repos/local/cache/760e1dcded444891
- Checking dynamic dependencies on other CM scripts:
- Processing env after dependencies ...
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,loadgen,inference,inference-loadgen,mlperf,mlcommons": 0.10 sec.
* Searching for collective script(s) with tags "get,mlcommons,inference,src"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,mlcommons,inference,src
- Number of cached script outputs found: 1
- Found script::get-mlperf-inference-src,4b57186581024797 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-mlperf-inference-src
Prepared variations: _default,_short-history
- Found cached script output: /home/val/CM/repos/local/cache/200bfc700b804564
- Checking dynamic dependencies on other CM scripts:
- Processing env after dependencies ...
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,src,source,inference,inference-src,inference-source,mlperf,mlcommons": 0.06 sec.
* Searching for collective script(s) with tags "get,sut,configs"
- Number of scripts found: 1
- Found script::get-mlperf-inference-sut-configs,c2fbf72009e2445b in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-mlperf-inference-sut-configs
- Running preprocess ...
- running time of script "get,mlperf,sut,configs,sut-configs": 0.08 sec.
* Searching for collective script(s) with tags "get,dataset,object-detection,open-images,openimages,preprocessed" and variations "validation,NCHW,500"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,dataset,object-detection,open-images,openimages,preprocessed
- Number of cached script outputs found: 1
- Found script::get-preprocessed-dataset-openimages,9842f1be8cba4c7b in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-preprocessed-dataset-openimages
Prepared variations: _validation,_NCHW,_500
- Found cached script output: /home/val/CM/repos/local/cache/57ab2b01ae0b4674
- Checking dynamic dependencies on other CM scripts:
- Processing env after dependencies ...
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,dataset,openimages,open-images,object-detection,preprocessed": 0.08 sec.
* Searching for collective script(s) with tags "get,generic-python-lib" and variations "pycocotools"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,generic-python-lib
- Number of cached script outputs found: 7
- Found script::get-generic-python-lib,94b62a682bc44791 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-generic-python-lib
Prepared variations: _pycocotools
- Found cached script output: /home/val/CM/repos/local/cache/05c48b7351184399
- Checking dynamic dependencies on other CM scripts:
- Processing env after dependencies ...
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,generic,generic-python-lib": 0.08 sec.
* Searching for collective script(s) with tags "get,ml-model,object-detection,resnext50,fp32" and variations "onnx"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,ml-model,object-detection,resnext50,fp32
- Number of cached script outputs found: 1
- Found script::get-ml-model-retinanet,427bc5665e4541c2 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-ml-model-retinanet
Prepared variations: _onnx
- Found cached script output: /home/val/CM/repos/local/cache/b30a624ab3954fe3
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,ml-model,resnext50,retinanet,object-detection,fp32": 0.07 sec.
- Processing env after dependencies ...
- Running preprocess ...
Using MLCommons Inference source from '/home/val/CM/repos/local/cache/200bfc700b804564/inference'
Output Dir: '/home/val/CM/repos/local/cache/200bfc700b804564/inference/vision/classification_and_detection/test_results/onnxruntime-cpu/retinanet/offline/performance/run_1'
retinanet.Offline.target_qps = 1
retinanet.Offline.max_query_count = 10
retinanet.Offline.min_query_count = 10
Requirement already satisfied: pycocotools in /home/val/CM/repos/local/cache/a4149a33cd464596/venv/lib/python3.10/site-packages (from -r /home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference/requirements.txt (line 1)) (2.0.6)
Requirement already satisfied: numpy in /home/val/CM/repos/local/cache/a4149a33cd464596/venv/lib/python3.10/site-packages (from -r /home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference/requirements.txt (line 2)) (1.23.4)
Requirement already satisfied: opencv-python in /home/val/CM/repos/local/cache/a4149a33cd464596/venv/lib/python3.10/site-packages (from -r /home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference/requirements.txt (line 3)) (4.6.0.66)
Requirement already satisfied: protobuf in /home/val/CM/repos/local/cache/a4149a33cd464596/venv/lib/python3.10/site-packages (from -r /home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference/requirements.txt (line 4)) (4.21.9)
Requirement already satisfied: future in /home/val/CM/repos/local/cache/a4149a33cd464596/venv/lib/python3.10/site-packages (from -r /home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference/requirements.txt (line 5)) (0.18.2)
Requirement already satisfied: matplotlib>=2.1.0 in /home/val/CM/repos/local/cache/a4149a33cd464596/venv/lib/python3.10/site-packages (from pycocotools->-r /home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference/requirements.txt (line 1)) (3.6.2)
Requirement already satisfied: python-dateutil>=2.7 in /home/val/CM/repos/local/cache/a4149a33cd464596/venv/lib/python3.10/site-packages (from matplotlib>=2.1.0->pycocotools->-r /home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference/requirements.txt (line 1)) (2.8.2)
Requirement already satisfied: fonttools>=4.22.0 in /home/val/CM/repos/local/cache/a4149a33cd464596/venv/lib/python3.10/site-packages (from matplotlib>=2.1.0->pycocotools->-r /home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference/requirements.txt (line 1)) (4.38.0)
Requirement already satisfied: pyparsing>=2.2.1 in /home/val/CM/repos/local/cache/a4149a33cd464596/venv/lib/python3.10/site-packages (from matplotlib>=2.1.0->pycocotools->-r /home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference/requirements.txt (line 1)) (3.0.9)
Requirement already satisfied: kiwisolver>=1.0.1 in /home/val/CM/repos/local/cache/a4149a33cd464596/venv/lib/python3.10/site-packages (from matplotlib>=2.1.0->pycocotools->-r /home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference/requirements.txt (line 1)) (1.4.4)
Requirement already satisfied: contourpy>=1.0.1 in /home/val/CM/repos/local/cache/a4149a33cd464596/venv/lib/python3.10/site-packages (from matplotlib>=2.1.0->pycocotools->-r /home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference/requirements.txt (line 1)) (1.0.6)
Requirement already satisfied: cycler>=0.10 in /home/val/CM/repos/local/cache/a4149a33cd464596/venv/lib/python3.10/site-packages (from matplotlib>=2.1.0->pycocotools->-r /home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference/requirements.txt (line 1)) (0.11.0)
Requirement already satisfied: packaging>=20.0 in /home/val/CM/repos/local/cache/a4149a33cd464596/venv/lib/python3.10/site-packages (from matplotlib>=2.1.0->pycocotools->-r /home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference/requirements.txt (line 1)) (21.3)
Requirement already satisfied: pillow>=6.2.0 in /home/val/CM/repos/local/cache/a4149a33cd464596/venv/lib/python3.10/site-packages (from matplotlib>=2.1.0->pycocotools->-r /home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference/requirements.txt (line 1)) (9.3.0)
Requirement already satisfied: six>=1.5 in /home/val/CM/repos/local/cache/a4149a33cd464596/venv/lib/python3.10/site-packages (from python-dateutil>=2.7->matplotlib>=2.1.0->pycocotools->-r /home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference/requirements.txt (line 1)) (1.16.0)
[notice] A new release of pip available: 22.2.2 -> 22.3.1
[notice] To update, run: pip install --upgrade pip
cd '/home/val/CM/repos/local/cache/200bfc700b804564/inference/vision/classification_and_detection' && OUTPUT_DIR='/home/val/CM/repos/local/cache/200bfc700b804564/inference/vision/classification_and_detection/test_results/onnxruntime-cpu/retinanet/offline/performance/run_1' ./run_local.sh onnxruntime retinanet cpu --scenario Offline --mlperf_conf '/home/val/CM/repos/local/cache/200bfc700b804564/inference/mlperf.conf' --threads 8 --count 10 --user_conf '/home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference/tmp/a6b43a978a59452393db6c028a042365.conf' --cache_dir /home/val/CM/repos/local/cache/57ab2b01ae0b4674
python3 python/main.py --profile retinanet-onnxruntime --mlperf_conf ../../mlperf.conf --model "/home/val/CM/repos/local/cache/b30a624ab3954fe3/resnext50_32x4d_fpn.onnx" --dataset-path /home/val/CM/repos/local/cache/2e696c40fb444966/install --output "/home/val/CM/repos/local/cache/200bfc700b804564/inference/vision/classification_and_detection/test_results/onnxruntime-cpu/retinanet/offline/performance/run_1" --scenario Offline --mlperf_conf /home/val/CM/repos/local/cache/200bfc700b804564/inference/mlperf.conf --threads 8 --count 10 --user_conf /home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference/tmp/a6b43a978a59452393db6c028a042365.conf --cache_dir /home/val/CM/repos/local/cache/57ab2b01ae0b4674
INFO:main:Namespace(dataset='openimages-800-retinanet-onnx', dataset_path='/home/val/CM/repos/local/cache/2e696c40fb444966/install', dataset_list=None, data_format=None, profile='retinanet-onnxruntime', scenario='Offline', max_batchsize=1, model='/home/val/CM/repos/local/cache/b30a624ab3954fe3/resnext50_32x4d_fpn.onnx', output='/home/val/CM/repos/local/cache/200bfc700b804564/inference/vision/classification_and_detection/test_results/onnxruntime-cpu/retinanet/offline/performance/run_1', inputs=['images'], outputs=['boxes', 'labels', 'scores'], backend='onnxruntime', model_name='retinanet', threads=8, qps=None, cache=0, cache_dir='/home/val/CM/repos/local/cache/57ab2b01ae0b4674', accuracy=False, find_peak_performance=False, debug=False, mlperf_conf='/home/val/CM/repos/local/cache/200bfc700b804564/inference/mlperf.conf', user_conf='/home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference/tmp/a6b43a978a59452393db6c028a042365.conf', audit_conf='audit.config', time=None, count=10, performance_sample_count=None, max_latency=None, samples_per_query=8)
INFO:coco:loaded 10 images, cache=0, took=0.0sec
/home/val/CM/repos/local/cache/200bfc700b804564/inference/vision/classification_and_detection/python/openimages.py:109: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray.
self.label_list = np.array(self.label_list)
INFO:main:starting TestScenario.Offline
TestScenario.Offline qps=0.62, mean=64.8140, time=1071.136, queries=660, tiles=50.0:66.4436,80.0:67.6869,90.0:68.4008,95.0:68.9592,99.0:70.2112,99.9:71.6938
================================================
MLPerf Results Summary
================================================
SUT name : PySUT
Scenario : Offline
Mode : PerformanceOnly
Samples per second: 0.616186
Result is : VALID
Min duration satisfied : Yes
Min queries satisfied : Yes
Early stopping satisfied: Yes
================================================
Additional Stats
================================================
Min latency (ns) : 11212364461
Max latency (ns) : 1071105936252
Mean latency (ns) : 542021530752
50.00 percentile latency (ns) : 541829682355
90.00 percentile latency (ns) : 971472132485
95.00 percentile latency (ns) : 1025570196436
97.00 percentile latency (ns) : 1045385864847
99.00 percentile latency (ns) : 1067564734675
99.90 percentile latency (ns) : 1071105936252
================================================
Test Parameters Used
================================================
samples_per_query : 660
target_qps : 1
target_latency (ns): 0
max_async_queries : 1
min_duration (ms): 600000
max_duration (ms): 0
min_query_count : 1
max_query_count : 10
qsl_rng_seed : 14284205019438841327
sample_index_rng_seed : 4163916728725999944
schedule_rng_seed : 299063814864929621
accuracy_log_rng_seed : 0
accuracy_log_probability : 0
accuracy_log_sampling_target : 0
print_timestamps : 0
performance_issue_unique : 0
performance_issue_same : 0
performance_issue_same_index : 0
performance_sample_count : 64
No warnings encountered during test.
No errors encountered during test.
- running time of script "app,vision,language,mlcommons,mlperf,inference,reference,generic,ref": 1082.57 sec.
* Searching for collective script(s) with tags "app,mlperf,inference,generic" and variations "python,retinanet,onnxruntime,cpu,test"
- Number of scripts found: 1
- Found script::app-mlperf-inference,d775cac873ee4231 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference
Prepared variations: _python,_retinanet,_onnxruntime,_cpu,_test
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,os"
- Number of scripts found: 1
- Found script::detect-os,863735b7db8c44fc in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-os
- Running preprocess ...
- running time of script "detect-os,detect,os,info": 0.08 sec.
* Searching for collective script(s) with tags "detect,cpu"
- Number of scripts found: 1
- Found script::detect-cpu,586c8a43320142f7 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-cpu
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,os"
- Number of scripts found: 1
- Found script::detect-os,863735b7db8c44fc in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-os
- Running preprocess ...
- running time of script "detect-os,detect,os,info": 0.08 sec.
- Processing env after dependencies ...
- Running preprocess ...
- running time of script "detect,cpu,detect-cpu,info": 0.16 sec.
* Searching for collective script(s) with tags "get,sys-utils-cm"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,sys-utils-cm
- Number of cached script outputs found: 1
- Found script::get-sys-utils-cm,bc90993277e84b8e in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-sys-utils-cm
- Found cached script output: /home/val/CM/repos/local/cache/e97b5ea0489546fd
- Checking dynamic dependencies on other CM scripts:
- Processing env after dependencies ...
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,sys-utils-cm": 0.09 sec.
* Searching for collective script(s) with tags "get,python"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,python
- Number of cached script outputs found: 2
- Found script::get-python3,d0b5dd74373f4a62 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-python3
- Requested version: >= 3.8
- Found cached script output: /home/val/CM/repos/local/cache/01ee27882c904360
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,python,python3,get-python,get-python3": 0.07 sec.
* Searching for collective script(s) with tags "get,generic-python-lib" and variations "onnxruntime"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,generic-python-lib
- Number of cached script outputs found: 7
- Found script::get-generic-python-lib,94b62a682bc44791 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-generic-python-lib
Prepared variations: _onnxruntime
- Found cached script output: /home/val/CM/repos/local/cache/3698977e5b3e4807
- Checking dynamic dependencies on other CM scripts:
- Processing env after dependencies ...
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,generic,generic-python-lib": 0.07 sec.
* Searching for collective script(s) with tags "get,loadgen"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,loadgen
- Number of cached script outputs found: 1
- Found script::get-mlperf-inference-loadgen,64c3d98d0ba04950 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-mlperf-inference-loadgen
- Found cached script output: /home/val/CM/repos/local/cache/760e1dcded444891
- Checking dynamic dependencies on other CM scripts:
- Processing env after dependencies ...
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,loadgen,inference,inference-loadgen,mlperf,mlcommons": 0.09 sec.
* Searching for collective script(s) with tags "get,mlcommons,inference,src"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,mlcommons,inference,src
- Number of cached script outputs found: 1
- Found script::get-mlperf-inference-src,4b57186581024797 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-mlperf-inference-src
Prepared variations: _default,_short-history
- Found cached script output: /home/val/CM/repos/local/cache/200bfc700b804564
- Checking dynamic dependencies on other CM scripts:
- Processing env after dependencies ...
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,src,source,inference,inference-src,inference-source,mlperf,mlcommons": 0.06 sec.
* Searching for collective script(s) with tags "get,sut,configs"
- Number of scripts found: 1
- Found script::get-mlperf-inference-sut-configs,c2fbf72009e2445b in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-mlperf-inference-sut-configs
- Running preprocess ...
- running time of script "get,mlperf,sut,configs,sut-configs": 0.06 sec.
* Searching for collective script(s) with tags "get,dataset,object-detection,open-images,openimages,preprocessed" and variations "validation,NCHW,500"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,dataset,object-detection,open-images,openimages,preprocessed
- Number of cached script outputs found: 1
- Found script::get-preprocessed-dataset-openimages,9842f1be8cba4c7b in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-preprocessed-dataset-openimages
Prepared variations: _validation,_NCHW,_500
- Found cached script output: /home/val/CM/repos/local/cache/57ab2b01ae0b4674
- Checking dynamic dependencies on other CM scripts:
- Processing env after dependencies ...
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,dataset,openimages,open-images,object-detection,preprocessed": 0.07 sec.
* Searching for collective script(s) with tags "get,generic-python-lib" and variations "pycocotools"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,generic-python-lib
- Number of cached script outputs found: 7
- Found script::get-generic-python-lib,94b62a682bc44791 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-generic-python-lib
Prepared variations: _pycocotools
- Found cached script output: /home/val/CM/repos/local/cache/05c48b7351184399
- Checking dynamic dependencies on other CM scripts:
- Processing env after dependencies ...
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,generic,generic-python-lib": 0.07 sec.
* Searching for collective script(s) with tags "get,ml-model,object-detection,resnext50,fp32" and variations "onnx"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,ml-model,object-detection,resnext50,fp32
- Number of cached script outputs found: 1
- Found script::get-ml-model-retinanet,427bc5665e4541c2 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-ml-model-retinanet
Prepared variations: _onnx
- Found cached script output: /home/val/CM/repos/local/cache/b30a624ab3954fe3
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,ml-model,resnext50,retinanet,object-detection,fp32": 0.06 sec.
- Processing env after dependencies ...
- Running preprocess ...
Using MLCommons Inference source from '/home/val/CM/repos/local/cache/200bfc700b804564/inference'
Output Dir: '/home/val/CM/repos/local/cache/200bfc700b804564/inference/vision/classification_and_detection/test_results/onnxruntime-cpu/retinanet/offline/accuracy'
retinanet.Offline.target_qps = 1
retinanet.Offline.max_query_count = 10
retinanet.Offline.min_query_count = 10
Requirement already satisfied: pycocotools in /home/val/CM/repos/local/cache/a4149a33cd464596/venv/lib/python3.10/site-packages (from -r /home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference/requirements.txt (line 1)) (2.0.6)
Requirement already satisfied: numpy in /home/val/CM/repos/local/cache/a4149a33cd464596/venv/lib/python3.10/site-packages (from -r /home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference/requirements.txt (line 2)) (1.23.4)
Requirement already satisfied: opencv-python in /home/val/CM/repos/local/cache/a4149a33cd464596/venv/lib/python3.10/site-packages (from -r /home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference/requirements.txt (line 3)) (4.6.0.66)
Requirement already satisfied: protobuf in /home/val/CM/repos/local/cache/a4149a33cd464596/venv/lib/python3.10/site-packages (from -r /home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference/requirements.txt (line 4)) (4.21.9)
Requirement already satisfied: future in /home/val/CM/repos/local/cache/a4149a33cd464596/venv/lib/python3.10/site-packages (from -r /home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference/requirements.txt (line 5)) (0.18.2)
Requirement already satisfied: matplotlib>=2.1.0 in /home/val/CM/repos/local/cache/a4149a33cd464596/venv/lib/python3.10/site-packages (from pycocotools->-r /home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference/requirements.txt (line 1)) (3.6.2)
Requirement already satisfied: contourpy>=1.0.1 in /home/val/CM/repos/local/cache/a4149a33cd464596/venv/lib/python3.10/site-packages (from matplotlib>=2.1.0->pycocotools->-r /home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference/requirements.txt (line 1)) (1.0.6)
Requirement already satisfied: fonttools>=4.22.0 in /home/val/CM/repos/local/cache/a4149a33cd464596/venv/lib/python3.10/site-packages (from matplotlib>=2.1.0->pycocotools->-r /home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference/requirements.txt (line 1)) (4.38.0)
Requirement already satisfied: kiwisolver>=1.0.1 in /home/val/CM/repos/local/cache/a4149a33cd464596/venv/lib/python3.10/site-packages (from matplotlib>=2.1.0->pycocotools->-r /home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference/requirements.txt (line 1)) (1.4.4)
Requirement already satisfied: cycler>=0.10 in /home/val/CM/repos/local/cache/a4149a33cd464596/venv/lib/python3.10/site-packages (from matplotlib>=2.1.0->pycocotools->-r /home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference/requirements.txt (line 1)) (0.11.0)
Requirement already satisfied: python-dateutil>=2.7 in /home/val/CM/repos/local/cache/a4149a33cd464596/venv/lib/python3.10/site-packages (from matplotlib>=2.1.0->pycocotools->-r /home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference/requirements.txt (line 1)) (2.8.2)
Requirement already satisfied: pyparsing>=2.2.1 in /home/val/CM/repos/local/cache/a4149a33cd464596/venv/lib/python3.10/site-packages (from matplotlib>=2.1.0->pycocotools->-r /home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference/requirements.txt (line 1)) (3.0.9)
Requirement already satisfied: packaging>=20.0 in /home/val/CM/repos/local/cache/a4149a33cd464596/venv/lib/python3.10/site-packages (from matplotlib>=2.1.0->pycocotools->-r /home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference/requirements.txt (line 1)) (21.3)
Requirement already satisfied: pillow>=6.2.0 in /home/val/CM/repos/local/cache/a4149a33cd464596/venv/lib/python3.10/site-packages (from matplotlib>=2.1.0->pycocotools->-r /home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference/requirements.txt (line 1)) (9.3.0)
Requirement already satisfied: six>=1.5 in /home/val/CM/repos/local/cache/a4149a33cd464596/venv/lib/python3.10/site-packages (from python-dateutil>=2.7->matplotlib>=2.1.0->pycocotools->-r /home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference/requirements.txt (line 1)) (1.16.0)
[notice] A new release of pip available: 22.2.2 -> 22.3.1
[notice] To update, run: pip install --upgrade pip
cd '/home/val/CM/repos/local/cache/200bfc700b804564/inference/vision/classification_and_detection' && OUTPUT_DIR='/home/val/CM/repos/local/cache/200bfc700b804564/inference/vision/classification_and_detection/test_results/onnxruntime-cpu/retinanet/offline/accuracy' ./run_local.sh onnxruntime retinanet cpu --scenario Offline --mlperf_conf '/home/val/CM/repos/local/cache/200bfc700b804564/inference/mlperf.conf' --threads 8 --count 10 --user_conf '/home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference/tmp/103c417fcc2e41aa881dcb55c1a73a0a.conf' --accuracy --cache_dir /home/val/CM/repos/local/cache/57ab2b01ae0b4674
python3 python/main.py --profile retinanet-onnxruntime --mlperf_conf ../../mlperf.conf --model "/home/val/CM/repos/local/cache/b30a624ab3954fe3/resnext50_32x4d_fpn.onnx" --dataset-path /home/val/CM/repos/local/cache/2e696c40fb444966/install --output "/home/val/CM/repos/local/cache/200bfc700b804564/inference/vision/classification_and_detection/test_results/onnxruntime-cpu/retinanet/offline/accuracy" --scenario Offline --mlperf_conf /home/val/CM/repos/local/cache/200bfc700b804564/inference/mlperf.conf --threads 8 --count 10 --user_conf /home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference/tmp/103c417fcc2e41aa881dcb55c1a73a0a.conf --accuracy --cache_dir /home/val/CM/repos/local/cache/57ab2b01ae0b4674
INFO:main:Namespace(dataset='openimages-800-retinanet-onnx', dataset_path='/home/val/CM/repos/local/cache/2e696c40fb444966/install', dataset_list=None, data_format=None, profile='retinanet-onnxruntime', scenario='Offline', max_batchsize=1, model='/home/val/CM/repos/local/cache/b30a624ab3954fe3/resnext50_32x4d_fpn.onnx', output='/home/val/CM/repos/local/cache/200bfc700b804564/inference/vision/classification_and_detection/test_results/onnxruntime-cpu/retinanet/offline/accuracy', inputs=['images'], outputs=['boxes', 'labels', 'scores'], backend='onnxruntime', model_name='retinanet', threads=8, qps=None, cache=0, cache_dir='/home/val/CM/repos/local/cache/57ab2b01ae0b4674', accuracy=True, find_peak_performance=False, debug=False, mlperf_conf='/home/val/CM/repos/local/cache/200bfc700b804564/inference/mlperf.conf', user_conf='/home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference/tmp/103c417fcc2e41aa881dcb55c1a73a0a.conf', audit_conf='audit.config', time=None, count=10, performance_sample_count=None, max_latency=None, samples_per_query=8)
INFO:coco:loaded 10 images, cache=0, took=0.0sec
/home/val/CM/repos/local/cache/200bfc700b804564/inference/vision/classification_and_detection/python/openimages.py:109: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray.
self.label_list = np.array(self.label_list)
INFO:main:starting TestScenario.Offline
loading annotations into memory...
Done (t=0.01s)
creating index...
index created!
Loading and preparing results...
Converting ndarray to lists...
(1008, 7)
0/1008
DONE (t=0.00s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=0.07s).
Accumulating evaluation results...
DONE (t=0.07s).
Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.548
Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.787
Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.714
Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = -1.000
Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.304
Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.631
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.433
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.648
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.663
Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = -1.000
Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.343
Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.731
TestScenario.Offline qps=0.57, mean=14.0468, time=17.550, acc=48.413%, mAP=54.814%, queries=10, tiles=50.0:13.8647,80.0:14.6810,90.0:16.7305,95.0:16.9965,99.0:17.2093,99.9:17.2572
No warnings encountered during test.
No errors encountered during test.
* Searching for collective script(s) with tags "run,accuracy,mlperf" and variations "openimages"
- Number of scripts found: 1
- Found script::process-mlperf-accuracy,6e809013816b42ea in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/process-mlperf-accuracy
Prepared variations: _openimages
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "get,python"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,python
- Number of cached script outputs found: 2
- Found script::get-python3,d0b5dd74373f4a62 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-python3
- Requested version: >= 3.8
- Found cached script output: /home/val/CM/repos/local/cache/01ee27882c904360
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,python,python3,get-python,get-python3": 0.07 sec.
* Searching for collective script(s) with tags "get,dataset,original,object-detection,open-images"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,dataset,original,object-detection,open-images
- Number of cached script outputs found: 1
- Found script::get-dataset-openimages,0a9d49b644cf4142 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-dataset-openimages
Prepared variations: _validation
- Found cached script output: /home/val/CM/repos/local/cache/2e696c40fb444966
- Checking dynamic dependencies on other CM scripts:
- Processing env after dependencies ...
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,dataset,openimages,open-images,object-detection,original": 0.06 sec.
* Searching for collective script(s) with tags "get,mlcommons,inference,src"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,mlcommons,inference,src
- Number of cached script outputs found: 1
- Found script::get-mlperf-inference-src,4b57186581024797 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-mlperf-inference-src
Prepared variations: _default,_short-history
- Found cached script output: /home/val/CM/repos/local/cache/200bfc700b804564
- Checking dynamic dependencies on other CM scripts:
- Processing env after dependencies ...
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,src,source,inference,inference-src,inference-source,mlperf,mlcommons": 0.06 sec.
* Searching for collective script(s) with tags "get,generic-python-lib" and variations "pycocotools"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,generic-python-lib
- Number of cached script outputs found: 7
- Found script::get-generic-python-lib,94b62a682bc44791 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-generic-python-lib
Prepared variations: _pycocotools
- Found cached script output: /home/val/CM/repos/local/cache/05c48b7351184399
- Checking dynamic dependencies on other CM scripts:
- Processing env after dependencies ...
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,generic,generic-python-lib": 0.07 sec.
- Processing env after dependencies ...
- Running preprocess ...
python3 '/home/val/CM/repos/local/cache/200bfc700b804564/inference/vision/classification_and_detection/tools/accuracy-openimages.py' --mlperf-accuracy-file '/home/val/CM/repos/local/cache/200bfc700b804564/inference/vision/classification_and_detection/test_results/onnxruntime-cpu/retinanet/offline/accuracy/mlperf_log_accuracy.json' --openimages-dir '/home/val/CM/repos/local/cache/2e696c40fb444966/install' > '/home/val/CM/repos/local/cache/200bfc700b804564/inference/vision/classification_and_detection/test_results/onnxruntime-cpu/retinanet/offline/accuracy/accuracy.txt'
loading annotations into memory...
Done (t=0.01s)
creating index...
index created!
Loading and preparing results...
DONE (t=0.01s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=0.06s).
Accumulating evaluation results...
DONE (t=0.07s).
Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.548
Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.787
Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.714
Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = -1.000
Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.304
Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.631
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.433
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.648
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.663
Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = -1.000
Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.343
Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.731
mAP=54.814%
- running time of script "run,mlperf,mlcommons,accuracy,mlc,process-accuracy": 0.70 sec.
- running time of script "app,vision,language,mlcommons,mlperf,inference,reference,generic,ref": 33.79 sec.
* Searching for collective script(s) with tags "get,sut,description"
- Number of scripts found: 1
- Found script::get-mlperf-inference-sut-description,e49a3f758b2d4e7b in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-mlperf-inference-sut-description
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,os"
- Number of scripts found: 1
- Found script::detect-os,863735b7db8c44fc in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-os
- Running preprocess ...
- running time of script "detect-os,detect,os,info": 0.08 sec.
* Searching for collective script(s) with tags "detect,cpu"
- Number of scripts found: 1
- Found script::detect-cpu,586c8a43320142f7 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-cpu
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,os"
- Number of scripts found: 1
- Found script::detect-os,863735b7db8c44fc in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-os
- Running preprocess ...
- running time of script "detect-os,detect,os,info": 0.08 sec.
- Processing env after dependencies ...
- Running preprocess ...
- running time of script "detect,cpu,detect-cpu,info": 0.16 sec.
* Searching for collective script(s) with tags "get,python3"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,python3
- Number of cached script outputs found: 2
- Found script::get-python3,d0b5dd74373f4a62 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-python3
- Requested version: >= 3.8
- Found cached script output: /home/val/CM/repos/local/cache/01ee27882c904360
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,python,python3,get-python,get-python3": 0.07 sec.
* Searching for collective script(s) with tags "get,compiler,gcc"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,compiler,gcc
- Number of cached script outputs found: 1
- Found script::get-gcc,dbf4ab5cbed74372 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-gcc
- Found cached script output: /home/val/CM/repos/local/cache/aa24bd324cbe4ef9
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
* Searching for collective script(s) with tags "get,compiler-flags"
- Number of scripts found: 1
- Found script::get-compiler-flags,31be8b74a69742f8 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-compiler-flags
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,cpu"
- Number of scripts found: 1
- Found script::detect-cpu,586c8a43320142f7 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-cpu
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,os"
- Number of scripts found: 1
- Found script::detect-os,863735b7db8c44fc in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-os
- Running preprocess ...
- running time of script "detect-os,detect,os,info": 0.08 sec.
- Processing env after dependencies ...
- Running preprocess ...
- running time of script "detect,cpu,detect-cpu,info": 0.17 sec.
- Processing env after dependencies ...
- Running preprocess ...
- running time of script "get,compiler-flags": 0.28 sec.
- running time of script "get,gcc,compiler,c-compiler,cpp-compiler,get-gcc": 0.35 sec.
- Processing env after dependencies ...
- Running preprocess ...
Generating SUT description file for default-onnxruntime-1.13.1
- running time of script "get,mlperf,sut,description,system-under-test,system-description": 0.77 sec.
* Searching for collective script(s) with tags "generate,mlperf,inference,submission"
- Number of scripts found: 1
- Found script::generate-mlperf-inference-submission,5f8ab2d0b5874d53 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/generate-mlperf-inference-submission
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "get,python"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,python
- Number of cached script outputs found: 2
- Found script::get-python3,d0b5dd74373f4a62 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-python3
- Requested version: >= 3.8
- Found cached script output: /home/val/CM/repos/local/cache/01ee27882c904360
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,python,python3,get-python,get-python3": 0.07 sec.
* Searching for collective script(s) with tags "mlcommons,inference,src"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,mlcommons,inference,src
- Number of cached script outputs found: 1
- Found script::get-mlperf-inference-src,4b57186581024797 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-mlperf-inference-src
Prepared variations: _default,_short-history
- Found cached script output: /home/val/CM/repos/local/cache/200bfc700b804564
- Checking dynamic dependencies on other CM scripts:
- Processing env after dependencies ...
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,src,source,inference,inference-src,inference-source,mlperf,mlcommons": 0.06 sec.
* Searching for collective script(s) with tags "get,sut,system-description"
- Number of scripts found: 1
- Found script::get-mlperf-inference-sut-description,e49a3f758b2d4e7b in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-mlperf-inference-sut-description
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,os"
- Number of scripts found: 1
- Found script::detect-os,863735b7db8c44fc in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-os
- Running preprocess ...
- running time of script "detect-os,detect,os,info": 0.08 sec.
* Searching for collective script(s) with tags "detect,cpu"
- Number of scripts found: 1
- Found script::detect-cpu,586c8a43320142f7 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-cpu
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,os"
- Number of scripts found: 1
- Found script::detect-os,863735b7db8c44fc in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-os
- Running preprocess ...
- running time of script "detect-os,detect,os,info": 0.08 sec.
- Processing env after dependencies ...
- Running preprocess ...
- running time of script "detect,cpu,detect-cpu,info": 0.17 sec.
* Searching for collective script(s) with tags "get,python3"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,python3
- Number of cached script outputs found: 2
- Found script::get-python3,d0b5dd74373f4a62 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-python3
- Requested version: >= 3.8
- Found cached script output: /home/val/CM/repos/local/cache/01ee27882c904360
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,python,python3,get-python,get-python3": 0.07 sec.
* Searching for collective script(s) with tags "get,compiler,gcc"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,compiler,gcc
- Number of cached script outputs found: 1
- Found script::get-gcc,dbf4ab5cbed74372 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-gcc
- Found cached script output: /home/val/CM/repos/local/cache/aa24bd324cbe4ef9
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
* Searching for collective script(s) with tags "get,compiler-flags"
- Number of scripts found: 1
- Found script::get-compiler-flags,31be8b74a69742f8 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-compiler-flags
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,cpu"
- Number of scripts found: 1
- Found script::detect-cpu,586c8a43320142f7 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-cpu
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,os"
- Number of scripts found: 1
- Found script::detect-os,863735b7db8c44fc in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-os
- Running preprocess ...
- running time of script "detect-os,detect,os,info": 0.08 sec.
- Processing env after dependencies ...
- Running preprocess ...
- running time of script "detect,cpu,detect-cpu,info": 0.16 sec.
- Processing env after dependencies ...
- Running preprocess ...
- running time of script "get,compiler-flags": 0.24 sec.
- running time of script "get,gcc,compiler,c-compiler,cpp-compiler,get-gcc": 0.31 sec.
- Processing env after dependencies ...
- Running preprocess ...
Generating SUT description file for default-onnxruntime-1.13.1
- running time of script "get,mlperf,sut,description,system-under-test,system-description": 0.72 sec.
- Processing env after dependencies ...
- Running preprocess ...
* MLPerf inference submission dir: /home/val/mlperf_submission
* MLPerf inference results dir: /home/val/CM/repos/local/cache/200bfc700b804564/inference/vision/classification_and_detection/test_results
* MLPerf inference division: open
* MLPerf inference submitter: OctoML
* Target: cpu
* Framework: onnxruntime
* MLPerf inference model: retinanet
* mlperf_log_detail.txt
* mlperf_log_accuracy.json
* mlperf_log_summary.txt
* mlperf_log_detail.txt
* mlperf_log_accuracy.json
* mlperf_log_summary.txt
* accuracy.txt
* Searching for collective script(s) with tags "accuracy,truncate,mlc"
- Number of scripts found: 1
- Found script::run-mlperf-accuracy-log-truncator,9d5ec20434084d14 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/run-mlperf-accuracy-log-truncator
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "get,python"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,python
- Number of cached script outputs found: 2
- Found script::get-python3,d0b5dd74373f4a62 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-python3
- Requested version: >= 3.8
- Found cached script output: /home/val/CM/repos/local/cache/01ee27882c904360
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,python,python3,get-python,get-python3": 0.07 sec.
* Searching for collective script(s) with tags "get,mlcommons,inference,src"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,mlcommons,inference,src
- Number of cached script outputs found: 1
- Found script::get-mlperf-inference-src,4b57186581024797 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-mlperf-inference-src
Prepared variations: _default,_short-history
- Found cached script output: /home/val/CM/repos/local/cache/200bfc700b804564
- Checking dynamic dependencies on other CM scripts:
- Processing env after dependencies ...
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,src,source,inference,inference-src,inference-source,mlperf,mlcommons": 0.05 sec.
- Processing env after dependencies ...
- Running preprocess ...
python3 '/home/val/CM/repos/local/cache/200bfc700b804564/inference/tools/submission/truncate_accuracy_log.py' --input /home/val/mlperf_submission --submitter OctoML --backup /home/val/mlperf_submission_logs
INFO:main:open/OctoML/results/onnxruntime-cpu/retinanet/offline/accuracy/mlperf_log_accuracy.json truncated
ERROR:main:no submission in open/OctoML/compliance
INFO:main:Make sure you keep a backup of /home/val/mlperf_submission_logs in case mlperf wants to see the original accuracy logs
- running time of script "run,mlc,mlcommons,mlperf,inference,mlperf-inference,truncation,truncator,truncate,accuracy,accuracy-log,accuracy-log-trancation,accuracy-log-truncator,mlc-accuracy-log-trancation,mlc-accuracy-log-truncator": 0.23 sec.
* Searching for collective script(s) with tags "submission,checker,mlc" and variations "short-run"
- Number of scripts found: 1
- Found script::run-mlperf-inference-submission-checker,15d03ec2c1af4297 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/run-mlperf-inference-submission-checker
Prepared variations: _short-run
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "get,python"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,python
- Number of cached script outputs found: 2
- Found script::get-python3,d0b5dd74373f4a62 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-python3
- Requested version: >= 3.8
- Found cached script output: /home/val/CM/repos/local/cache/01ee27882c904360
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,python,python3,get-python,get-python3": 0.07 sec.
* Searching for collective script(s) with tags "get,mlcommons,inference,src"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,mlcommons,inference,src
- Number of cached script outputs found: 1
- Found script::get-mlperf-inference-src,4b57186581024797 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-mlperf-inference-src
Prepared variations: _default,_short-history
- Found cached script output: /home/val/CM/repos/local/cache/200bfc700b804564
- Checking dynamic dependencies on other CM scripts:
- Processing env after dependencies ...
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,src,source,inference,inference-src,inference-source,mlperf,mlcommons": 0.06 sec.
* Searching for collective script(s) with tags "get,generic-python-lib" and variations "pandas"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,generic-python-lib
- Number of cached script outputs found: 7
- Found script::get-generic-python-lib,94b62a682bc44791 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-generic-python-lib
Prepared variations: _pandas
- Found cached script output: /home/val/CM/repos/local/cache/05a92a7971744135
- Checking dynamic dependencies on other CM scripts:
- Processing env after dependencies ...
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,generic,generic-python-lib": 0.07 sec.
- Processing env after dependencies ...
- Running preprocess ...
=================================================
python3 /home/val/CM/repos/local/cache/200bfc700b804564/inference/tools/submission/submission_checker1.py --input /home/val/mlperf_submission --submitter OctoML
=================================================
[2022-11-15 20:32:09,236 log_parser.py:50 INFO] Sucessfully loaded MLPerf log from open/OctoML/results/onnxruntime-cpu/retinanet/offline/accuracy/mlperf_log_detail.txt.
[2022-11-15 20:32:09,236 log_parser.py:50 INFO] Sucessfully loaded MLPerf log from open/OctoML/results/onnxruntime-cpu/retinanet/offline/performance/run_1/mlperf_log_detail.txt.
[2022-11-15 20:32:09,237 log_parser.py:50 INFO] Sucessfully loaded MLPerf log from open/OctoML/results/onnxruntime-cpu/retinanet/offline/performance/run_1/mlperf_log_detail.txt.
[2022-11-15 20:32:09,237 submission_checker1.py:1516 INFO] Target latency: None, Latency: 1067564734675, Scenario: Offline
[2022-11-15 20:32:09,237 submission_checker1.py:2455 INFO] ---
[2022-11-15 20:32:09,237 submission_checker1.py:2459 INFO] Results open/OctoML/results/onnxruntime-cpu/retinanet/offline 0.616186
[2022-11-15 20:32:09,237 submission_checker1.py:2461 INFO] ---
[2022-11-15 20:32:09,237 submission_checker1.py:2467 INFO] ---
[2022-11-15 20:32:09,237 submission_checker1.py:2468 INFO] Results=1, NoResults=0
[2022-11-15 20:32:09,237 submission_checker1.py:2474 INFO] SUMMARY: submission looks OK
Traceback (most recent call last):
File "/home/val/Coding/mlcommon/venv-cm/bin/cm", line 8, in <module>
sys.exit(run())
File "/home/val/Coding/mlcommon/venv-cm/lib/python3.10/site-packages/cmind/cli.py", line 35, in run
r = cm.access(argv, out='con')
File "/home/val/Coding/mlcommon/venv-cm/lib/python3.10/site-packages/cmind/core.py", line 486, in access
r = action_addr(i)
File "/home/val/CM/repos/mlcommons@ck/cm-mlops/automation/script/module.py", line 1309, in run
r = self._run_deps(post_deps, clean_env_keys_post_deps, env, state, const, const_state, add_deps_recursive, recursion_spaces,
File "/home/val/CM/repos/mlcommons@ck/cm-mlops/automation/script/module.py", line 1847, in _run_deps
r = self.cmind.access(ii)
File "/home/val/Coding/mlcommon/venv-cm/lib/python3.10/site-packages/cmind/core.py", line 486, in access
r = action_addr(i)
File "/home/val/CM/repos/mlcommons@ck/cm-mlops/automation/script/module.py", line 1309, in run
r = self._run_deps(post_deps, clean_env_keys_post_deps, env, state, const, const_state, add_deps_recursive, recursion_spaces,
File "/home/val/CM/repos/mlcommons@ck/cm-mlops/automation/script/module.py", line 1847, in _run_deps
r = self.cmind.access(ii)
File "/home/val/Coding/mlcommon/venv-cm/lib/python3.10/site-packages/cmind/core.py", line 486, in access
r = action_addr(i)
File "/home/val/CM/repos/mlcommons@ck/cm-mlops/automation/script/module.py", line 1291, in run
r = prepare_and_run_script_with_postprocessing(run_script_input)
File "/home/val/CM/repos/mlcommons@ck/cm-mlops/automation/script/module.py", line 3019, in prepare_and_run_script_with_postprocessing
rr = run_postprocess(customize_code, customize_common_input, recursion_spaces, env, state, const,
File "/home/val/CM/repos/mlcommons@ck/cm-mlops/automation/script/module.py", line 3066, in run_postprocess
r = customize_code.postprocess(ii)
File "/home/val/CM/repos/mlcommons@ck/cm-mlops/script/run-mlperf-inference-submission-checker/customize.py", line 49, in postprocess
import pandas
ModuleNotFoundError: No module named 'pandas'
(venv-cm) mlcommon python -m pip install cmind -U
Requirement already satisfied: cmind in ./venv-cm/lib/python3.10/site-packages (1.1.1)
Requirement already satisfied: pyyaml in ./venv-cm/lib/python3.10/site-packages (from cmind) (6.0)
Requirement already satisfied: requests in ./venv-cm/lib/python3.10/site-packages (from cmind) (2.28.1)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in ./venv-cm/lib/python3.10/site-packages (from requests->cmind) (1.26.12)
Requirement already satisfied: charset-normalizer<3,>=2 in ./venv-cm/lib/python3.10/site-packages (from requests->cmind) (2.1.1)
Requirement already satisfied: idna<4,>=2.5 in ./venv-cm/lib/python3.10/site-packages (from requests->cmind) (3.4)
Requirement already satisfied: certifi>=2017.4.17 in ./venv-cm/lib/python3.10/site-packages (from requests->cmind) (2022.9.24)
[notice] A new release of pip available: 22.2.2 -> 22.3.1
[notice] To update, run: pip install --upgrade pip
(venv-cm) mlcommon cm rm cache -f
Deleting CM artifact in /home/val/CM/repos/local/cache/6045b93b5b82418e ...
Deleted!
Deleting CM artifact in /home/val/CM/repos/local/cache/b06c5f8c7a1a4c49 ...
Deleted!
Deleting CM artifact in /home/val/CM/repos/local/cache/3406fea4cdad4090 ...
Deleted!
Deleting CM artifact in /home/val/CM/repos/local/cache/c40185b00eb044b4 ...
Deleted!
Deleting CM artifact in /home/val/CM/repos/local/cache/e97b5ea0489546fd ...
Deleted!
Deleting CM artifact in /home/val/CM/repos/local/cache/def8d54e29fe42ff ...
Deleted!
Deleting CM artifact in /home/val/CM/repos/local/cache/2e696c40fb444966 ...
Deleted!
Deleting CM artifact in /home/val/CM/repos/local/cache/a4149a33cd464596 ...
Deleted!
Deleting CM artifact in /home/val/CM/repos/local/cache/2a2c80023c6f458d ...
Deleted!
Deleting CM artifact in /home/val/CM/repos/local/cache/95304a56e5194df9 ...
Deleted!
Deleting CM artifact in /home/val/CM/repos/local/cache/a276c55152894f68 ...
Deleted!
Deleting CM artifact in /home/val/CM/repos/local/cache/ab50a4a4895047a5 ...
Deleted!
Deleting CM artifact in /home/val/CM/repos/local/cache/01ee27882c904360 ...
Deleted!
Deleting CM artifact in /home/val/CM/repos/local/cache/3698977e5b3e4807 ...
Deleted!
Deleting CM artifact in /home/val/CM/repos/local/cache/a8cc1a6bd3e44e6b ...
Deleted!
Deleting CM artifact in /home/val/CM/repos/local/cache/aa24bd324cbe4ef9 ...
Deleted!
Deleting CM artifact in /home/val/CM/repos/local/cache/ec84afc3e3144d97 ...
Deleted!
Deleting CM artifact in /home/val/CM/repos/local/cache/200bfc700b804564 ...
Deleted!
Deleting CM artifact in /home/val/CM/repos/local/cache/05a92a7971744135 ...
Deleted!
Deleting CM artifact in /home/val/CM/repos/local/cache/cc3c4ddf5ca4466a ...
Deleted!
Deleting CM artifact in /home/val/CM/repos/local/cache/b30a624ab3954fe3 ...
Deleted!
Deleting CM artifact in /home/val/CM/repos/local/cache/760e1dcded444891 ...
Deleted!
Deleting CM artifact in /home/val/CM/repos/local/cache/f399055efa7b448e ...
Deleted!
Deleting CM artifact in /home/val/CM/repos/local/cache/57ab2b01ae0b4674 ...
Deleted!
Deleting CM artifact in /home/val/CM/repos/local/cache/05c48b7351184399 ...
Deleted!
(venv-cm) mlcommon cm run script "get sys-utils-cm" --quiet
* Searching for collective script(s) with tags "get,sys-utils-cm"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,sys-utils-cm
- Number of cached script outputs found: 0
- Found script::get-sys-utils-cm,bc90993277e84b8e in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-sys-utils-cm
- Creating new "cache" script artifact in the CM local repository ...
- Tags: tmp,get,sys-utils-cm,script-artifact-bc90993277e84b8e
- Changing to /home/val/CM/repos/local/cache/f6e03c4418764e25
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,os"
- Number of scripts found: 1
- Found script::detect-os,863735b7db8c44fc in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-os
- Running preprocess ...
- running time of script "detect-os,detect,os,info": 0.10 sec.
- Processing env after dependencies ...
- Running preprocess ...
***********************************************************************
This script will attempt to install minimal system dependencies for CM.
Note that you may be asked for your SUDO password ...
***********************************************************************
- Removing tmp tag in the script cached output f6e03c4418764e25 ...
- running time of script "get,sys-utils-cm": 0.22 sec.
(venv-cm) mlcommon cm run script "install python-venv" --version=3.10.8 --name=mlperf
* Searching for collective script(s) with tags "install,python-venv"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,install,python-venv
- Number of cached script outputs found: 0
- Found script::install-python-venv,7633ebada4584c6c in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/install-python-venv
- Requested version: == 3.10.8
- Creating new "cache" script artifact in the CM local repository ...
- Tags: tmp,install,python-venv,python,get-python-venv,version-3.10.8,name-mlperf,script-artifact-7633ebada4584c6c
- Changing to /home/val/CM/repos/local/cache/ad69a6d50fc2461e
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "get,python,-virtual"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,python,-virtual
- Number of cached script outputs found: 0
- Found script::get-python3,d0b5dd74373f4a62 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-python3
- Requested version: == 3.10.8
- Creating new "cache" script artifact in the CM local repository ...
- Tags: tmp,get,python,-virtual,python3,get-python,get-python3,version-3.10.8,script-artifact-d0b5dd74373f4a62
- Changing to /home/val/CM/repos/local/cache/5b9e7d6f4a154616
- Running preprocess ...
- Searching for versions: == 3.10.8
* /usr/bin/python3
Detected version: 3.10.8
# Found artifact in /usr/bin/python3
# potential PIP version string (if needed): ==3.10.8
- Checking prehook dependencies on other CM scripts:
Detected version: 3.10.8
- Removing tmp tag in the script cached output 5b9e7d6f4a154616 ...
- running time of script "get,python,python3,get-python,get-python3": 0.14 sec.
- Processing env after dependencies ...
- Running preprocess ...
# potential PIP version string (if needed): ==3.10.8
Defaulting to user installation because normal site-packages is not writeable
Requirement already satisfied: virtualenv in /home/val/.local/lib/python3.10/site-packages (20.16.6)
Requirement already satisfied: filelock<4,>=3.4.1 in /home/val/.local/lib/python3.10/site-packages (from virtualenv) (3.8.0)
Requirement already satisfied: distlib<1,>=0.3.6 in /home/val/.local/lib/python3.10/site-packages (from virtualenv) (0.3.6)
Requirement already satisfied: platformdirs<3,>=2.4 in /home/val/.local/lib/python3.10/site-packages (from virtualenv) (2.5.3)
* Searching for collective script(s) with tags "get,python"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,python
- Number of cached script outputs found: 1
- Found script::get-python3,d0b5dd74373f4a62 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-python3
- Requested version: == 3.10.8
- Creating new "cache" script artifact in the CM local repository ...
- Tags: tmp,get,python,python3,get-python,get-python3,version-3.10.8,virtual,name-mlperf,script-artifact-d0b5dd74373f4a62
- Changing to /home/val/CM/repos/local/cache/becbd4357f9f4838
- Running preprocess ...
# potential PIP version string (if needed): ==3.10.8
- Checking prehook dependencies on other CM scripts:
Detected version: 3.10.8
- Removing tmp tag in the script cached output becbd4357f9f4838 ...
- running time of script "get,python,python3,get-python,get-python3": 0.15 sec.
- Removing tmp tag in the script cached output ad69a6d50fc2461e ...
- running time of script "install,python,get-python-venv,python-venv": 3.80 sec.
(venv-cm) mlcommon cm run script --tags=run,mlperf,inference,generate-run-cmds,_submission,_short,_dashboard \
--adr.python.name=mlperf \
--adr.python.version_min=3.8 \
--adr.compiler.tags=gcc \
--adr.openimages-preprocessed.tags=_500 \
--submitter="OctoML" \
--hw_name=default \
--model=retinanet \
--backend=onnxruntime \
--device=cpu \
--scenario=Offline \
--test_query_count=10
* Searching for collective script(s) with tags "run,mlperf,inference,generate-run-cmds" and variations "submission,short,dashboard"
- Number of scripts found: 1
- Found script::run-mlperf-inference-app,4a5d5b13fd7e4ac8 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/run-mlperf-inference-app
Prepared variations: _submission,_short,_dashboard,_all-modes
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,os"
- Number of scripts found: 1
- Found script::detect-os,863735b7db8c44fc in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-os
- Running preprocess ...
- running time of script "detect-os,detect,os,info": 0.10 sec.
* Searching for collective script(s) with tags "detect,cpu"
- Number of scripts found: 1
- Found script::detect-cpu,586c8a43320142f7 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-cpu
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,os"
- Number of scripts found: 1
- Found script::detect-os,863735b7db8c44fc in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-os
- Running preprocess ...
- running time of script "detect-os,detect,os,info": 0.09 sec.
- Processing env after dependencies ...
- Running preprocess ...
- running time of script "detect,cpu,detect-cpu,info": 0.19 sec.
* Searching for collective script(s) with tags "get,python"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,python
- Number of cached script outputs found: 2
- Found script::get-python3,d0b5dd74373f4a62 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-python3
- Requested version: >= 3.8
- Found cached script output: /home/val/CM/repos/local/cache/becbd4357f9f4838
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,python,python3,get-python,get-python3": 0.07 sec.
* Searching for collective script(s) with tags "get,mlcommons,inference,src"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,mlcommons,inference,src
- Number of cached script outputs found: 0
- Found script::get-mlperf-inference-src,4b57186581024797 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-mlperf-inference-src
Prepared variations: _default,_short-history
- Creating new "cache" script artifact in the CM local repository ...
- Tags: tmp,get,mlcommons,inference,src,source,inference-src,inference-source,mlperf,_default,_short-history,script-artifact-4b57186581024797
- Changing to /home/val/CM/repos/local/cache/ce4ce58fe2e44ed5
- Version is not specified - use either default_version from meta or min/max/usable: master
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,os"
- Number of scripts found: 1
- Found script::detect-os,863735b7db8c44fc in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-os
- Running preprocess ...
- running time of script "detect-os,detect,os,info": 0.10 sec.
* Searching for collective script(s) with tags "get,python3"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,python3
- Number of cached script outputs found: 2
- Found script::get-python3,d0b5dd74373f4a62 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-python3
- Requested version: >= 3.8
- Found cached script output: /home/val/CM/repos/local/cache/becbd4357f9f4838
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,python,python3,get-python,get-python3": 0.08 sec.
- Processing env after dependencies ...
- Running preprocess ...
# potential PIP version string (if needed): ==master
******************************************************
Cloning Mlcommons from https://github.com/mlcommons/inference.git with branch master --depth 10 --recurse-submodules...
Cloning into 'inference'...
remote: Enumerating objects: 519, done.
remote: Counting objects: 100% (519/519), done.
remote: Compressing objects: 100% (423/423), done.
remote: Total 519 (delta 103), reused 297 (delta 60), pack-reused 0
Receiving objects: 100% (519/519), 1.64 MiB | 4.88 MiB/s, done.
Resolving deltas: 100% (103/103), done.
Submodule 'language/bert/DeepLearningExamples' (https://github.com/NVIDIA/DeepLearningExamples.git) registered for path 'language/bert/DeepLearningExamples'
Submodule 'third_party/pybind' (https://github.com/pybind/pybind11.git) registered for path 'third_party/pybind'
Submodule 'tools/submission/power-dev' (https://github.com/mlcommons/power-dev) registered for path 'tools/submission/power-dev'
Submodule 'vision/medical_imaging/3d-unet-brats19/nnUnet' (https://github.com/MIC-DKFZ/nnUNet.git) registered for path 'vision/medical_imaging/3d-unet-brats19/nnUnet'
Cloning into '/home/val/CM/repos/local/cache/ce4ce58fe2e44ed5/inference/language/bert/DeepLearningExamples'...
remote: Enumerating objects: 31659, done.
remote: Counting objects: 100% (588/588), done.
remote: Compressing objects: 100% (347/347), done.
remote: Total 31659 (delta 246), reused 518 (delta 228), pack-reused 31071
Receiving objects: 100% (31659/31659), 98.13 MiB | 23.92 MiB/s, done.
Resolving deltas: 100% (22474/22474), done.
Cloning into '/home/val/CM/repos/local/cache/ce4ce58fe2e44ed5/inference/third_party/pybind'...
remote: Enumerating objects: 17856, done.
remote: Counting objects: 100% (55/55), done.
remote: Compressing objects: 100% (53/53), done.
remote: Total 17856 (delta 8), reused 38 (delta 1), pack-reused 17801
Receiving objects: 100% (17856/17856), 8.01 MiB | 1.78 MiB/s, done.
Resolving deltas: 100% (12318/12318), done.
Cloning into '/home/val/CM/repos/local/cache/ce4ce58fe2e44ed5/inference/tools/submission/power-dev'...
remote: Enumerating objects: 1266, done.
remote: Counting objects: 100% (399/399), done.
remote: Compressing objects: 100% (175/175), done.
remote: Total 1266 (delta 260), reused 308 (delta 220), pack-reused 867
Receiving objects: 100% (1266/1266), 590.56 KiB | 4.65 MiB/s, done.
Resolving deltas: 100% (794/794), done.
Cloning into '/home/val/CM/repos/local/cache/ce4ce58fe2e44ed5/inference/vision/medical_imaging/3d-unet-brats19/nnUnet'...
remote: Enumerating objects: 6002, done.
remote: Counting objects: 100% (9/9), done.
remote: Compressing objects: 100% (9/9), done.
remote: Total 6002 (delta 0), reused 9 (delta 0), pack-reused 5993
Receiving objects: 100% (6002/6002), 1.50 MiB | 8.51 MiB/s, done.
Resolving deltas: 100% (4723/4723), done.
Submodule path 'language/bert/DeepLearningExamples': checked out 'b03375bd6c2c5233130e61a3be49e26d1a20ac7c'
Submodule 'PyTorch/SpeechRecognition/Jasper/external/tensorrt-inference-server' (https://github.com/NVIDIA/tensorrt-inference-server.git) registered for path 'language/bert/DeepLearningExamples/PyTorch/SpeechRecognition/Jasper/external/tensorrt-inference-server'
Submodule 'PyTorch/Translation/Transformer/cutlass' (https://github.com/NVIDIA/cutlass.git) registered for path 'language/bert/DeepLearningExamples/PyTorch/Translation/Transformer/cutlass'
Cloning into '/home/val/CM/repos/local/cache/ce4ce58fe2e44ed5/inference/language/bert/DeepLearningExamples/PyTorch/SpeechRecognition/Jasper/external/tensorrt-inference-server'...
remote: Enumerating objects: 34200, done.
remote: Counting objects: 100% (29/29), done.
remote: Compressing objects: 100% (23/23), done.
remote: Total 34200 (delta 8), reused 16 (delta 5), pack-reused 34171
Receiving objects: 100% (34200/34200), 24.39 MiB | 7.18 MiB/s, done.
Resolving deltas: 100% (24602/24602), done.
Cloning into '/home/val/CM/repos/local/cache/ce4ce58fe2e44ed5/inference/language/bert/DeepLearningExamples/PyTorch/Translation/Transformer/cutlass'...
remote: Enumerating objects: 16745, done.
remote: Counting objects: 100% (811/811), done.
remote: Compressing objects: 100% (438/438), done.
remote: Total 16745 (delta 387), reused 688 (delta 333), pack-reused 15934
Receiving objects: 100% (16745/16745), 23.54 MiB | 6.91 MiB/s, done.
Resolving deltas: 100% (12114/12114), done.
Submodule path 'language/bert/DeepLearningExamples/PyTorch/SpeechRecognition/Jasper/external/tensorrt-inference-server': checked out '71f0771cb8cb2a2eb1c6a9433f9a56dd1f206c96'
Submodule path 'language/bert/DeepLearningExamples/PyTorch/Translation/Transformer/cutlass': checked out 'ed2ed4d667ce95e1371bd62db32b6a114e774336'
Submodule 'tools/external/googletest' (https://github.com/google/googletest.git) registered for path 'language/bert/DeepLearningExamples/PyTorch/Translation/Transformer/cutlass/tools/external/googletest'
Cloning into '/home/val/CM/repos/local/cache/ce4ce58fe2e44ed5/inference/language/bert/DeepLearningExamples/PyTorch/Translation/Transformer/cutlass/tools/external/googletest'...
remote: Enumerating objects: 25054, done.
remote: Counting objects: 100% (40/40), done.
remote: Compressing objects: 100% (26/26), done.
remote: Total 25054 (delta 20), reused 27 (delta 12), pack-reused 25014
Receiving objects: 100% (25054/25054), 10.92 MiB | 12.22 MiB/s, done.
Resolving deltas: 100% (18547/18547), done.
Submodule path 'language/bert/DeepLearningExamples/PyTorch/Translation/Transformer/cutlass/tools/external/googletest': checked out '9077ec7efe5b652468ab051e93c67589d5cb8f85'
Submodule path 'third_party/pybind': checked out '25abf7efba0b2990f5a6dfb0a31bc65c0f2f4d17'
Submodule 'tools/clang' (https://github.com/wjakob/clang-cindex-python3) registered for path 'third_party/pybind/tools/clang'
Cloning into '/home/val/CM/repos/local/cache/ce4ce58fe2e44ed5/inference/third_party/pybind/tools/clang'...
remote: Enumerating objects: 355, done.
remote: Counting objects: 100% (14/14), done.
remote: Compressing objects: 100% (10/10), done.
remote: Total 355 (delta 6), reused 8 (delta 4), pack-reused 341
Receiving objects: 100% (355/355), 145.66 KiB | 2.14 MiB/s, done.
Resolving deltas: 100% (147/147), done.
remote: Enumerating objects: 9, done.
remote: Counting objects: 100% (3/3), done.
remote: Total 9 (delta 3), reused 3 (delta 3), pack-reused 6
Unpacking objects: 100% (9/9), 11.12 KiB | 437.00 KiB/s, done.
From https://github.com/wjakob/clang-cindex-python3
* branch 6a00cbc4a9b8e68b71caf7f774b3f9c753ae84d5 -> FETCH_HEAD
Submodule path 'third_party/pybind/tools/clang': checked out '6a00cbc4a9b8e68b71caf7f774b3f9c753ae84d5'
remote: Enumerating objects: 7, done.
remote: Counting objects: 100% (7/7), done.
remote: Compressing objects: 100% (1/1), done.
remote: Total 4 (delta 3), reused 3 (delta 3), pack-reused 0
Unpacking objects: 100% (4/4), 397 bytes | 132.00 KiB/s, done.
From https://github.com/mlcommons/power-dev
* branch e2baedb555f5360811dd9d2d14ed41766db76dfb -> FETCH_HEAD
Submodule path 'tools/submission/power-dev': checked out 'e2baedb555f5360811dd9d2d14ed41766db76dfb'
Submodule path 'vision/medical_imaging/3d-unet-brats19/nnUnet': checked out 'b38c69b345b2f60cd0d053039669e8f988b0c0af'
- Removing tmp tag in the script cached output ce4ce58fe2e44ed5 ...
- running time of script "get,src,source,inference,inference-src,inference-source,mlperf,mlcommons": 29.72 sec.
- Processing env after dependencies ...
- Running preprocess ...
Using MLCommons Inference source from /home/val/CM/repos/local/cache/ce4ce58fe2e44ed5/inference
* Searching for collective script(s) with tags "app,mlperf,inference,generic" and variations "python,retinanet,onnxruntime,cpu,test"
- Number of scripts found: 1
- Found script::app-mlperf-inference,d775cac873ee4231 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference
Prepared variations: _python,_retinanet,_onnxruntime,_cpu,_test
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,os"
- Number of scripts found: 1
- Found script::detect-os,863735b7db8c44fc in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-os
- Running preprocess ...
- running time of script "detect-os,detect,os,info": 0.10 sec.
* Searching for collective script(s) with tags "detect,cpu"
- Number of scripts found: 1
- Found script::detect-cpu,586c8a43320142f7 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-cpu
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,os"
- Number of scripts found: 1
- Found script::detect-os,863735b7db8c44fc in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-os
- Running preprocess ...
- running time of script "detect-os,detect,os,info": 0.09 sec.
- Processing env after dependencies ...
- Running preprocess ...
- running time of script "detect,cpu,detect-cpu,info": 0.18 sec.
* Searching for collective script(s) with tags "get,sys-utils-cm"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,sys-utils-cm
- Number of cached script outputs found: 1
- Found script::get-sys-utils-cm,bc90993277e84b8e in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-sys-utils-cm
- Found cached script output: /home/val/CM/repos/local/cache/f6e03c4418764e25
- Checking dynamic dependencies on other CM scripts:
- Processing env after dependencies ...
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,sys-utils-cm": 0.10 sec.
* Searching for collective script(s) with tags "get,python"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,python
- Number of cached script outputs found: 2
- Found script::get-python3,d0b5dd74373f4a62 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-python3
- Requested version: >= 3.8
- Found cached script output: /home/val/CM/repos/local/cache/becbd4357f9f4838
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,python,python3,get-python,get-python3": 0.07 sec.
* Searching for collective script(s) with tags "get,generic-python-lib" and variations "onnxruntime"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,generic-python-lib
- Number of cached script outputs found: 0
- Found script::get-generic-python-lib,94b62a682bc44791 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-generic-python-lib
Prepared variations: _onnxruntime
- Creating new "cache" script artifact in the CM local repository ...
- Tags: tmp,get,generic-python-lib,generic,_onnxruntime,deps-python-version-3.10.8,deps-python-virtual,deps-python-name-mlperf,script-artifact-94b62a682bc44791
- Changing to /home/val/CM/repos/local/cache/6ddcf39f09084b49
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,cpu"
- Number of scripts found: 1
- Found script::detect-cpu,586c8a43320142f7 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-cpu
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,os"
- Number of scripts found: 1
- Found script::detect-os,863735b7db8c44fc in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-os
- Running preprocess ...
- running time of script "detect-os,detect,os,info": 0.10 sec.
- Processing env after dependencies ...
- Running preprocess ...
- running time of script "detect,cpu,detect-cpu,info": 0.19 sec.
* Searching for collective script(s) with tags "get,python3"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,python3
- Number of cached script outputs found: 2
- Found script::get-python3,d0b5dd74373f4a62 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-python3
- Requested version: >= 3.8
- Found cached script output: /home/val/CM/repos/local/cache/becbd4357f9f4838
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,python,python3,get-python,get-python3": 0.07 sec.
- Processing env after dependencies ...
- Running preprocess ...
- Checking prehook dependencies on other CM scripts:
* Searching for collective script(s) with tags "install,generic-python-lib" and variations "onnxruntime"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,install,generic-python-lib
- Number of cached script outputs found: 0
- Found script::install-generic-python-lib,f4f502b7b5d545c8 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/install-generic-python-lib
Prepared variations: _onnxruntime
- Creating new "cache" script artifact in the CM local repository ...
- Tags: tmp,install,generic-python-lib,generic,_onnxruntime,deps-python-version-3.10.8,deps-python-virtual,deps-python-name-mlperf,script-artifact-f4f502b7b5d545c8
- Changing to /home/val/CM/repos/local/cache/4f9534e6d3484fd8
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,cpu"
- Number of scripts found: 1
- Found script::detect-cpu,586c8a43320142f7 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-cpu
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,os"
- Number of scripts found: 1
- Found script::detect-os,863735b7db8c44fc in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-os
- Running preprocess ...
- running time of script "detect-os,detect,os,info": 0.09 sec.
- Processing env after dependencies ...
- Running preprocess ...
- running time of script "detect,cpu,detect-cpu,info": 0.18 sec.
* Searching for collective script(s) with tags "get,python"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,python
- Number of cached script outputs found: 2
- Found script::get-python3,d0b5dd74373f4a62 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-python3
- Requested version: >= 3.8
- Found cached script output: /home/val/CM/repos/local/cache/becbd4357f9f4838
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,python,python3,get-python,get-python3": 0.07 sec.
- Processing env after dependencies ...
Collecting onnxruntime
Using cached onnxruntime-1.13.1-cp310-cp310-manylinux_2_27_x86_64.whl (4.5 MB)
Collecting numpy>=1.21.6
Using cached numpy-1.23.4-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (17.1 MB)
Collecting flatbuffers
Using cached flatbuffers-22.10.26-py2.py3-none-any.whl (26 kB)
Collecting packaging
Using cached packaging-21.3-py3-none-any.whl (40 kB)
Collecting coloredlogs
Using cached coloredlogs-15.0.1-py2.py3-none-any.whl (46 kB)
Collecting protobuf
Using cached protobuf-4.21.9-cp37-abi3-manylinux2014_x86_64.whl (408 kB)
Collecting sympy
Using cached sympy-1.11.1-py3-none-any.whl (6.5 MB)
Collecting humanfriendly>=9.1
Using cached humanfriendly-10.0-py2.py3-none-any.whl (86 kB)
Collecting pyparsing!=3.0.5,>=2.0.2
Using cached pyparsing-3.0.9-py3-none-any.whl (98 kB)
Collecting mpmath>=0.19
Using cached mpmath-1.2.1-py3-none-any.whl (532 kB)
Installing collected packages: mpmath, flatbuffers, sympy, pyparsing, protobuf, numpy, humanfriendly, packaging, coloredlogs, onnxruntime
Successfully installed coloredlogs-15.0.1 flatbuffers-22.10.26 humanfriendly-10.0 mpmath-1.2.1 numpy-1.23.4 onnxruntime-1.13.1 packaging-21.3 protobuf-4.21.9 pyparsing-3.0.9 sympy-1.11.1
[notice] A new release of pip available: 22.2.2 -> 22.3.1
[notice] To update, run: pip install --upgrade pip
- Removing tmp tag in the script cached output 4f9534e6d3484fd8 ...
- running time of script "install,generic,generic-python-lib": 10.08 sec.
Detected version: 1.13.1
- Removing tmp tag in the script cached output 6ddcf39f09084b49 ...
- running time of script "get,generic,generic-python-lib": 10.70 sec.
* Searching for collective script(s) with tags "get,loadgen"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,loadgen
- Number of cached script outputs found: 0
- Found script::get-mlperf-inference-loadgen,64c3d98d0ba04950 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-mlperf-inference-loadgen
- Creating new "cache" script artifact in the CM local repository ...
- Tags: tmp,get,loadgen,inference,inference-loadgen,mlperf,mlcommons,deps-python-version-3.10.8,deps-python-virtual,deps-python-name-mlperf,script-artifact-64c3d98d0ba04950
- Changing to /home/val/CM/repos/local/cache/d4e616c7c2014060
- Version is not specified - use either default_version from meta or min/max/usable: master
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,os"
- Number of scripts found: 1
- Found script::detect-os,863735b7db8c44fc in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-os
- Running preprocess ...
- running time of script "detect-os,detect,os,info": 0.10 sec.
* Searching for collective script(s) with tags "get,python3"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,python3
- Number of cached script outputs found: 2
- Found script::get-python3,d0b5dd74373f4a62 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-python3
- Requested version: >= 3.8
- Found cached script output: /home/val/CM/repos/local/cache/becbd4357f9f4838
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,python,python3,get-python,get-python3": 0.07 sec.
* Searching for collective script(s) with tags "get,mlcommons,inference,src"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,mlcommons,inference,src
- Number of cached script outputs found: 1
- Found script::get-mlperf-inference-src,4b57186581024797 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-mlperf-inference-src
Prepared variations: _default,_short-history
- Requested version: == master
- Found cached script output: /home/val/CM/repos/local/cache/ce4ce58fe2e44ed5
- Checking dynamic dependencies on other CM scripts:
- Processing env after dependencies ...
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,src,source,inference,inference-src,inference-source,mlperf,mlcommons": 0.06 sec.
* Searching for collective script(s) with tags "get,compiler,gcc"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,compiler,gcc
- Number of cached script outputs found: 0
- Found script::get-gcc,dbf4ab5cbed74372 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-gcc
- Creating new "cache" script artifact in the CM local repository ...
- Tags: tmp,get,compiler,gcc,c-compiler,cpp-compiler,get-gcc,script-artifact-dbf4ab5cbed74372
- Changing to /home/val/CM/repos/local/cache/128d5e94b35742f5
- Running preprocess ...
* /usr/bin/gcc
/usr/bin/gcc --version
gcc (GCC) 12.2.0
Copyright (C) 2022 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Detected version: 12.2.0
# Found artifact in /usr/bin/gcc
/usr/bin/gcc --version
gcc (GCC) 12.2.0
Copyright (C) 2022 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Detected version: 12.2.0
* Searching for collective script(s) with tags "get,compiler-flags"
- Number of scripts found: 1
- Found script::get-compiler-flags,31be8b74a69742f8 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-compiler-flags
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,cpu"
- Number of scripts found: 1
- Found script::detect-cpu,586c8a43320142f7 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-cpu
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,os"
- Number of scripts found: 1
- Found script::detect-os,863735b7db8c44fc in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-os
- Running preprocess ...
- running time of script "detect-os,detect,os,info": 0.10 sec.
- Processing env after dependencies ...
- Running preprocess ...
- running time of script "detect,cpu,detect-cpu,info": 0.19 sec.
- Processing env after dependencies ...
- Running preprocess ...
- running time of script "get,compiler-flags": 0.27 sec.
- Removing tmp tag in the script cached output 128d5e94b35742f5 ...
- running time of script "get,gcc,compiler,c-compiler,cpp-compiler,get-gcc": 0.44 sec.
* Searching for collective script(s) with tags "get,cmake"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,cmake
- Number of cached script outputs found: 0
- Found script::get-cmake,52bf974d791b4fc8 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-cmake
- Creating new "cache" script artifact in the CM local repository ...
- Tags: tmp,get,cmake,get-cmake,script-artifact-52bf974d791b4fc8
- Changing to /home/val/CM/repos/local/cache/1c4324b74cc14355
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,cpu"
- Number of scripts found: 1
- Found script::detect-cpu,586c8a43320142f7 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-cpu
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,os"
- Number of scripts found: 1
- Found script::detect-os,863735b7db8c44fc in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-os
- Running preprocess ...
- running time of script "detect-os,detect,os,info": 0.09 sec.
- Processing env after dependencies ...
- Running preprocess ...
- running time of script "detect,cpu,detect-cpu,info": 0.18 sec.
- Processing env after dependencies ...
- Running preprocess ...
- Checking prehook dependencies on other CM scripts:
* Searching for collective script(s) with tags "install,cmake,prebuilt"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,install,cmake,prebuilt
- Number of cached script outputs found: 0
- Found script::install-cmake-prebuilt,5a39ef05992b4103 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/install-cmake-prebuilt
- Creating new "cache" script artifact in the CM local repository ...
- Tags: tmp,install,cmake,prebuilt,prebuilt-cmake,install-prebuilt-cmake,script-artifact-5a39ef05992b4103
- Changing to /home/val/CM/repos/local/cache/0ee94c61e1ed491d
- Version is not specified - use either default_version from meta or min/max/usable: 3.21.1
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,os"
- Number of scripts found: 1
- Found script::detect-os,863735b7db8c44fc in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-os
- Running preprocess ...
- running time of script "detect-os,detect,os,info": 0.10 sec.
- Processing env after dependencies ...
- Running preprocess ...
# Requested version: 3.21.1
# Prepared package URL: https://github.com/Kitware/CMake/releases/download/v3.21.1/cmake-3.21.1-linux-x86_64.tar.gz
Downloading from https://github.com/Kitware/CMake/releases/download/v3.21.1/cmake-3.21.1-linux-x86_64.tar.gz ...
Downloaded: 100%
# potential PIP version string (if needed): ==3.21.1
Unarchiving cmake-3.21.1-linux-x86_64.tar.gz ...
- Removing tmp tag in the script cached output 0ee94c61e1ed491d ...
- running time of script "install,prebuilt,cmake,prebuilt-cmake,install-prebuilt-cmake": 4.71 sec.
Detected version: 3.21.1
- Removing tmp tag in the script cached output 1c4324b74cc14355 ...
- running time of script "get,cmake,get-cmake": 5.04 sec.
- Processing env after dependencies ...
- Running preprocess ...
# potential PIP version string (if needed): ==master
******************************************************
-- The C compiler identification is GNU 12.2.0
-- The CXX compiler identification is GNU 12.2.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
mlperf_loadgen v1.1
-- Using C++ compiler flags: -O2 -O3 -W -Wall
-- Using C++ standard: 14
-- Using static linker flags:
-- Using shared linker flags: -O2
-- Using output path: /home/val/CM/repos/local/cache/d4e616c7c2014060/build
-- Found PythonInterp: /home/val/CM/repos/local/cache/ad69a6d50fc2461e/venv/bin/python (found version "3.10.8")
-- Using Python interpreter: /home/val/CM/repos/local/cache/ad69a6d50fc2461e/venv/bin/python
-- Configuring done
-- Generating done
-- Build files have been written to: /home/val/CM/repos/local/cache/d4e616c7c2014060/build
******************************************************
[ 7%] Building CXX object CMakeFiles/mlperf_loadgen.dir/bindings/c_api.cc.o
[ 15%] Building CXX object CMakeFiles/mlperf_loadgen.dir/loadgen.cc.o
[ 23%] Building CXX object CMakeFiles/mlperf_loadgen.dir/issue_query_controller.cc.o
[ 30%] Building CXX object CMakeFiles/mlperf_loadgen.dir/logging.cc.o
[ 38%] Building CXX object CMakeFiles/mlperf_loadgen.dir/test_settings_internal.cc.o
[ 46%] Building CXX object CMakeFiles/mlperf_loadgen.dir/early_stopping.cc.o
[ 53%] Building CXX object CMakeFiles/mlperf_loadgen.dir/utils.cc.o
[ 61%] Building CXX object CMakeFiles/mlperf_loadgen.dir/results.cc.o
[ 69%] Building CXX object CMakeFiles/mlperf_loadgen.dir/version.cc.o
[ 76%] Building CXX object CMakeFiles/mlperf_loadgen.dir/version_generated.cc.o
[ 84%] Linking CXX static library libmlperf_loadgen.a
[ 84%] Built target mlperf_loadgen
[ 92%] Building CXX object CMakeFiles/benchmark.dir/benchmark/repro.cpp.o
/home/val/CM/repos/local/cache/ce4ce58fe2e44ed5/inference/loadgen/benchmark/repro.cpp: In member function ‘virtual void QSL::LoadSamplesToRam(const std::vector<long unsigned int>&)’:
/home/val/CM/repos/local/cache/ce4ce58fe2e44ed5/inference/loadgen/benchmark/repro.cpp:37:52: warning: unused parameter ‘samples’ [-Wunused-parameter]
37 | const std::vector<mlperf::QuerySampleIndex>& samples) override {}
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~
/home/val/CM/repos/local/cache/ce4ce58fe2e44ed5/inference/loadgen/benchmark/repro.cpp: In member function ‘virtual void QSL::UnloadSamplesFromRam(const std::vector<long unsigned int>&)’:
/home/val/CM/repos/local/cache/ce4ce58fe2e44ed5/inference/loadgen/benchmark/repro.cpp:39:52: warning: unused parameter ‘samples’ [-Wunused-parameter]
39 | const std::vector<mlperf::QuerySampleIndex>& samples) override {}
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~
/home/val/CM/repos/local/cache/ce4ce58fe2e44ed5/inference/loadgen/benchmark/repro.cpp: In member function ‘virtual void BasicSUT::IssueQuery(const std::vector<mlperf::QuerySample>&)’:
/home/val/CM/repos/local/cache/ce4ce58fe2e44ed5/inference/loadgen/benchmark/repro.cpp:55:11: warning: comparison of integer expressions of different signedness: ‘int’ and ‘std::vector<mlperf::QuerySampleResponse>::size_type’ {aka ‘long unsigned int’} [-Wsign-compare]
55 | if (n > mResponses.size()) {
| ~~^~~~~~~~~~~~~~~~~~~
/home/val/CM/repos/local/cache/ce4ce58fe2e44ed5/inference/loadgen/benchmark/repro.cpp: In member function ‘void QueueSUT::CompleteThread(int)’:
/home/val/CM/repos/local/cache/ce4ce58fe2e44ed5/inference/loadgen/benchmark/repro.cpp:125:27: warning: comparison of integer expressions of different signedness: ‘int’ and ‘size_t’ {aka ‘long unsigned int’} [-Wsign-compare]
125 | for (int i = 0; i < actualSize; i++) {
| ~~^~~~~~~~~~~~
/home/val/CM/repos/local/cache/ce4ce58fe2e44ed5/inference/loadgen/benchmark/repro.cpp: In member function ‘virtual void MultiBasicSUT::IssueQuery(const std::vector<mlperf::QuerySample>&)’:
/home/val/CM/repos/local/cache/ce4ce58fe2e44ed5/inference/loadgen/benchmark/repro.cpp:171:11: warning: comparison of integer expressions of different signedness: ‘int’ and ‘std::vector<mlperf::QuerySampleResponse>::size_type’ {aka ‘long unsigned int’} [-Wsign-compare]
171 | if (n > reponses.size()) {
| ~~^~~~~~~~~~~~~~~~~
[100%] Linking CXX executable benchmark
[100%] Built target benchmark
Install the project...
-- Install configuration: ""
-- Installing: /home/val/CM/repos/local/cache/d4e616c7c2014060/install/lib/libmlperf_loadgen.a
-- Installing: /home/val/CM/repos/local/cache/d4e616c7c2014060/install/include
-- Installing: /home/val/CM/repos/local/cache/d4e616c7c2014060/install/include/benchmark
-- Installing: /home/val/CM/repos/local/cache/d4e616c7c2014060/install/include/results.h
-- Installing: /home/val/CM/repos/local/cache/d4e616c7c2014060/install/include/tests
-- Installing: /home/val/CM/repos/local/cache/d4e616c7c2014060/install/include/tests/loadgen_test.h
-- Installing: /home/val/CM/repos/local/cache/d4e616c7c2014060/install/include/query_sample_library.h
-- Installing: /home/val/CM/repos/local/cache/d4e616c7c2014060/install/include/logging.h
-- Installing: /home/val/CM/repos/local/cache/d4e616c7c2014060/install/include/early_stopping.h
-- Installing: /home/val/CM/repos/local/cache/d4e616c7c2014060/install/include/test_settings_internal.h
-- Installing: /home/val/CM/repos/local/cache/d4e616c7c2014060/install/include/query_sample.h
-- Installing: /home/val/CM/repos/local/cache/d4e616c7c2014060/install/include/system_under_test.h
-- Installing: /home/val/CM/repos/local/cache/d4e616c7c2014060/install/include/demos
-- Installing: /home/val/CM/repos/local/cache/d4e616c7c2014060/install/include/demos/lon
-- Installing: /home/val/CM/repos/local/cache/d4e616c7c2014060/install/include/test_settings.h
-- Installing: /home/val/CM/repos/local/cache/d4e616c7c2014060/install/include/tools
-- Installing: /home/val/CM/repos/local/cache/d4e616c7c2014060/install/include/bindings
-- Installing: /home/val/CM/repos/local/cache/d4e616c7c2014060/install/include/bindings/c_api.h
-- Installing: /home/val/CM/repos/local/cache/d4e616c7c2014060/install/include/version.h
-- Installing: /home/val/CM/repos/local/cache/d4e616c7c2014060/install/include/docs
-- Installing: /home/val/CM/repos/local/cache/d4e616c7c2014060/install/include/docs/src
-- Installing: /home/val/CM/repos/local/cache/d4e616c7c2014060/install/include/utils.h
-- Installing: /home/val/CM/repos/local/cache/d4e616c7c2014060/install/include/loadgen.h
-- Installing: /home/val/CM/repos/local/cache/d4e616c7c2014060/install/include/issue_query_controller.h
Collecting wheel
Using cached wheel-0.38.4-py3-none-any.whl (36 kB)
Installing collected packages: wheel
Successfully installed wheel-0.38.4
[notice] A new release of pip available: 22.2.2 -> 22.3.1
[notice] To update, run: pip install --upgrade pip
running bdist_wheel
running build
running build_ext
building 'mlperf_loadgen' extension
creating build
creating build/temp.linux-x86_64-cpython-310
creating build/temp.linux-x86_64-cpython-310/bindings
creating build/temp.linux-x86_64-cpython-310/generated
gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -ffat-lto-objects -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -std=c++14 -O3 -fPIC -DMAJOR_VERSION=2 -DMINOR_VERSION=1 -I. -I../third_party/pybind/include -I/home/val/CM/repos/local/cache/ad69a6d50fc2461e/venv/include -I/usr/include/python3.10 -c bindings/python_api.cc -o build/temp.linux-x86_64-cpython-310/bindings/python_api.o
In file included from ../third_party/pybind/include/pybind11/cast.h:16,
from ../third_party/pybind/include/pybind11/attr.h:13,
from ../third_party/pybind/include/pybind11/pybind11.h:44,
from ../third_party/pybind/include/pybind11/functional.h:12,
from bindings/python_api.cc:26:
../third_party/pybind/include/pybind11/detail/internals.h: In function ‘pybind11::detail::internals& pybind11::detail::get_internals()’:
../third_party/pybind/include/pybind11/detail/internals.h:200:27: warning: ‘void PyEval_InitThreads()’ is deprecated [-Wdeprecated-declarations]
200 | PyEval_InitThreads();
| ~~~~~~~~~~~~~~~~~~^~
In file included from /usr/include/python3.10/Python.h:130,
from ../third_party/pybind/include/pybind11/detail/common.h:112,
from ../third_party/pybind/include/pybind11/pytypes.h:12,
from ../third_party/pybind/include/pybind11/cast.h:13:
/usr/include/python3.10/ceval.h:122:37: note: declared here
122 | Py_DEPRECATED(3.9) PyAPI_FUNC(void) PyEval_InitThreads(void);
| ^~~~~~~~~~~~~~~~~~
gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -ffat-lto-objects -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -std=c++14 -O3 -fPIC -DMAJOR_VERSION=2 -DMINOR_VERSION=1 -I. -I../third_party/pybind/include -I/home/val/CM/repos/local/cache/ad69a6d50fc2461e/venv/include -I/usr/include/python3.10 -c early_stopping.cc -o build/temp.linux-x86_64-cpython-310/early_stopping.o
gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -ffat-lto-objects -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -std=c++14 -O3 -fPIC -DMAJOR_VERSION=2 -DMINOR_VERSION=1 -I. -I../third_party/pybind/include -I/home/val/CM/repos/local/cache/ad69a6d50fc2461e/venv/include -I/usr/include/python3.10 -c generated/version_generated.cc -o build/temp.linux-x86_64-cpython-310/generated/version_generated.o
gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -ffat-lto-objects -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -std=c++14 -O3 -fPIC -DMAJOR_VERSION=2 -DMINOR_VERSION=1 -I. -I../third_party/pybind/include -I/home/val/CM/repos/local/cache/ad69a6d50fc2461e/venv/include -I/usr/include/python3.10 -c issue_query_controller.cc -o build/temp.linux-x86_64-cpython-310/issue_query_controller.o
gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -ffat-lto-objects -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -std=c++14 -O3 -fPIC -DMAJOR_VERSION=2 -DMINOR_VERSION=1 -I. -I../third_party/pybind/include -I/home/val/CM/repos/local/cache/ad69a6d50fc2461e/venv/include -I/usr/include/python3.10 -c loadgen.cc -o build/temp.linux-x86_64-cpython-310/loadgen.o
gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -ffat-lto-objects -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -std=c++14 -O3 -fPIC -DMAJOR_VERSION=2 -DMINOR_VERSION=1 -I. -I../third_party/pybind/include -I/home/val/CM/repos/local/cache/ad69a6d50fc2461e/venv/include -I/usr/include/python3.10 -c logging.cc -o build/temp.linux-x86_64-cpython-310/logging.o
gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -ffat-lto-objects -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -std=c++14 -O3 -fPIC -DMAJOR_VERSION=2 -DMINOR_VERSION=1 -I. -I../third_party/pybind/include -I/home/val/CM/repos/local/cache/ad69a6d50fc2461e/venv/include -I/usr/include/python3.10 -c results.cc -o build/temp.linux-x86_64-cpython-310/results.o
gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -ffat-lto-objects -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -std=c++14 -O3 -fPIC -DMAJOR_VERSION=2 -DMINOR_VERSION=1 -I. -I../third_party/pybind/include -I/home/val/CM/repos/local/cache/ad69a6d50fc2461e/venv/include -I/usr/include/python3.10 -c test_settings_internal.cc -o build/temp.linux-x86_64-cpython-310/test_settings_internal.o
gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -ffat-lto-objects -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -std=c++14 -O3 -fPIC -DMAJOR_VERSION=2 -DMINOR_VERSION=1 -I. -I../third_party/pybind/include -I/home/val/CM/repos/local/cache/ad69a6d50fc2461e/venv/include -I/usr/include/python3.10 -c utils.cc -o build/temp.linux-x86_64-cpython-310/utils.o
gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -ffat-lto-objects -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -std=c++14 -O3 -fPIC -DMAJOR_VERSION=2 -DMINOR_VERSION=1 -I. -I../third_party/pybind/include -I/home/val/CM/repos/local/cache/ad69a6d50fc2461e/venv/include -I/usr/include/python3.10 -c version.cc -o build/temp.linux-x86_64-cpython-310/version.o
creating build/lib.linux-x86_64-cpython-310
g++ -shared -Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now -flto=auto -Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now -flto=auto -O2 -std=c++14 -O3 build/temp.linux-x86_64-cpython-310/bindings/python_api.o build/temp.linux-x86_64-cpython-310/early_stopping.o build/temp.linux-x86_64-cpython-310/generated/version_generated.o build/temp.linux-x86_64-cpython-310/issue_query_controller.o build/temp.linux-x86_64-cpython-310/loadgen.o build/temp.linux-x86_64-cpython-310/logging.o build/temp.linux-x86_64-cpython-310/results.o build/temp.linux-x86_64-cpython-310/test_settings_internal.o build/temp.linux-x86_64-cpython-310/utils.o build/temp.linux-x86_64-cpython-310/version.o -L/usr/lib -o build/lib.linux-x86_64-cpython-310/mlperf_loadgen.cpython-310-x86_64-linux-gnu.so
/home/val/CM/repos/local/cache/ad69a6d50fc2461e/venv/lib/python3.10/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
warnings.warn(
installing to build/bdist.linux-x86_64/wheel
running install
running install_lib
creating build/bdist.linux-x86_64
creating build/bdist.linux-x86_64/wheel
copying build/lib.linux-x86_64-cpython-310/mlperf_loadgen.cpython-310-x86_64-linux-gnu.so -> build/bdist.linux-x86_64/wheel
running install_egg_info
running egg_info
creating mlperf_loadgen.egg-info
writing mlperf_loadgen.egg-info/PKG-INFO
writing dependency_links to mlperf_loadgen.egg-info/dependency_links.txt
writing top-level names to mlperf_loadgen.egg-info/top_level.txt
writing manifest file 'mlperf_loadgen.egg-info/SOURCES.txt'
reading manifest file 'mlperf_loadgen.egg-info/SOURCES.txt'
writing manifest file 'mlperf_loadgen.egg-info/SOURCES.txt'
Copying mlperf_loadgen.egg-info to build/bdist.linux-x86_64/wheel/mlperf_loadgen-2.1-py3.10.egg-info
running install_scripts
creating build/bdist.linux-x86_64/wheel/mlperf_loadgen-2.1.dist-info/WHEEL
creating 'dist/mlperf_loadgen-2.1-cp310-cp310-linux_x86_64.whl' and adding 'build/bdist.linux-x86_64/wheel' to it
adding 'mlperf_loadgen.cpython-310-x86_64-linux-gnu.so'
adding 'mlperf_loadgen-2.1.dist-info/METADATA'
adding 'mlperf_loadgen-2.1.dist-info/WHEEL'
adding 'mlperf_loadgen-2.1.dist-info/top_level.txt'
adding 'mlperf_loadgen-2.1.dist-info/RECORD'
removing build/bdist.linux-x86_64/wheel
Processing ./dist/mlperf_loadgen-2.1-cp310-cp310-linux_x86_64.whl
Installing collected packages: mlperf-loadgen
Successfully installed mlperf-loadgen-2.1
[notice] A new release of pip available: 22.2.2 -> 22.3.1
[notice] To update, run: pip install --upgrade pip
******************************************************
Loadgen is built and installed to /home/val/CM/repos/local/cache/d4e616c7c2014060/install ...
- Removing tmp tag in the script cached output d4e616c7c2014060 ...
- running time of script "get,loadgen,inference,inference-loadgen,mlperf,mlcommons": 71.39 sec.
* Searching for collective script(s) with tags "get,mlcommons,inference,src"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,mlcommons,inference,src
- Number of cached script outputs found: 1
- Found script::get-mlperf-inference-src,4b57186581024797 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-mlperf-inference-src
Prepared variations: _default,_short-history
- Found cached script output: /home/val/CM/repos/local/cache/ce4ce58fe2e44ed5
- Checking dynamic dependencies on other CM scripts:
- Processing env after dependencies ...
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,src,source,inference,inference-src,inference-source,mlperf,mlcommons": 0.07 sec.
* Searching for collective script(s) with tags "get,sut,configs"
- Number of scripts found: 1
- Found script::get-mlperf-inference-sut-configs,c2fbf72009e2445b in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-mlperf-inference-sut-configs
- Running preprocess ...
- running time of script "get,mlperf,sut,configs,sut-configs": 0.08 sec.
* Searching for collective script(s) with tags "get,dataset,object-detection,open-images,openimages,preprocessed" and variations "validation,NCHW,500"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,dataset,object-detection,open-images,openimages,preprocessed
- Number of cached script outputs found: 0
- Found script::get-preprocessed-dataset-openimages,9842f1be8cba4c7b in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-preprocessed-dataset-openimages
Prepared variations: _validation,_NCHW,_500
- Creating new "cache" script artifact in the CM local repository ...
- Tags: tmp,get,dataset,object-detection,open-images,openimages,preprocessed,_validation,_NCHW,_500,script-artifact-9842f1be8cba4c7b
- Changing to /home/val/CM/repos/local/cache/8ae2ae9d3c984709
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "get,python3"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,python3
- Number of cached script outputs found: 2
- Found script::get-python3,d0b5dd74373f4a62 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-python3
- Requested version: >= 3.8
- Found cached script output: /home/val/CM/repos/local/cache/becbd4357f9f4838
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,python,python3,get-python,get-python3": 0.08 sec.
* Searching for collective script(s) with tags "get,dataset,object-detection,openimages,original" and variations "validation"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,dataset,object-detection,openimages,original
- Number of cached script outputs found: 0
- Found script::get-dataset-openimages,0a9d49b644cf4142 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-dataset-openimages
Prepared variations: _validation
- Creating new "cache" script artifact in the CM local repository ...
- Tags: tmp,get,dataset,object-detection,openimages,original,open-images,_validation,script-artifact-0a9d49b644cf4142
- Changing to /home/val/CM/repos/local/cache/46883742a2904037
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "get,sys-utils-cm"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,sys-utils-cm
- Number of cached script outputs found: 1
- Found script::get-sys-utils-cm,bc90993277e84b8e in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-sys-utils-cm
- Found cached script output: /home/val/CM/repos/local/cache/f6e03c4418764e25
- Checking dynamic dependencies on other CM scripts:
- Processing env after dependencies ...
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,sys-utils-cm": 0.10 sec.
* Searching for collective script(s) with tags "get,python3"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,python3
- Number of cached script outputs found: 2
- Found script::get-python3,d0b5dd74373f4a62 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-python3
- Requested version: >= 3.8
- Found cached script output: /home/val/CM/repos/local/cache/becbd4357f9f4838
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,python,python3,get-python,get-python3": 0.07 sec.
* Searching for collective script(s) with tags "mlperf,inference,source"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,mlperf,inference,source
- Number of cached script outputs found: 1
- Found script::get-mlperf-inference-src,4b57186581024797 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-mlperf-inference-src
Prepared variations: _default,_short-history
- Found cached script output: /home/val/CM/repos/local/cache/ce4ce58fe2e44ed5
- Checking dynamic dependencies on other CM scripts:
- Processing env after dependencies ...
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,src,source,inference,inference-src,inference-source,mlperf,mlcommons": 0.06 sec.
* Searching for collective script(s) with tags "get,generic-python-lib" and variations "boto3"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,generic-python-lib
- Number of cached script outputs found: 1
- Found script::get-generic-python-lib,94b62a682bc44791 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-generic-python-lib
Prepared variations: _boto3
- Creating new "cache" script artifact in the CM local repository ...
- Tags: tmp,get,generic-python-lib,generic,_boto3,deps-python-version-3.10.8,deps-python-virtual,deps-python-name-mlperf,script-artifact-94b62a682bc44791
- Changing to /home/val/CM/repos/local/cache/ff6b888fe6024611
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,cpu"
- Number of scripts found: 1
- Found script::detect-cpu,586c8a43320142f7 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-cpu
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,os"
- Number of scripts found: 1
- Found script::detect-os,863735b7db8c44fc in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-os
- Running preprocess ...
- running time of script "detect-os,detect,os,info": 0.10 sec.
- Processing env after dependencies ...
- Running preprocess ...
- running time of script "detect,cpu,detect-cpu,info": 0.19 sec.
* Searching for collective script(s) with tags "get,python3"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,python3
- Number of cached script outputs found: 2
- Found script::get-python3,d0b5dd74373f4a62 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-python3
- Requested version: >= 3.8
- Found cached script output: /home/val/CM/repos/local/cache/becbd4357f9f4838
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,python,python3,get-python,get-python3": 0.08 sec.
- Processing env after dependencies ...
- Running preprocess ...
- Checking prehook dependencies on other CM scripts:
* Searching for collective script(s) with tags "install,generic-python-lib" and variations "boto3"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,install,generic-python-lib
- Number of cached script outputs found: 1
- Found script::install-generic-python-lib,f4f502b7b5d545c8 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/install-generic-python-lib
Prepared variations: _boto3
- Creating new "cache" script artifact in the CM local repository ...
- Tags: tmp,install,generic-python-lib,generic,_boto3,deps-python-version-3.10.8,deps-python-virtual,deps-python-name-mlperf,script-artifact-f4f502b7b5d545c8
- Changing to /home/val/CM/repos/local/cache/f0cace7103ed47df
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,cpu"
- Number of scripts found: 1
- Found script::detect-cpu,586c8a43320142f7 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-cpu
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,os"
- Number of scripts found: 1
- Found script::detect-os,863735b7db8c44fc in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-os
- Running preprocess ...
- running time of script "detect-os,detect,os,info": 0.10 sec.
- Processing env after dependencies ...
- Running preprocess ...
- running time of script "detect,cpu,detect-cpu,info": 0.19 sec.
* Searching for collective script(s) with tags "get,python"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,python
- Number of cached script outputs found: 2
- Found script::get-python3,d0b5dd74373f4a62 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-python3
- Requested version: >= 3.8
- Found cached script output: /home/val/CM/repos/local/cache/becbd4357f9f4838
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,python,python3,get-python,get-python3": 0.08 sec.
- Processing env after dependencies ...
Collecting boto3
Using cached boto3-1.26.9-py3-none-any.whl (132 kB)
Collecting botocore<1.30.0,>=1.29.9
Using cached botocore-1.29.9-py3-none-any.whl (9.9 MB)
Collecting jmespath<2.0.0,>=0.7.1
Using cached jmespath-1.0.1-py3-none-any.whl (20 kB)
Collecting s3transfer<0.7.0,>=0.6.0
Using cached s3transfer-0.6.0-py3-none-any.whl (79 kB)
Collecting python-dateutil<3.0.0,>=2.1
Using cached python_dateutil-2.8.2-py2.py3-none-any.whl (247 kB)
Collecting urllib3<1.27,>=1.25.4
Using cached urllib3-1.26.12-py2.py3-none-any.whl (140 kB)
Collecting six>=1.5
Using cached six-1.16.0-py2.py3-none-any.whl (11 kB)
Installing collected packages: urllib3, six, jmespath, python-dateutil, botocore, s3transfer, boto3
Successfully installed boto3-1.26.9 botocore-1.29.9 jmespath-1.0.1 python-dateutil-2.8.2 s3transfer-0.6.0 six-1.16.0 urllib3-1.26.12
[notice] A new release of pip available: 22.2.2 -> 22.3.1
[notice] To update, run: pip install --upgrade pip
- Removing tmp tag in the script cached output f0cace7103ed47df ...
- running time of script "install,generic,generic-python-lib": 3.19 sec.
Detected version: 1.26.9
- Removing tmp tag in the script cached output ff6b888fe6024611 ...
- running time of script "get,generic,generic-python-lib": 3.82 sec.
* Searching for collective script(s) with tags "get,generic-python-lib" and variations "tqdm"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,generic-python-lib
- Number of cached script outputs found: 2
- Found script::get-generic-python-lib,94b62a682bc44791 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-generic-python-lib
Prepared variations: _tqdm
- Creating new "cache" script artifact in the CM local repository ...
- Tags: tmp,get,generic-python-lib,generic,_tqdm,deps-python-version-3.10.8,deps-python-virtual,deps-python-name-mlperf,script-artifact-94b62a682bc44791
- Changing to /home/val/CM/repos/local/cache/db91ca6eb98c42e5
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,cpu"
- Number of scripts found: 1
- Found script::detect-cpu,586c8a43320142f7 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-cpu
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,os"
- Number of scripts found: 1
- Found script::detect-os,863735b7db8c44fc in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-os
- Running preprocess ...
- running time of script "detect-os,detect,os,info": 0.09 sec.
- Processing env after dependencies ...
- Running preprocess ...
- running time of script "detect,cpu,detect-cpu,info": 0.19 sec.
* Searching for collective script(s) with tags "get,python3"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,python3
- Number of cached script outputs found: 2
- Found script::get-python3,d0b5dd74373f4a62 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-python3
- Requested version: >= 3.8
- Found cached script output: /home/val/CM/repos/local/cache/becbd4357f9f4838
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,python,python3,get-python,get-python3": 0.08 sec.
- Processing env after dependencies ...
- Running preprocess ...
- Checking prehook dependencies on other CM scripts:
* Searching for collective script(s) with tags "install,generic-python-lib" and variations "tqdm"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,install,generic-python-lib
- Number of cached script outputs found: 2
- Found script::install-generic-python-lib,f4f502b7b5d545c8 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/install-generic-python-lib
Prepared variations: _tqdm
- Creating new "cache" script artifact in the CM local repository ...
- Tags: tmp,install,generic-python-lib,generic,_tqdm,deps-python-version-3.10.8,deps-python-virtual,deps-python-name-mlperf,script-artifact-f4f502b7b5d545c8
- Changing to /home/val/CM/repos/local/cache/31e54495fe6148ed
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,cpu"
- Number of scripts found: 1
- Found script::detect-cpu,586c8a43320142f7 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-cpu
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,os"
- Number of scripts found: 1
- Found script::detect-os,863735b7db8c44fc in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-os
- Running preprocess ...
- running time of script "detect-os,detect,os,info": 0.09 sec.
- Processing env after dependencies ...
- Running preprocess ...
- running time of script "detect,cpu,detect-cpu,info": 0.19 sec.
* Searching for collective script(s) with tags "get,python"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,python
- Number of cached script outputs found: 2
- Found script::get-python3,d0b5dd74373f4a62 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-python3
- Requested version: >= 3.8
- Found cached script output: /home/val/CM/repos/local/cache/becbd4357f9f4838
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,python,python3,get-python,get-python3": 0.08 sec.
- Processing env after dependencies ...
Collecting tqdm
Using cached tqdm-4.64.1-py2.py3-none-any.whl (78 kB)
Installing collected packages: tqdm
Successfully installed tqdm-4.64.1
[notice] A new release of pip available: 22.2.2 -> 22.3.1
[notice] To update, run: pip install --upgrade pip
- Removing tmp tag in the script cached output 31e54495fe6148ed ...
- running time of script "install,generic,generic-python-lib": 1.11 sec.
Detected version: 4.64.1
- Removing tmp tag in the script cached output db91ca6eb98c42e5 ...
- running time of script "get,generic,generic-python-lib": 1.76 sec.
* Searching for collective script(s) with tags "get,generic-python-lib" and variations "numpy"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,generic-python-lib
- Number of cached script outputs found: 3
- Found script::get-generic-python-lib,94b62a682bc44791 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-generic-python-lib
Prepared variations: _numpy
- Creating new "cache" script artifact in the CM local repository ...
- Tags: tmp,get,generic-python-lib,generic,_numpy,deps-python-version-3.10.8,deps-python-virtual,deps-python-name-mlperf,script-artifact-94b62a682bc44791
- Changing to /home/val/CM/repos/local/cache/efd41b9f30d64c0a
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,cpu"
- Number of scripts found: 1
- Found script::detect-cpu,586c8a43320142f7 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-cpu
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,os"
- Number of scripts found: 1
- Found script::detect-os,863735b7db8c44fc in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-os
- Running preprocess ...
- running time of script "detect-os,detect,os,info": 0.10 sec.
- Processing env after dependencies ...
- Running preprocess ...
- running time of script "detect,cpu,detect-cpu,info": 0.19 sec.
* Searching for collective script(s) with tags "get,python3"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,python3
- Number of cached script outputs found: 2
- Found script::get-python3,d0b5dd74373f4a62 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-python3
- Requested version: >= 3.8
- Found cached script output: /home/val/CM/repos/local/cache/becbd4357f9f4838
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,python,python3,get-python,get-python3": 0.08 sec.
- Processing env after dependencies ...
- Running preprocess ...
Detected version: 1.23.4
- Checking prehook dependencies on other CM scripts:
Detected version: 1.23.4
- Removing tmp tag in the script cached output efd41b9f30d64c0a ...
- running time of script "get,generic,generic-python-lib": 0.63 sec.
* Searching for collective script(s) with tags "get,generic-python-lib" and variations "opencv-python"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,generic-python-lib
- Number of cached script outputs found: 4
- Found script::get-generic-python-lib,94b62a682bc44791 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-generic-python-lib
Prepared variations: _opencv-python
- Creating new "cache" script artifact in the CM local repository ...
- Tags: tmp,get,generic-python-lib,generic,_opencv-python,deps-python-version-3.10.8,deps-python-virtual,deps-python-name-mlperf,script-artifact-94b62a682bc44791
- Changing to /home/val/CM/repos/local/cache/bd266163e3a74259
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,cpu"
- Number of scripts found: 1
- Found script::detect-cpu,586c8a43320142f7 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-cpu
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,os"
- Number of scripts found: 1
- Found script::detect-os,863735b7db8c44fc in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-os
- Running preprocess ...
- running time of script "detect-os,detect,os,info": 0.10 sec.
- Processing env after dependencies ...
- Running preprocess ...
- running time of script "detect,cpu,detect-cpu,info": 0.19 sec.
* Searching for collective script(s) with tags "get,python3"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,python3
- Number of cached script outputs found: 2
- Found script::get-python3,d0b5dd74373f4a62 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-python3
- Requested version: >= 3.8
- Found cached script output: /home/val/CM/repos/local/cache/becbd4357f9f4838
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,python,python3,get-python,get-python3": 0.09 sec.
- Processing env after dependencies ...
- Running preprocess ...
- Checking prehook dependencies on other CM scripts:
* Searching for collective script(s) with tags "install,generic-python-lib" and variations "opencv-python"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,install,generic-python-lib
- Number of cached script outputs found: 3
- Found script::install-generic-python-lib,f4f502b7b5d545c8 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/install-generic-python-lib
Prepared variations: _opencv-python
- Creating new "cache" script artifact in the CM local repository ...
- Tags: tmp,install,generic-python-lib,generic,_opencv-python,deps-python-version-3.10.8,deps-python-virtual,deps-python-name-mlperf,script-artifact-f4f502b7b5d545c8
- Changing to /home/val/CM/repos/local/cache/b509c6ff9ac4468f
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,cpu"
- Number of scripts found: 1
- Found script::detect-cpu,586c8a43320142f7 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-cpu
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,os"
- Number of scripts found: 1
- Found script::detect-os,863735b7db8c44fc in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-os
- Running preprocess ...
- running time of script "detect-os,detect,os,info": 0.09 sec.
- Processing env after dependencies ...
- Running preprocess ...
- running time of script "detect,cpu,detect-cpu,info": 0.18 sec.
* Searching for collective script(s) with tags "get,python"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,python
- Number of cached script outputs found: 2
- Found script::get-python3,d0b5dd74373f4a62 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-python3
- Requested version: >= 3.8
- Found cached script output: /home/val/CM/repos/local/cache/becbd4357f9f4838
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,python,python3,get-python,get-python3": 0.07 sec.
- Processing env after dependencies ...
Collecting opencv-python
Using cached opencv_python-4.6.0.66-cp36-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (60.9 MB)
Requirement already satisfied: numpy>=1.19.3 in /home/val/CM/repos/local/cache/ad69a6d50fc2461e/venv/lib/python3.10/site-packages (from opencv-python) (1.23.4)
Installing collected packages: opencv-python
Successfully installed opencv-python-4.6.0.66
[notice] A new release of pip available: 22.2.2 -> 22.3.1
[notice] To update, run: pip install --upgrade pip
- Removing tmp tag in the script cached output b509c6ff9ac4468f ...
- running time of script "install,generic,generic-python-lib": 2.22 sec.
Detected version: 4.6.0.66
- Removing tmp tag in the script cached output bd266163e3a74259 ...
- running time of script "get,generic,generic-python-lib": 2.86 sec.
* Searching for collective script(s) with tags "get,generic-python-lib" and variations "pandas"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,generic-python-lib
- Number of cached script outputs found: 5
- Found script::get-generic-python-lib,94b62a682bc44791 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-generic-python-lib
Prepared variations: _pandas
- Creating new "cache" script artifact in the CM local repository ...
- Tags: tmp,get,generic-python-lib,generic,_pandas,deps-python-version-3.10.8,deps-python-virtual,deps-python-name-mlperf,script-artifact-94b62a682bc44791
- Changing to /home/val/CM/repos/local/cache/a50f88a9b6f74e96
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,cpu"
- Number of scripts found: 1
- Found script::detect-cpu,586c8a43320142f7 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-cpu
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,os"
- Number of scripts found: 1
- Found script::detect-os,863735b7db8c44fc in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-os
- Running preprocess ...
- running time of script "detect-os,detect,os,info": 0.10 sec.
- Processing env after dependencies ...
- Running preprocess ...
- running time of script "detect,cpu,detect-cpu,info": 0.20 sec.
* Searching for collective script(s) with tags "get,python3"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,python3
- Number of cached script outputs found: 2
- Found script::get-python3,d0b5dd74373f4a62 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-python3
- Requested version: >= 3.8
- Found cached script output: /home/val/CM/repos/local/cache/becbd4357f9f4838
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,python,python3,get-python,get-python3": 0.08 sec.
- Processing env after dependencies ...
- Running preprocess ...
- Checking prehook dependencies on other CM scripts:
* Searching for collective script(s) with tags "install,generic-python-lib" and variations "pandas"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,install,generic-python-lib
- Number of cached script outputs found: 4
- Found script::install-generic-python-lib,f4f502b7b5d545c8 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/install-generic-python-lib
Prepared variations: _pandas
- Creating new "cache" script artifact in the CM local repository ...
- Tags: tmp,install,generic-python-lib,generic,_pandas,deps-python-version-3.10.8,deps-python-virtual,deps-python-name-mlperf,script-artifact-f4f502b7b5d545c8
- Changing to /home/val/CM/repos/local/cache/3f98a9a321b742c5
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,cpu"
- Number of scripts found: 1
- Found script::detect-cpu,586c8a43320142f7 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-cpu
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,os"
- Number of scripts found: 1
- Found script::detect-os,863735b7db8c44fc in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-os
- Running preprocess ...
- running time of script "detect-os,detect,os,info": 0.09 sec.
- Processing env after dependencies ...
- Running preprocess ...
- running time of script "detect,cpu,detect-cpu,info": 0.19 sec.
* Searching for collective script(s) with tags "get,python"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,python
- Number of cached script outputs found: 2
- Found script::get-python3,d0b5dd74373f4a62 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-python3
- Requested version: >= 3.8
- Found cached script output: /home/val/CM/repos/local/cache/becbd4357f9f4838
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,python,python3,get-python,get-python3": 0.08 sec.
- Processing env after dependencies ...
Collecting pandas
Using cached pandas-1.5.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (12.1 MB)
Requirement already satisfied: numpy>=1.21.0 in /home/val/CM/repos/local/cache/ad69a6d50fc2461e/venv/lib/python3.10/site-packages (from pandas) (1.23.4)
Collecting pytz>=2020.1
Using cached pytz-2022.6-py2.py3-none-any.whl (498 kB)
Requirement already satisfied: python-dateutil>=2.8.1 in /home/val/CM/repos/local/cache/ad69a6d50fc2461e/venv/lib/python3.10/site-packages (from pandas) (2.8.2)
Requirement already satisfied: six>=1.5 in /home/val/CM/repos/local/cache/ad69a6d50fc2461e/venv/lib/python3.10/site-packages (from python-dateutil>=2.8.1->pandas) (1.16.0)
Installing collected packages: pytz, pandas
Successfully installed pandas-1.5.1 pytz-2022.6
[notice] A new release of pip available: 22.2.2 -> 22.3.1
[notice] To update, run: pip install --upgrade pip
- Removing tmp tag in the script cached output 3f98a9a321b742c5 ...
- running time of script "install,generic,generic-python-lib": 4.09 sec.
Detected version: 1.5.1
- Removing tmp tag in the script cached output a50f88a9b6f74e96 ...
- running time of script "get,generic,generic-python-lib": 4.74 sec.
- Processing env after dependencies ...
Collecting requests
Using cached requests-2.28.1-py3-none-any.whl (62 kB)
Collecting charset-normalizer<3,>=2
Using cached charset_normalizer-2.1.1-py3-none-any.whl (39 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2022.9.24-py3-none-any.whl (161 kB)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/val/CM/repos/local/cache/ad69a6d50fc2461e/venv/lib/python3.10/site-packages (from requests->-r /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-dataset-openimages/requirements.txt (line 1)) (1.26.12)
Collecting idna<4,>=2.5
Using cached idna-3.4-py3-none-any.whl (61 kB)
Installing collected packages: idna, charset-normalizer, certifi, requests
Successfully installed certifi-2022.9.24 charset-normalizer-2.1.1 idna-3.4 requests-2.28.1
[notice] A new release of pip available: 22.2.2 -> 22.3.1
[notice] To update, run: pip install --upgrade pip
./openimages_mlperf.sh -d /home/val/CM/repos/local/cache/46883742a2904037/install -m 500
Downloading images: 100%|███████████████████████████████████████████████████████| 500/500 [00:17<00:00, 27.98it/s]
Converting annotations to COCO format...
- Removing tmp tag in the script cached output 46883742a2904037 ...
- running time of script "get,dataset,openimages,open-images,object-detection,original": 77.10 sec.
* Searching for collective script(s) with tags "mlperf,mlcommons,inference,source,src"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,mlperf,mlcommons,inference,source,src
- Number of cached script outputs found: 1
- Found script::get-mlperf-inference-src,4b57186581024797 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-mlperf-inference-src
Prepared variations: _default,_short-history
- Found cached script output: /home/val/CM/repos/local/cache/ce4ce58fe2e44ed5
- Checking dynamic dependencies on other CM scripts:
- Processing env after dependencies ...
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,src,source,inference,inference-src,inference-source,mlperf,mlcommons": 0.07 sec.
* Searching for collective script(s) with tags "get,generic-python-lib" and variations "pycocotools"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,generic-python-lib
- Number of cached script outputs found: 6
- Found script::get-generic-python-lib,94b62a682bc44791 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-generic-python-lib
Prepared variations: _pycocotools
- Creating new "cache" script artifact in the CM local repository ...
- Tags: tmp,get,generic-python-lib,generic,_pycocotools,deps-python-version-3.10.8,deps-python-virtual,deps-python-name-mlperf,script-artifact-94b62a682bc44791
- Changing to /home/val/CM/repos/local/cache/80fb4233795e4444
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,cpu"
- Number of scripts found: 1
- Found script::detect-cpu,586c8a43320142f7 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-cpu
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,os"
- Number of scripts found: 1
- Found script::detect-os,863735b7db8c44fc in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-os
- Running preprocess ...
- running time of script "detect-os,detect,os,info": 0.10 sec.
- Processing env after dependencies ...
- Running preprocess ...
- running time of script "detect,cpu,detect-cpu,info": 0.20 sec.
* Searching for collective script(s) with tags "get,python3"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,python3
- Number of cached script outputs found: 2
- Found script::get-python3,d0b5dd74373f4a62 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-python3
- Requested version: >= 3.8
- Found cached script output: /home/val/CM/repos/local/cache/becbd4357f9f4838
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,python,python3,get-python,get-python3": 0.08 sec.
- Processing env after dependencies ...
- Running preprocess ...
- Checking prehook dependencies on other CM scripts:
* Searching for collective script(s) with tags "install,generic-python-lib" and variations "pycocotools"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,install,generic-python-lib
- Number of cached script outputs found: 5
- Found script::install-generic-python-lib,f4f502b7b5d545c8 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/install-generic-python-lib
Prepared variations: _pycocotools
- Creating new "cache" script artifact in the CM local repository ...
- Tags: tmp,install,generic-python-lib,generic,_pycocotools,deps-python-version-3.10.8,deps-python-virtual,deps-python-name-mlperf,script-artifact-f4f502b7b5d545c8
- Changing to /home/val/CM/repos/local/cache/e48c5ef79b644811
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,cpu"
- Number of scripts found: 1
- Found script::detect-cpu,586c8a43320142f7 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-cpu
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,os"
- Number of scripts found: 1
- Found script::detect-os,863735b7db8c44fc in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-os
- Running preprocess ...
- running time of script "detect-os,detect,os,info": 0.09 sec.
- Processing env after dependencies ...
- Running preprocess ...
- running time of script "detect,cpu,detect-cpu,info": 0.19 sec.
* Searching for collective script(s) with tags "get,python"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,python
- Number of cached script outputs found: 2
- Found script::get-python3,d0b5dd74373f4a62 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-python3
- Requested version: >= 3.8
- Found cached script output: /home/val/CM/repos/local/cache/becbd4357f9f4838
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,python,python3,get-python,get-python3": 0.08 sec.
- Processing env after dependencies ...
Collecting pycocotools
Using cached pycocotools-2.0.6-cp310-cp310-linux_x86_64.whl
Collecting matplotlib>=2.1.0
Using cached matplotlib-3.6.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.8 MB)
Requirement already satisfied: numpy in /home/val/CM/repos/local/cache/ad69a6d50fc2461e/venv/lib/python3.10/site-packages (from pycocotools) (1.23.4)
Requirement already satisfied: pyparsing>=2.2.1 in /home/val/CM/repos/local/cache/ad69a6d50fc2461e/venv/lib/python3.10/site-packages (from matplotlib>=2.1.0->pycocotools) (3.0.9)
Collecting contourpy>=1.0.1
Using cached contourpy-1.0.6-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (296 kB)
Requirement already satisfied: python-dateutil>=2.7 in /home/val/CM/repos/local/cache/ad69a6d50fc2461e/venv/lib/python3.10/site-packages (from matplotlib>=2.1.0->pycocotools) (2.8.2)
Requirement already satisfied: packaging>=20.0 in /home/val/CM/repos/local/cache/ad69a6d50fc2461e/venv/lib/python3.10/site-packages (from matplotlib>=2.1.0->pycocotools) (21.3)
Collecting pillow>=6.2.0
Using cached Pillow-9.3.0-cp310-cp310-manylinux_2_28_x86_64.whl (3.3 MB)
Collecting kiwisolver>=1.0.1
Using cached kiwisolver-1.4.4-cp310-cp310-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (1.6 MB)
Collecting fonttools>=4.22.0
Using cached fonttools-4.38.0-py3-none-any.whl (965 kB)
Collecting cycler>=0.10
Using cached cycler-0.11.0-py3-none-any.whl (6.4 kB)
Requirement already satisfied: six>=1.5 in /home/val/CM/repos/local/cache/ad69a6d50fc2461e/venv/lib/python3.10/site-packages (from python-dateutil>=2.7->matplotlib>=2.1.0->pycocotools) (1.16.0)
Installing collected packages: pillow, kiwisolver, fonttools, cycler, contourpy, matplotlib, pycocotools
Successfully installed contourpy-1.0.6 cycler-0.11.0 fonttools-4.38.0 kiwisolver-1.4.4 matplotlib-3.6.2 pillow-9.3.0 pycocotools-2.0.6
[notice] A new release of pip available: 22.2.2 -> 22.3.1
[notice] To update, run: pip install --upgrade pip
- Removing tmp tag in the script cached output e48c5ef79b644811 ...
- running time of script "install,generic,generic-python-lib": 5.67 sec.
Detected version: 2.0.6
- Removing tmp tag in the script cached output 80fb4233795e4444 ...
- running time of script "get,generic,generic-python-lib": 6.34 sec.
- Processing env after dependencies ...
- Running preprocess ...
Using MLCommons Inference source from '/home/val/CM/repos/local/cache/ce4ce58fe2e44ed5/inference'
Requirement already satisfied: opencv-python in /home/val/CM/repos/local/cache/ad69a6d50fc2461e/venv/lib/python3.10/site-packages (from -r /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-preprocessed-dataset-openimages/requirements.txt (line 1)) (4.6.0.66)
Requirement already satisfied: Pillow in /home/val/CM/repos/local/cache/ad69a6d50fc2461e/venv/lib/python3.10/site-packages (from -r /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-preprocessed-dataset-openimages/requirements.txt (line 2)) (9.3.0)
Requirement already satisfied: numpy>=1.14.5 in /home/val/CM/repos/local/cache/ad69a6d50fc2461e/venv/lib/python3.10/site-packages (from opencv-python->-r /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-preprocessed-dataset-openimages/requirements.txt (line 1)) (1.23.4)
[notice] A new release of pip available: 22.2.2 -> 22.3.1
[notice] To update, run: pip install --upgrade pip
INFO:coco:loaded 495 images, cache=True, took=31.8sec
/home/val/CM/repos/local/cache/ce4ce58fe2e44ed5/inference/vision/classification_and_detection/python/openimages.py:109: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray.
self.label_list = np.array(self.label_list)
- Removing tmp tag in the script cached output 8ae2ae9d3c984709 ...
- running time of script "get,dataset,openimages,open-images,object-detection,preprocessed": 116.33 sec.
* Searching for collective script(s) with tags "get,generic-python-lib" and variations "pycocotools"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,generic-python-lib
- Number of cached script outputs found: 7
- Found script::get-generic-python-lib,94b62a682bc44791 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-generic-python-lib
Prepared variations: _pycocotools
- Found cached script output: /home/val/CM/repos/local/cache/80fb4233795e4444
- Checking dynamic dependencies on other CM scripts:
- Processing env after dependencies ...
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,generic,generic-python-lib": 0.08 sec.
* Searching for collective script(s) with tags "get,ml-model,object-detection,resnext50,fp32" and variations "onnx"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,ml-model,object-detection,resnext50,fp32
- Number of cached script outputs found: 0
- Found script::get-ml-model-retinanet,427bc5665e4541c2 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-ml-model-retinanet
Prepared variations: _onnx
- Creating new "cache" script artifact in the CM local repository ...
- Tags: tmp,get,ml-model,object-detection,resnext50,fp32,retinanet,_onnx,script-artifact-427bc5665e4541c2
- Changing to /home/val/CM/repos/local/cache/4a2de0083ace466f
- Running preprocess ...
Downloading from https://zenodo.org/record/6617879/files/resnext50_32x4d_fpn.onnx
Downloaded: 100%
- Removing tmp tag in the script cached output 4a2de0083ace466f ...
- running time of script "get,ml-model,resnext50,retinanet,object-detection,fp32": 9.76 sec.
- Processing env after dependencies ...
- Running preprocess ...
Using MLCommons Inference source from '/home/val/CM/repos/local/cache/ce4ce58fe2e44ed5/inference'
Output Dir: '/home/val/CM/repos/local/cache/ce4ce58fe2e44ed5/inference/vision/classification_and_detection/test_results/onnxruntime-cpu/retinanet/offline/performance/run_1'
retinanet.Offline.target_qps = 1
retinanet.Offline.max_query_count = 10
retinanet.Offline.min_query_count = 10
Requirement already satisfied: pycocotools in /home/val/CM/repos/local/cache/ad69a6d50fc2461e/venv/lib/python3.10/site-packages (from -r /home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference/requirements.txt (line 1)) (2.0.6)
Requirement already satisfied: numpy in /home/val/CM/repos/local/cache/ad69a6d50fc2461e/venv/lib/python3.10/site-packages (from -r /home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference/requirements.txt (line 2)) (1.23.4)
Requirement already satisfied: opencv-python in /home/val/CM/repos/local/cache/ad69a6d50fc2461e/venv/lib/python3.10/site-packages (from -r /home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference/requirements.txt (line 3)) (4.6.0.66)
Requirement already satisfied: protobuf in /home/val/CM/repos/local/cache/ad69a6d50fc2461e/venv/lib/python3.10/site-packages (from -r /home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference/requirements.txt (line 4)) (4.21.9)
Collecting future
Using cached future-0.18.2-py3-none-any.whl
Requirement already satisfied: matplotlib>=2.1.0 in /home/val/CM/repos/local/cache/ad69a6d50fc2461e/venv/lib/python3.10/site-packages (from pycocotools->-r /home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference/requirements.txt (line 1)) (3.6.2)
Requirement already satisfied: python-dateutil>=2.7 in /home/val/CM/repos/local/cache/ad69a6d50fc2461e/venv/lib/python3.10/site-packages (from matplotlib>=2.1.0->pycocotools->-r /home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference/requirements.txt (line 1)) (2.8.2)
Requirement already satisfied: kiwisolver>=1.0.1 in /home/val/CM/repos/local/cache/ad69a6d50fc2461e/venv/lib/python3.10/site-packages (from matplotlib>=2.1.0->pycocotools->-r /home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference/requirements.txt (line 1)) (1.4.4)
Requirement already satisfied: cycler>=0.10 in /home/val/CM/repos/local/cache/ad69a6d50fc2461e/venv/lib/python3.10/site-packages (from matplotlib>=2.1.0->pycocotools->-r /home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference/requirements.txt (line 1)) (0.11.0)
Requirement already satisfied: pillow>=6.2.0 in /home/val/CM/repos/local/cache/ad69a6d50fc2461e/venv/lib/python3.10/site-packages (from matplotlib>=2.1.0->pycocotools->-r /home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference/requirements.txt (line 1)) (9.3.0)
Requirement already satisfied: pyparsing>=2.2.1 in /home/val/CM/repos/local/cache/ad69a6d50fc2461e/venv/lib/python3.10/site-packages (from matplotlib>=2.1.0->pycocotools->-r /home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference/requirements.txt (line 1)) (3.0.9)
Requirement already satisfied: fonttools>=4.22.0 in /home/val/CM/repos/local/cache/ad69a6d50fc2461e/venv/lib/python3.10/site-packages (from matplotlib>=2.1.0->pycocotools->-r /home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference/requirements.txt (line 1)) (4.38.0)
Requirement already satisfied: packaging>=20.0 in /home/val/CM/repos/local/cache/ad69a6d50fc2461e/venv/lib/python3.10/site-packages (from matplotlib>=2.1.0->pycocotools->-r /home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference/requirements.txt (line 1)) (21.3)
Requirement already satisfied: contourpy>=1.0.1 in /home/val/CM/repos/local/cache/ad69a6d50fc2461e/venv/lib/python3.10/site-packages (from matplotlib>=2.1.0->pycocotools->-r /home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference/requirements.txt (line 1)) (1.0.6)
Requirement already satisfied: six>=1.5 in /home/val/CM/repos/local/cache/ad69a6d50fc2461e/venv/lib/python3.10/site-packages (from python-dateutil>=2.7->matplotlib>=2.1.0->pycocotools->-r /home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference/requirements.txt (line 1)) (1.16.0)
Installing collected packages: future
Successfully installed future-0.18.2
[notice] A new release of pip available: 22.2.2 -> 22.3.1
[notice] To update, run: pip install --upgrade pip
cd '/home/val/CM/repos/local/cache/ce4ce58fe2e44ed5/inference/vision/classification_and_detection' && OUTPUT_DIR='/home/val/CM/repos/local/cache/ce4ce58fe2e44ed5/inference/vision/classification_and_detection/test_results/onnxruntime-cpu/retinanet/offline/performance/run_1' ./run_local.sh onnxruntime retinanet cpu --scenario Offline --mlperf_conf '/home/val/CM/repos/local/cache/ce4ce58fe2e44ed5/inference/mlperf.conf' --threads 8 --count 10 --user_conf '/home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference/tmp/8dc7d738e5844e02b9b2c29d30459937.conf' --cache_dir /home/val/CM/repos/local/cache/8ae2ae9d3c984709
python3 python/main.py --profile retinanet-onnxruntime --mlperf_conf ../../mlperf.conf --model "/home/val/CM/repos/local/cache/4a2de0083ace466f/resnext50_32x4d_fpn.onnx" --dataset-path /home/val/CM/repos/local/cache/46883742a2904037/install --output "/home/val/CM/repos/local/cache/ce4ce58fe2e44ed5/inference/vision/classification_and_detection/test_results/onnxruntime-cpu/retinanet/offline/performance/run_1" --scenario Offline --mlperf_conf /home/val/CM/repos/local/cache/ce4ce58fe2e44ed5/inference/mlperf.conf --threads 8 --count 10 --user_conf /home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference/tmp/8dc7d738e5844e02b9b2c29d30459937.conf --cache_dir /home/val/CM/repos/local/cache/8ae2ae9d3c984709
INFO:main:Namespace(dataset='openimages-800-retinanet-onnx', dataset_path='/home/val/CM/repos/local/cache/46883742a2904037/install', dataset_list=None, data_format=None, profile='retinanet-onnxruntime', scenario='Offline', max_batchsize=1, model='/home/val/CM/repos/local/cache/4a2de0083ace466f/resnext50_32x4d_fpn.onnx', output='/home/val/CM/repos/local/cache/ce4ce58fe2e44ed5/inference/vision/classification_and_detection/test_results/onnxruntime-cpu/retinanet/offline/performance/run_1', inputs=['images'], outputs=['boxes', 'labels', 'scores'], backend='onnxruntime', model_name='retinanet', threads=8, qps=None, cache=0, cache_dir='/home/val/CM/repos/local/cache/8ae2ae9d3c984709', accuracy=False, find_peak_performance=False, debug=False, mlperf_conf='/home/val/CM/repos/local/cache/ce4ce58fe2e44ed5/inference/mlperf.conf', user_conf='/home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference/tmp/8dc7d738e5844e02b9b2c29d30459937.conf', audit_conf='audit.config', time=None, count=10, performance_sample_count=None, max_latency=None, samples_per_query=8)
INFO:coco:loaded 10 images, cache=0, took=0.0sec
/home/val/CM/repos/local/cache/ce4ce58fe2e44ed5/inference/vision/classification_and_detection/python/openimages.py:109: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray.
self.label_list = np.array(self.label_list)
INFO:main:starting TestScenario.Offline
TestScenario.Offline qps=0.62, mean=64.9124, time=1072.671, queries=660, tiles=50.0:66.4727,80.0:67.5883,90.0:68.2680,95.0:68.9376,99.0:70.8438,99.9:73.4441
================================================
MLPerf Results Summary
================================================
SUT name : PySUT
Scenario : Offline
Mode : PerformanceOnly
Samples per second: 0.615298
Result is : VALID
Min duration satisfied : Yes
Min queries satisfied : Yes
Early stopping satisfied: Yes
================================================
Additional Stats
================================================
Min latency (ns) : 12138914384
Max latency (ns) : 1072650486931
Mean latency (ns) : 542711421726
50.00 percentile latency (ns) : 542865112566
90.00 percentile latency (ns) : 971675872127
95.00 percentile latency (ns) : 1027907247230
97.00 percentile latency (ns) : 1046186413545
99.00 percentile latency (ns) : 1070121324577
99.90 percentile latency (ns) : 1072650486931
================================================
Test Parameters Used
================================================
samples_per_query : 660
target_qps : 1
target_latency (ns): 0
max_async_queries : 1
min_duration (ms): 600000
max_duration (ms): 0
min_query_count : 1
max_query_count : 10
qsl_rng_seed : 14284205019438841327
sample_index_rng_seed : 4163916728725999944
schedule_rng_seed : 299063814864929621
accuracy_log_rng_seed : 0
accuracy_log_probability : 0
accuracy_log_sampling_target : 0
print_timestamps : 0
performance_issue_unique : 0
performance_issue_same : 0
performance_issue_same_index : 0
performance_sample_count : 64
No warnings encountered during test.
No errors encountered during test.
- running time of script "app,vision,language,mlcommons,mlperf,inference,reference,generic,ref": 1291.33 sec.
* Searching for collective script(s) with tags "app,mlperf,inference,generic" and variations "python,retinanet,onnxruntime,cpu,test"
- Number of scripts found: 1
- Found script::app-mlperf-inference,d775cac873ee4231 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference
Prepared variations: _python,_retinanet,_onnxruntime,_cpu,_test
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,os"
- Number of scripts found: 1
- Found script::detect-os,863735b7db8c44fc in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-os
- Running preprocess ...
- running time of script "detect-os,detect,os,info": 0.10 sec.
* Searching for collective script(s) with tags "detect,cpu"
- Number of scripts found: 1
- Found script::detect-cpu,586c8a43320142f7 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-cpu
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,os"
- Number of scripts found: 1
- Found script::detect-os,863735b7db8c44fc in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-os
- Running preprocess ...
- running time of script "detect-os,detect,os,info": 0.09 sec.
- Processing env after dependencies ...
- Running preprocess ...
- running time of script "detect,cpu,detect-cpu,info": 0.18 sec.
* Searching for collective script(s) with tags "get,sys-utils-cm"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,sys-utils-cm
- Number of cached script outputs found: 1
- Found script::get-sys-utils-cm,bc90993277e84b8e in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-sys-utils-cm
- Found cached script output: /home/val/CM/repos/local/cache/f6e03c4418764e25
- Checking dynamic dependencies on other CM scripts:
- Processing env after dependencies ...
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,sys-utils-cm": 0.10 sec.
* Searching for collective script(s) with tags "get,python"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,python
- Number of cached script outputs found: 2
- Found script::get-python3,d0b5dd74373f4a62 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-python3
- Requested version: >= 3.8
- Found cached script output: /home/val/CM/repos/local/cache/becbd4357f9f4838
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,python,python3,get-python,get-python3": 0.09 sec.
* Searching for collective script(s) with tags "get,generic-python-lib" and variations "onnxruntime"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,generic-python-lib
- Number of cached script outputs found: 7
- Found script::get-generic-python-lib,94b62a682bc44791 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-generic-python-lib
Prepared variations: _onnxruntime
- Found cached script output: /home/val/CM/repos/local/cache/6ddcf39f09084b49
- Checking dynamic dependencies on other CM scripts:
- Processing env after dependencies ...
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,generic,generic-python-lib": 0.08 sec.
* Searching for collective script(s) with tags "get,loadgen"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,loadgen
- Number of cached script outputs found: 1
- Found script::get-mlperf-inference-loadgen,64c3d98d0ba04950 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-mlperf-inference-loadgen
- Found cached script output: /home/val/CM/repos/local/cache/d4e616c7c2014060
- Checking dynamic dependencies on other CM scripts:
- Processing env after dependencies ...
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,loadgen,inference,inference-loadgen,mlperf,mlcommons": 0.11 sec.
* Searching for collective script(s) with tags "get,mlcommons,inference,src"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,mlcommons,inference,src
- Number of cached script outputs found: 1
- Found script::get-mlperf-inference-src,4b57186581024797 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-mlperf-inference-src
Prepared variations: _default,_short-history
- Found cached script output: /home/val/CM/repos/local/cache/ce4ce58fe2e44ed5
- Checking dynamic dependencies on other CM scripts:
- Processing env after dependencies ...
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,src,source,inference,inference-src,inference-source,mlperf,mlcommons": 0.07 sec.
* Searching for collective script(s) with tags "get,sut,configs"
- Number of scripts found: 1
- Found script::get-mlperf-inference-sut-configs,c2fbf72009e2445b in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-mlperf-inference-sut-configs
- Running preprocess ...
- running time of script "get,mlperf,sut,configs,sut-configs": 0.07 sec.
* Searching for collective script(s) with tags "get,dataset,object-detection,open-images,openimages,preprocessed" and variations "validation,NCHW,500"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,dataset,object-detection,open-images,openimages,preprocessed
- Number of cached script outputs found: 1
- Found script::get-preprocessed-dataset-openimages,9842f1be8cba4c7b in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-preprocessed-dataset-openimages
Prepared variations: _validation,_NCHW,_500
- Found cached script output: /home/val/CM/repos/local/cache/8ae2ae9d3c984709
- Checking dynamic dependencies on other CM scripts:
- Processing env after dependencies ...
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,dataset,openimages,open-images,object-detection,preprocessed": 0.08 sec.
* Searching for collective script(s) with tags "get,generic-python-lib" and variations "pycocotools"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,generic-python-lib
- Number of cached script outputs found: 7
- Found script::get-generic-python-lib,94b62a682bc44791 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-generic-python-lib
Prepared variations: _pycocotools
- Found cached script output: /home/val/CM/repos/local/cache/80fb4233795e4444
- Checking dynamic dependencies on other CM scripts:
- Processing env after dependencies ...
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,generic,generic-python-lib": 0.08 sec.
* Searching for collective script(s) with tags "get,ml-model,object-detection,resnext50,fp32" and variations "onnx"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,ml-model,object-detection,resnext50,fp32
- Number of cached script outputs found: 1
- Found script::get-ml-model-retinanet,427bc5665e4541c2 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-ml-model-retinanet
Prepared variations: _onnx
- Found cached script output: /home/val/CM/repos/local/cache/4a2de0083ace466f
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,ml-model,resnext50,retinanet,object-detection,fp32": 0.07 sec.
- Processing env after dependencies ...
- Running preprocess ...
Using MLCommons Inference source from '/home/val/CM/repos/local/cache/ce4ce58fe2e44ed5/inference'
Output Dir: '/home/val/CM/repos/local/cache/ce4ce58fe2e44ed5/inference/vision/classification_and_detection/test_results/onnxruntime-cpu/retinanet/offline/accuracy'
retinanet.Offline.target_qps = 1
retinanet.Offline.max_query_count = 10
retinanet.Offline.min_query_count = 10
Requirement already satisfied: pycocotools in /home/val/CM/repos/local/cache/ad69a6d50fc2461e/venv/lib/python3.10/site-packages (from -r /home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference/requirements.txt (line 1)) (2.0.6)
Requirement already satisfied: numpy in /home/val/CM/repos/local/cache/ad69a6d50fc2461e/venv/lib/python3.10/site-packages (from -r /home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference/requirements.txt (line 2)) (1.23.4)
Requirement already satisfied: opencv-python in /home/val/CM/repos/local/cache/ad69a6d50fc2461e/venv/lib/python3.10/site-packages (from -r /home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference/requirements.txt (line 3)) (4.6.0.66)
Requirement already satisfied: protobuf in /home/val/CM/repos/local/cache/ad69a6d50fc2461e/venv/lib/python3.10/site-packages (from -r /home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference/requirements.txt (line 4)) (4.21.9)
Requirement already satisfied: future in /home/val/CM/repos/local/cache/ad69a6d50fc2461e/venv/lib/python3.10/site-packages (from -r /home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference/requirements.txt (line 5)) (0.18.2)
Requirement already satisfied: matplotlib>=2.1.0 in /home/val/CM/repos/local/cache/ad69a6d50fc2461e/venv/lib/python3.10/site-packages (from pycocotools->-r /home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference/requirements.txt (line 1)) (3.6.2)
Requirement already satisfied: cycler>=0.10 in /home/val/CM/repos/local/cache/ad69a6d50fc2461e/venv/lib/python3.10/site-packages (from matplotlib>=2.1.0->pycocotools->-r /home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference/requirements.txt (line 1)) (0.11.0)
Requirement already satisfied: kiwisolver>=1.0.1 in /home/val/CM/repos/local/cache/ad69a6d50fc2461e/venv/lib/python3.10/site-packages (from matplotlib>=2.1.0->pycocotools->-r /home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference/requirements.txt (line 1)) (1.4.4)
Requirement already satisfied: python-dateutil>=2.7 in /home/val/CM/repos/local/cache/ad69a6d50fc2461e/venv/lib/python3.10/site-packages (from matplotlib>=2.1.0->pycocotools->-r /home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference/requirements.txt (line 1)) (2.8.2)
Requirement already satisfied: fonttools>=4.22.0 in /home/val/CM/repos/local/cache/ad69a6d50fc2461e/venv/lib/python3.10/site-packages (from matplotlib>=2.1.0->pycocotools->-r /home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference/requirements.txt (line 1)) (4.38.0)
Requirement already satisfied: pyparsing>=2.2.1 in /home/val/CM/repos/local/cache/ad69a6d50fc2461e/venv/lib/python3.10/site-packages (from matplotlib>=2.1.0->pycocotools->-r /home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference/requirements.txt (line 1)) (3.0.9)
Requirement already satisfied: packaging>=20.0 in /home/val/CM/repos/local/cache/ad69a6d50fc2461e/venv/lib/python3.10/site-packages (from matplotlib>=2.1.0->pycocotools->-r /home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference/requirements.txt (line 1)) (21.3)
Requirement already satisfied: pillow>=6.2.0 in /home/val/CM/repos/local/cache/ad69a6d50fc2461e/venv/lib/python3.10/site-packages (from matplotlib>=2.1.0->pycocotools->-r /home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference/requirements.txt (line 1)) (9.3.0)
Requirement already satisfied: contourpy>=1.0.1 in /home/val/CM/repos/local/cache/ad69a6d50fc2461e/venv/lib/python3.10/site-packages (from matplotlib>=2.1.0->pycocotools->-r /home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference/requirements.txt (line 1)) (1.0.6)
Requirement already satisfied: six>=1.5 in /home/val/CM/repos/local/cache/ad69a6d50fc2461e/venv/lib/python3.10/site-packages (from python-dateutil>=2.7->matplotlib>=2.1.0->pycocotools->-r /home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference/requirements.txt (line 1)) (1.16.0)
[notice] A new release of pip available: 22.2.2 -> 22.3.1
[notice] To update, run: pip install --upgrade pip
cd '/home/val/CM/repos/local/cache/ce4ce58fe2e44ed5/inference/vision/classification_and_detection' && OUTPUT_DIR='/home/val/CM/repos/local/cache/ce4ce58fe2e44ed5/inference/vision/classification_and_detection/test_results/onnxruntime-cpu/retinanet/offline/accuracy' ./run_local.sh onnxruntime retinanet cpu --scenario Offline --mlperf_conf '/home/val/CM/repos/local/cache/ce4ce58fe2e44ed5/inference/mlperf.conf' --threads 8 --count 10 --user_conf '/home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference/tmp/1e6c18c29ec142cd9f59c2e757822785.conf' --accuracy --cache_dir /home/val/CM/repos/local/cache/8ae2ae9d3c984709
python3 python/main.py --profile retinanet-onnxruntime --mlperf_conf ../../mlperf.conf --model "/home/val/CM/repos/local/cache/4a2de0083ace466f/resnext50_32x4d_fpn.onnx" --dataset-path /home/val/CM/repos/local/cache/46883742a2904037/install --output "/home/val/CM/repos/local/cache/ce4ce58fe2e44ed5/inference/vision/classification_and_detection/test_results/onnxruntime-cpu/retinanet/offline/accuracy" --scenario Offline --mlperf_conf /home/val/CM/repos/local/cache/ce4ce58fe2e44ed5/inference/mlperf.conf --threads 8 --count 10 --user_conf /home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference/tmp/1e6c18c29ec142cd9f59c2e757822785.conf --accuracy --cache_dir /home/val/CM/repos/local/cache/8ae2ae9d3c984709
INFO:main:Namespace(dataset='openimages-800-retinanet-onnx', dataset_path='/home/val/CM/repos/local/cache/46883742a2904037/install', dataset_list=None, data_format=None, profile='retinanet-onnxruntime', scenario='Offline', max_batchsize=1, model='/home/val/CM/repos/local/cache/4a2de0083ace466f/resnext50_32x4d_fpn.onnx', output='/home/val/CM/repos/local/cache/ce4ce58fe2e44ed5/inference/vision/classification_and_detection/test_results/onnxruntime-cpu/retinanet/offline/accuracy', inputs=['images'], outputs=['boxes', 'labels', 'scores'], backend='onnxruntime', model_name='retinanet', threads=8, qps=None, cache=0, cache_dir='/home/val/CM/repos/local/cache/8ae2ae9d3c984709', accuracy=True, find_peak_performance=False, debug=False, mlperf_conf='/home/val/CM/repos/local/cache/ce4ce58fe2e44ed5/inference/mlperf.conf', user_conf='/home/val/CM/repos/mlcommons@ck/cm-mlops/script/app-mlperf-inference/tmp/1e6c18c29ec142cd9f59c2e757822785.conf', audit_conf='audit.config', time=None, count=10, performance_sample_count=None, max_latency=None, samples_per_query=8)
INFO:coco:loaded 10 images, cache=0, took=0.0sec
/home/val/CM/repos/local/cache/ce4ce58fe2e44ed5/inference/vision/classification_and_detection/python/openimages.py:109: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray.
self.label_list = np.array(self.label_list)
INFO:main:starting TestScenario.Offline
loading annotations into memory...
Done (t=0.01s)
creating index...
index created!
Loading and preparing results...
Converting ndarray to lists...
(1008, 7)
0/1008
DONE (t=0.00s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=0.10s).
Accumulating evaluation results...
DONE (t=0.07s).
Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.548
Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.787
Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.714
Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = -1.000
Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.304
Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.631
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.433
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.648
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.663
Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = -1.000
Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.343
Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.731
TestScenario.Offline qps=0.56, mean=13.9297, time=17.706, acc=48.413%, mAP=54.814%, queries=10, tiles=50.0:13.6907,80.0:14.6594,90.0:16.8167,95.0:17.0923,99.0:17.3128,99.9:17.3624
No warnings encountered during test.
No errors encountered during test.
* Searching for collective script(s) with tags "run,accuracy,mlperf" and variations "openimages"
- Number of scripts found: 1
- Found script::process-mlperf-accuracy,6e809013816b42ea in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/process-mlperf-accuracy
Prepared variations: _openimages
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "get,python"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,python
- Number of cached script outputs found: 2
- Found script::get-python3,d0b5dd74373f4a62 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-python3
- Requested version: >= 3.8
- Found cached script output: /home/val/CM/repos/local/cache/becbd4357f9f4838
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,python,python3,get-python,get-python3": 0.08 sec.
* Searching for collective script(s) with tags "get,dataset,original,object-detection,open-images"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,dataset,original,object-detection,open-images
- Number of cached script outputs found: 1
- Found script::get-dataset-openimages,0a9d49b644cf4142 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-dataset-openimages
Prepared variations: _validation
- Found cached script output: /home/val/CM/repos/local/cache/46883742a2904037
- Checking dynamic dependencies on other CM scripts:
- Processing env after dependencies ...
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,dataset,openimages,open-images,object-detection,original": 0.07 sec.
* Searching for collective script(s) with tags "get,mlcommons,inference,src"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,mlcommons,inference,src
- Number of cached script outputs found: 1
- Found script::get-mlperf-inference-src,4b57186581024797 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-mlperf-inference-src
Prepared variations: _default,_short-history
- Found cached script output: /home/val/CM/repos/local/cache/ce4ce58fe2e44ed5
- Checking dynamic dependencies on other CM scripts:
- Processing env after dependencies ...
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,src,source,inference,inference-src,inference-source,mlperf,mlcommons": 0.06 sec.
* Searching for collective script(s) with tags "get,generic-python-lib" and variations "pycocotools"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,generic-python-lib
- Number of cached script outputs found: 7
- Found script::get-generic-python-lib,94b62a682bc44791 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-generic-python-lib
Prepared variations: _pycocotools
- Found cached script output: /home/val/CM/repos/local/cache/80fb4233795e4444
- Checking dynamic dependencies on other CM scripts:
- Processing env after dependencies ...
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,generic,generic-python-lib": 0.08 sec.
- Processing env after dependencies ...
- Running preprocess ...
python3 '/home/val/CM/repos/local/cache/ce4ce58fe2e44ed5/inference/vision/classification_and_detection/tools/accuracy-openimages.py' --mlperf-accuracy-file '/home/val/CM/repos/local/cache/ce4ce58fe2e44ed5/inference/vision/classification_and_detection/test_results/onnxruntime-cpu/retinanet/offline/accuracy/mlperf_log_accuracy.json' --openimages-dir '/home/val/CM/repos/local/cache/46883742a2904037/install' > '/home/val/CM/repos/local/cache/ce4ce58fe2e44ed5/inference/vision/classification_and_detection/test_results/onnxruntime-cpu/retinanet/offline/accuracy/accuracy.txt'
loading annotations into memory...
Done (t=0.01s)
creating index...
index created!
Loading and preparing results...
DONE (t=0.01s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=0.06s).
Accumulating evaluation results...
DONE (t=0.06s).
Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.548
Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.787
Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.714
Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = -1.000
Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.304
Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.631
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.433
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.648
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.663
Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = -1.000
Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.343
Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.731
mAP=54.814%
- running time of script "run,mlperf,mlcommons,accuracy,mlc,process-accuracy": 0.75 sec.
- running time of script "app,vision,language,mlcommons,mlperf,inference,reference,generic,ref": 34.55 sec.
* Searching for collective script(s) with tags "get,sut,description"
- Number of scripts found: 1
- Found script::get-mlperf-inference-sut-description,e49a3f758b2d4e7b in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-mlperf-inference-sut-description
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,os"
- Number of scripts found: 1
- Found script::detect-os,863735b7db8c44fc in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-os
- Running preprocess ...
- running time of script "detect-os,detect,os,info": 0.10 sec.
* Searching for collective script(s) with tags "detect,cpu"
- Number of scripts found: 1
- Found script::detect-cpu,586c8a43320142f7 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-cpu
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,os"
- Number of scripts found: 1
- Found script::detect-os,863735b7db8c44fc in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-os
- Running preprocess ...
- running time of script "detect-os,detect,os,info": 0.09 sec.
- Processing env after dependencies ...
- Running preprocess ...
- running time of script "detect,cpu,detect-cpu,info": 0.19 sec.
* Searching for collective script(s) with tags "get,python3"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,python3
- Number of cached script outputs found: 2
- Found script::get-python3,d0b5dd74373f4a62 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-python3
- Requested version: >= 3.8
- Found cached script output: /home/val/CM/repos/local/cache/becbd4357f9f4838
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,python,python3,get-python,get-python3": 0.07 sec.
* Searching for collective script(s) with tags "get,compiler,gcc"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,compiler,gcc
- Number of cached script outputs found: 1
- Found script::get-gcc,dbf4ab5cbed74372 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-gcc
- Found cached script output: /home/val/CM/repos/local/cache/128d5e94b35742f5
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
* Searching for collective script(s) with tags "get,compiler-flags"
- Number of scripts found: 1
- Found script::get-compiler-flags,31be8b74a69742f8 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-compiler-flags
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,cpu"
- Number of scripts found: 1
- Found script::detect-cpu,586c8a43320142f7 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-cpu
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,os"
- Number of scripts found: 1
- Found script::detect-os,863735b7db8c44fc in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-os
- Running preprocess ...
- running time of script "detect-os,detect,os,info": 0.09 sec.
- Processing env after dependencies ...
- Running preprocess ...
- running time of script "detect,cpu,detect-cpu,info": 0.18 sec.
- Processing env after dependencies ...
- Running preprocess ...
- running time of script "get,compiler-flags": 0.26 sec.
- running time of script "get,gcc,compiler,c-compiler,cpp-compiler,get-gcc": 0.34 sec.
- Processing env after dependencies ...
- Running preprocess ...
Generating SUT description file for default-onnxruntime-1.13.1
- running time of script "get,mlperf,sut,description,system-under-test,system-description": 0.81 sec.
* Searching for collective script(s) with tags "generate,mlperf,inference,submission"
- Number of scripts found: 1
- Found script::generate-mlperf-inference-submission,5f8ab2d0b5874d53 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/generate-mlperf-inference-submission
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "get,python"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,python
- Number of cached script outputs found: 2
- Found script::get-python3,d0b5dd74373f4a62 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-python3
- Requested version: >= 3.8
- Found cached script output: /home/val/CM/repos/local/cache/becbd4357f9f4838
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,python,python3,get-python,get-python3": 0.07 sec.
* Searching for collective script(s) with tags "mlcommons,inference,src"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,mlcommons,inference,src
- Number of cached script outputs found: 1
- Found script::get-mlperf-inference-src,4b57186581024797 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-mlperf-inference-src
Prepared variations: _default,_short-history
- Found cached script output: /home/val/CM/repos/local/cache/ce4ce58fe2e44ed5
- Checking dynamic dependencies on other CM scripts:
- Processing env after dependencies ...
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,src,source,inference,inference-src,inference-source,mlperf,mlcommons": 0.07 sec.
* Searching for collective script(s) with tags "get,sut,system-description"
- Number of scripts found: 1
- Found script::get-mlperf-inference-sut-description,e49a3f758b2d4e7b in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-mlperf-inference-sut-description
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,os"
- Number of scripts found: 1
- Found script::detect-os,863735b7db8c44fc in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-os
- Running preprocess ...
- running time of script "detect-os,detect,os,info": 0.09 sec.
* Searching for collective script(s) with tags "detect,cpu"
- Number of scripts found: 1
- Found script::detect-cpu,586c8a43320142f7 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-cpu
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,os"
- Number of scripts found: 1
- Found script::detect-os,863735b7db8c44fc in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-os
- Running preprocess ...
- running time of script "detect-os,detect,os,info": 0.09 sec.
- Processing env after dependencies ...
- Running preprocess ...
- running time of script "detect,cpu,detect-cpu,info": 0.19 sec.
* Searching for collective script(s) with tags "get,python3"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,python3
- Number of cached script outputs found: 2
- Found script::get-python3,d0b5dd74373f4a62 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-python3
- Requested version: >= 3.8
- Found cached script output: /home/val/CM/repos/local/cache/becbd4357f9f4838
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,python,python3,get-python,get-python3": 0.08 sec.
* Searching for collective script(s) with tags "get,compiler,gcc"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,compiler,gcc
- Number of cached script outputs found: 1
- Found script::get-gcc,dbf4ab5cbed74372 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-gcc
- Found cached script output: /home/val/CM/repos/local/cache/128d5e94b35742f5
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
* Searching for collective script(s) with tags "get,compiler-flags"
- Number of scripts found: 1
- Found script::get-compiler-flags,31be8b74a69742f8 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-compiler-flags
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,cpu"
- Number of scripts found: 1
- Found script::detect-cpu,586c8a43320142f7 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-cpu
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "detect,os"
- Number of scripts found: 1
- Found script::detect-os,863735b7db8c44fc in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/detect-os
- Running preprocess ...
- running time of script "detect-os,detect,os,info": 0.09 sec.
- Processing env after dependencies ...
- Running preprocess ...
- running time of script "detect,cpu,detect-cpu,info": 0.18 sec.
- Processing env after dependencies ...
- Running preprocess ...
- running time of script "get,compiler-flags": 0.27 sec.
- running time of script "get,gcc,compiler,c-compiler,cpp-compiler,get-gcc": 0.35 sec.
- Processing env after dependencies ...
- Running preprocess ...
Generating SUT description file for default-onnxruntime-1.13.1
- running time of script "get,mlperf,sut,description,system-under-test,system-description": 0.81 sec.
- Processing env after dependencies ...
- Running preprocess ...
* MLPerf inference submission dir: /home/val/mlperf_submission
* MLPerf inference results dir: /home/val/CM/repos/local/cache/ce4ce58fe2e44ed5/inference/vision/classification_and_detection/test_results
* MLPerf inference division: open
* MLPerf inference submitter: OctoML
* Target: cpu
* Framework: onnxruntime
* MLPerf inference model: retinanet
* mlperf_log_detail.txt
* mlperf_log_accuracy.json
* mlperf_log_summary.txt
* mlperf_log_detail.txt
* mlperf_log_accuracy.json
* mlperf_log_summary.txt
* accuracy.txt
* Searching for collective script(s) with tags "accuracy,truncate,mlc"
- Number of scripts found: 1
- Found script::run-mlperf-accuracy-log-truncator,9d5ec20434084d14 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/run-mlperf-accuracy-log-truncator
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "get,python"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,python
- Number of cached script outputs found: 2
- Found script::get-python3,d0b5dd74373f4a62 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-python3
- Requested version: >= 3.8
- Found cached script output: /home/val/CM/repos/local/cache/becbd4357f9f4838
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,python,python3,get-python,get-python3": 0.07 sec.
* Searching for collective script(s) with tags "get,mlcommons,inference,src"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,mlcommons,inference,src
- Number of cached script outputs found: 1
- Found script::get-mlperf-inference-src,4b57186581024797 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-mlperf-inference-src
Prepared variations: _default,_short-history
- Found cached script output: /home/val/CM/repos/local/cache/ce4ce58fe2e44ed5
- Checking dynamic dependencies on other CM scripts:
- Processing env after dependencies ...
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,src,source,inference,inference-src,inference-source,mlperf,mlcommons": 0.07 sec.
- Processing env after dependencies ...
- Running preprocess ...
python3 '/home/val/CM/repos/local/cache/ce4ce58fe2e44ed5/inference/tools/submission/truncate_accuracy_log.py' --input /home/val/mlperf_submission --submitter OctoML --backup /home/val/mlperf_submission_logs
INFO:main:open/OctoML/results/onnxruntime-cpu/retinanet/offline/accuracy/mlperf_log_accuracy.json truncated
ERROR:main:no submission in open/OctoML/compliance
INFO:main:Make sure you keep a backup of /home/val/mlperf_submission_logs in case mlperf wants to see the original accuracy logs
- running time of script "run,mlc,mlcommons,mlperf,inference,mlperf-inference,truncation,truncator,truncate,accuracy,accuracy-log,accuracy-log-trancation,accuracy-log-truncator,mlc-accuracy-log-trancation,mlc-accuracy-log-truncator": 0.26 sec.
* Searching for collective script(s) with tags "submission,checker,mlc" and variations "short-run"
- Number of scripts found: 1
- Found script::run-mlperf-inference-submission-checker,15d03ec2c1af4297 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/run-mlperf-inference-submission-checker
Prepared variations: _short-run
- Checking dependencies on other CM scripts:
* Searching for collective script(s) with tags "get,python"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,python
- Number of cached script outputs found: 2
- Found script::get-python3,d0b5dd74373f4a62 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-python3
- Requested version: >= 3.8
- Found cached script output: /home/val/CM/repos/local/cache/becbd4357f9f4838
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,python,python3,get-python,get-python3": 0.07 sec.
* Searching for collective script(s) with tags "get,mlcommons,inference,src"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,mlcommons,inference,src
- Number of cached script outputs found: 1
- Found script::get-mlperf-inference-src,4b57186581024797 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-mlperf-inference-src
Prepared variations: _default,_short-history
- Found cached script output: /home/val/CM/repos/local/cache/ce4ce58fe2e44ed5
- Checking dynamic dependencies on other CM scripts:
- Processing env after dependencies ...
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,src,source,inference,inference-src,inference-source,mlperf,mlcommons": 0.06 sec.
* Searching for collective script(s) with tags "get,generic-python-lib" and variations "pandas"
- Number of scripts found: 1
- Searching for cached script outputs with the following tags: -tmp,get,generic-python-lib
- Number of cached script outputs found: 7
- Found script::get-generic-python-lib,94b62a682bc44791 in /home/val/CM/repos/mlcommons@ck/cm-mlops/script/get-generic-python-lib
Prepared variations: _pandas
- Found cached script output: /home/val/CM/repos/local/cache/a50f88a9b6f74e96
- Checking dynamic dependencies on other CM scripts:
- Processing env after dependencies ...
- Checking prehook dependencies on other CM scripts:
- Loading state from cached entry ...
- Checking posthook dependencies on other CM scripts:
- Checking post dependencies on other CM scripts:
- running time of script "get,generic,generic-python-lib": 0.08 sec.
- Processing env after dependencies ...
- Running preprocess ...
=================================================
python3 /home/val/CM/repos/local/cache/ce4ce58fe2e44ed5/inference/tools/submission/submission_checker1.py --input /home/val/mlperf_submission --submitter OctoML
=================================================
[2022-11-15 21:09:51,030 log_parser.py:50 INFO] Sucessfully loaded MLPerf log from open/OctoML/results/onnxruntime-cpu/retinanet/offline/accuracy/mlperf_log_detail.txt.
[2022-11-15 21:09:51,030 log_parser.py:50 INFO] Sucessfully loaded MLPerf log from open/OctoML/results/onnxruntime-cpu/retinanet/offline/performance/run_1/mlperf_log_detail.txt.
[2022-11-15 21:09:51,031 log_parser.py:50 INFO] Sucessfully loaded MLPerf log from open/OctoML/results/onnxruntime-cpu/retinanet/offline/performance/run_1/mlperf_log_detail.txt.
[2022-11-15 21:09:51,031 submission_checker1.py:1516 INFO] Target latency: None, Latency: 1070121324577, Scenario: Offline
[2022-11-15 21:09:51,031 submission_checker1.py:2455 INFO] ---
[2022-11-15 21:09:51,031 submission_checker1.py:2459 INFO] Results open/OctoML/results/onnxruntime-cpu/retinanet/offline 0.615298
[2022-11-15 21:09:51,031 submission_checker1.py:2461 INFO] ---
[2022-11-15 21:09:51,031 submission_checker1.py:2467 INFO] ---
[2022-11-15 21:09:51,031 submission_checker1.py:2468 INFO] Results=1, NoResults=0
[2022-11-15 21:09:51,031 submission_checker1.py:2474 INFO] SUMMARY: submission looks OK
Traceback (most recent call last):
File "/home/val/Coding/mlcommon/venv-cm/bin/cm", line 8, in <module>
sys.exit(run())
File "/home/val/Coding/mlcommon/venv-cm/lib/python3.10/site-packages/cmind/cli.py", line 35, in run
r = cm.access(argv, out='con')
File "/home/val/Coding/mlcommon/venv-cm/lib/python3.10/site-packages/cmind/core.py", line 486, in access
r = action_addr(i)
File "/home/val/CM/repos/mlcommons@ck/cm-mlops/automation/script/module.py", line 1309, in run
r = self._run_deps(post_deps, clean_env_keys_post_deps, env, state, const, const_state, add_deps_recursive, recursion_spaces,
File "/home/val/CM/repos/mlcommons@ck/cm-mlops/automation/script/module.py", line 1847, in _run_deps
r = self.cmind.access(ii)
File "/home/val/Coding/mlcommon/venv-cm/lib/python3.10/site-packages/cmind/core.py", line 486, in access
r = action_addr(i)
File "/home/val/CM/repos/mlcommons@ck/cm-mlops/automation/script/module.py", line 1309, in run
r = self._run_deps(post_deps, clean_env_keys_post_deps, env, state, const, const_state, add_deps_recursive, recursion_spaces,
File "/home/val/CM/repos/mlcommons@ck/cm-mlops/automation/script/module.py", line 1847, in _run_deps
r = self.cmind.access(ii)
File "/home/val/Coding/mlcommon/venv-cm/lib/python3.10/site-packages/cmind/core.py", line 486, in access
r = action_addr(i)
File "/home/val/CM/repos/mlcommons@ck/cm-mlops/automation/script/module.py", line 1291, in run
r = prepare_and_run_script_with_postprocessing(run_script_input)
File "/home/val/CM/repos/mlcommons@ck/cm-mlops/automation/script/module.py", line 3019, in prepare_and_run_script_with_postprocessing
rr = run_postprocess(customize_code, customize_common_input, recursion_spaces, env, state, const,
File "/home/val/CM/repos/mlcommons@ck/cm-mlops/automation/script/module.py", line 3066, in run_postprocess
r = customize_code.postprocess(ii)
File "/home/val/CM/repos/mlcommons@ck/cm-mlops/script/run-mlperf-inference-submission-checker/customize.py", line 49, in postprocess
import pandasSorry! I managed to reproduce this error from virtual env and I am doing another attempt to fix it ;) ...
gfursin
commented
1 year ago Hi @ValouBambou ;) . I did a new update. Can you please update mlcommons@ck and try again? Normally, you do not need to --rerun experiments. Thank you very much for your patience!
ValouBambou
commented
1 year ago It worked !!! Thank you :)
gfursin
commented
1 year ago Cool! Thanks a lot again for your feedback - it was very useful!!!
Hi, I'm trying to do the tutorial here on my laptop using an arch linux based distro. And from what I understand, the scripts in charge to detect and install system dependencies don't support this scenario. Also, I used a python 3.10.8 installed in a venv with only cmind installed (version 1.0.5 according to the cli).
The result from the detect os command
cm run script "get sys-utils-cm"return this without error but does nothing.And when trying to
cm run script "get mlperf inference src"it returns:Hopefully, this can help you.