Open sunpian1 opened 6 months ago
arjunsuresh
commented
6 months ago Can you please remove the CM repo and use the new one? I have just added the support to build the latest master branch of TF which requires clang.
cm rm repo mlcommons@ck -f
cm pull repo gateoverflow@cm4mlops
sunpian1
commented
6 months ago cmr "run mobilenet-models _tflite _accuracy-only" --adr.compiler.tags=gcc --results_dir=$HOME/mobilenet_results, does this command run on cpu?how to run it on gpu
arjunsuresh
commented
6 months ago @sunpian1 the mobilenet inference implementation currently supports only CPU and also only the singlestream scenario. If you would like to extend it we can guide you on how to do this.
sunpian1
commented
6 months ago I would like to run it on gpu,please guide me on how to do this. Thank you very much
arjunsuresh
commented
6 months ago The mobilenet models are being run using tflite framework and this is the implementation file. But this is an old implementation which works only for SingleStream (not been extended for sometime now).
If you would like to make TFLite framwoek run on GPU, this code should be the best option. Currently it supports onnxruntime - both CPU and CUDA for ResNet50 and RetinaNet.
sunpian1
commented
6 months ago Use the old implementation, error reports like below
(mlperf) susie.sun@yizhu-R5300-G5:~$ cm run script --tags=app,mlperf,inference,tflite-cpp,_gpu,_fp32,_mobilenet,_singlestream,_tflite
cm run script "app mlperf inference tflite-cpp _gpu _fp32 _mobilenet _singlestream _tflite"
cm run script "detect os" ! cd /home/susie.sun ! call /home/susie.sun/CM/repos/gateoverflow@cm4mlops/script/detect-os/run.sh from tmp-run.sh ! call "postprocess" from /home/susie.sun/CM/repos/gateoverflow@cm4mlops/script/detect-os/customize.py
cm run script "detect cpu"
cm run script "detect os" ! cd /home/susie.sun ! call /home/susie.sun/CM/repos/gateoverflow@cm4mlops/script/detect-os/run.sh from tmp-run.sh ! call "postprocess" from /home/susie.sun/CM/repos/gateoverflow@cm4mlops/script/detect-os/customize.py ! cd /home/susie.sun ! call /home/susie.sun/CM/repos/gateoverflow@cm4mlops/script/detect-cpu/run.sh from tmp-run.sh ! call "postprocess" from /home/susie.sun/CM/repos/gateoverflow@cm4mlops/script/detect-cpu/customize.py
cm run script "get sys-utils-cm" ! load /home/susie.sun/CM/repos/local/cache/efa2ef43a30c43ed/cm-cached-state.json
cm run script "get cuda"
0) /home/susie.sun/CM/repos/local/cache/93476d18b16e44c9 (get,cuda,cuda-compiler,cuda-lib,toolkit,lib,nvcc,get-nvcc,get-cuda,46d133d9ef92422d,_cudnn,script-artifact-46d133d9ef92422d,version-11.3) (Version 11.3) 1) /home/susie.sun/CM/repos/local/cache/61f4a2c3723f42d1 (get,cuda,cuda-compiler,cuda-lib,toolkit,lib,nvcc,get-nvcc,get-cuda,46d133d9ef92422d,_toolkit,script-artifact-46d133d9ef92422d,version-11.3) (Version 11.3) 2) /home/susie.sun/CM/repos/local/cache/f53f60e5071045cc (get,cuda,cuda-compiler,cuda-lib,toolkit,lib,nvcc,get-nvcc,get-cuda,46d133d9ef92422d,script-artifact-46d133d9ef92422d,version-11.3) (Version 11.3)
Make your selection or press Enter for 0 or use -1 to skip: -1
Skipped
cm run script "detect os" ! cd /home/susie.sun/CM/repos/local/cache/3498326c83724ec6 ! call /home/susie.sun/CM/repos/gateoverflow@cm4mlops/script/detect-os/run.sh from tmp-run.sh ! call "postprocess" from /home/susie.sun/CM/repos/gateoverflow@cm4mlops/script/detect-os/customize.py
/home/susie.sun/cuda113/bin/nvcc ! cd /home/susie.sun/CM/repos/local/cache/3498326c83724ec6 ! call /home/susie.sun/CM/repos/gateoverflow@cm4mlops/script/get-cuda/run.sh from tmp-run.sh ! call "detect_version" from /home/susie.sun/CM/repos/gateoverflow@cm4mlops/script/get-cuda/customize.py Detected version: 11.3
/usr/bin/nvcc ! cd /home/susie.sun/CM/repos/local/cache/3498326c83724ec6 ! call /home/susie.sun/CM/repos/gateoverflow@cm4mlops/script/get-cuda/run.sh from tmp-run.sh ! call "detect_version" from /home/susie.sun/CM/repos/gateoverflow@cm4mlops/script/get-cuda/customize.py Detected version: 12.1
More than 1 path found:
0) /home/susie.sun/cuda113/bin/nvcc 1) /usr/bin/nvcc
Make your selection or press Enter for 0: 0
Selected 0: /home/susie.sun/cuda113/bin/nvcc
! cd /home/susie.sun/CM/repos/local/cache/3498326c83724ec6
! call /home/susie.sun/CM/repos/gateoverflow@cm4mlops/script/get-cuda/run.sh from tmp-run.sh
! call "postprocess" from /home/susie.sun/CM/repos/gateoverflow@cm4mlops/script/get-cuda/customize.pyDetected version: 11.3
ENV[CM_CUDA_PATH_LIB_CUDNN_EXISTS]: no ENV[CM_CUDA_VERSION]: 11.3 ENV[CM_CUDA_VERSION_STRING]: cu113 ENV[CM_NVCC_BIN_WITH_PATH]: /home/susie.sun/cuda113/bin/nvcc ENV[CUDA_HOME]: /home/susie.sun/cuda113
Path to the tool: /home/susie.sun/CM/repos/local/cache/d2ba4ede47544bc9/install
cm run script "get mlcommons inference src"
More than 1 cached script output found for "get,mlcommons,inference,src":
0) /home/susie.sun/CM/repos/local/cache/18f5158760f640c7 (get,mlcommons,inference,src,source,inference-src,inference-source,mlperf,script-artifact-4b57186581024797,version-master-git-5ba8e81a73ff631363139ab3a2eb783daac97ac8) (Version master-git-5ba8e81a73ff631363139ab3a2eb783daac97ac8) 1) /home/susie.sun/CM/repos/local/cache/cad452ba15a149a6 (get,mlcommons,inference,src,source,inference-src,inference-source,mlperf,script-artifact-4b57186581024797,version-master-git-5ba8e81a73ff631363139ab3a2eb783daac97ac8) (Version master-git-5ba8e81a73ff631363139ab3a2eb783daac97ac8) 2) /home/susie.sun/CM/repos/local/cache/ec5f3cd908fa4ae3 (get,mlcommons,inference,src,source,inference-src,inference-source,mlperf,script-artifact-4b57186581024797,version-master-git-5ba8e81a73ff631363139ab3a2eb783daac97ac8) (Version master-git-5ba8e81a73ff631363139ab3a2eb783daac97ac8)
Make your selection or press Enter for 0 or use -1 to skip: -1
Skipped
cm run script "detect os" ! cd /home/susie.sun/CM/repos/local/cache/b7709d4bcac44ea8 ! call /home/susie.sun/CM/repos/gateoverflow@cm4mlops/script/detect-os/run.sh from tmp-run.sh ! call "postprocess" from /home/susie.sun/CM/repos/gateoverflow@cm4mlops/script/detect-os/customize.py
cm run script "get python3" ! load /home/susie.sun/CM/repos/local/cache/631007b15b4b4d1d/cm-cached-state.json
Path to Python: /home/susie.sun/anaconda3/envs/mlperf/bin/python3 Python version: 3.10.0
* cm run script "get git repo _branch.master _repo.https://github.com/mlcommons/inference.git"
! load /home/susie.sun/CM/repos/local/cache/6db3115c5e4e49af/cm-cached-state.jsonCM cache path to the Git repo: /home/susie.sun/CM/repos/local/cache/6db3115c5e4e49af/inference
! call "postprocess" from /home/susie.sun/CM/repos/gateoverflow@cm4mlops/script/get-mlperf-inference-src/customize.pycm run script "get ml-model mobilenet raw _tflite _fp32"
More than 1 cached script output found for "get,ml-model,mobilenet,raw":
0) /home/susie.sun/CM/repos/local/cache/b0957cb85dce4ebf (get,ml-model,mobilenet,raw,ml-model-mobilenet,image-classification,_tflite,_resolution-128,_v1,_multiplier-0.25,_fp32,script-artifact-ce46675a3ab249e4) 1) /home/susie.sun/CM/repos/local/cache/bef9aaea2d5e4c49 (get,ml-model,mobilenet,raw,ml-model-mobilenet,image-classification,_tflite,_resolution-160,_v1,_multiplier-0.25,_fp32,script-artifact-ce46675a3ab249e4) 2) /home/susie.sun/CM/repos/local/cache/9ec63564a9d64e00 (get,ml-model,mobilenet,raw,ml-model-mobilenet,image-classification,_tflite,_v1,_multiplier-0.25,_resolution-192,_fp32,script-artifact-ce46675a3ab249e4) 3) /home/susie.sun/CM/repos/local/cache/cd3e895bf68e44c7 (get,ml-model,mobilenet,raw,ml-model-mobilenet,image-classification,_tflite,_v1,_resolution-224,_multiplier-0.25,_fp32,script-artifact-ce46675a3ab249e4) 4) /home/susie.sun/CM/repos/local/cache/abb1441a24474311 (get,ml-model,mobilenet,raw,ml-model-mobilenet,image-classification,_tflite,_resolution-128,_v1,_multiplier-0.5,_fp32,script-artifact-ce46675a3ab249e4) 5) /home/susie.sun/CM/repos/local/cache/28fbfd36cf81492d (get,ml-model,mobilenet,raw,ml-model-mobilenet,image-classification,_tflite,_resolution-160,_v1,_multiplier-0.5,_fp32,script-artifact-ce46675a3ab249e4) 6) /home/susie.sun/CM/repos/local/cache/6f6b22b8b2974318 (get,ml-model,mobilenet,raw,ml-model-mobilenet,image-classification,_tflite,_v1,_multiplier-0.5,_resolution-192,_fp32,script-artifact-ce46675a3ab249e4) 7) /home/susie.sun/CM/repos/local/cache/627f7f1ed25d43fa (get,ml-model,mobilenet,raw,ml-model-mobilenet,image-classification,_tflite,_multiplier-0.5,_v1,_resolution-224,_fp32,script-artifact-ce46675a3ab249e4) 8) /home/susie.sun/CM/repos/local/cache/f709ae9104d8436d (get,ml-model,mobilenet,raw,ml-model-mobilenet,image-classification,_tflite,_resolution-128,_v1,_multiplier-0.75,_fp32,script-artifact-ce46675a3ab249e4) 9) /home/susie.sun/CM/repos/local/cache/85b48bda9ec143d7 (get,ml-model,mobilenet,raw,ml-model-mobilenet,image-classification,_tflite,_resolution-160,_v1,_multiplier-0.75,_fp32,script-artifact-ce46675a3ab249e4) 10) /home/susie.sun/CM/repos/local/cache/dd7a126d16604f6b (get,ml-model,mobilenet,raw,ml-model-mobilenet,image-classification,_tflite,_resolution-160,_v2,_multiplier-0.75,_fp32,script-artifact-ce46675a3ab249e4) 11) /home/susie.sun/CM/repos/local/cache/1508deea9e364dbf (get,ml-model,mobilenet,raw,ml-model-mobilenet,image-classification,_tflite,_v1,_multiplier-0.75,_resolution-192,_fp32,script-artifact-ce46675a3ab249e4) 12) /home/susie.sun/CM/repos/local/cache/ea978f31bd95441d (get,ml-model,mobilenet,raw,ml-model-mobilenet,image-classification,_tflite,_v2,_multiplier-0.75,_resolution-192,_fp32,script-artifact-ce46675a3ab249e4) 13) /home/susie.sun/CM/repos/local/cache/7a4c989a55874fdd (get,ml-model,mobilenet,raw,ml-model-mobilenet,image-classification,_tflite,_v1,_resolution-224,_multiplier-0.75,_fp32,script-artifact-ce46675a3ab249e4) 14) /home/susie.sun/CM/repos/local/cache/a6f5fca1844246f2 (get,ml-model,mobilenet,raw,ml-model-mobilenet,image-classification,_tflite,_v2,_resolution-224,_multiplier-0.75,_fp32,script-artifact-ce46675a3ab249e4) 15) /home/susie.sun/CM/repos/local/cache/d044a25cf1554cb3 (get,ml-model,mobilenet,raw,ml-model-mobilenet,image-classification,_tflite,_multiplier-1.0,_resolution-128,_v1,_fp32,script-artifact-ce46675a3ab249e4) 16) /home/susie.sun/CM/repos/local/cache/a38142fc062a4e33 (get,ml-model,mobilenet,raw,ml-model-mobilenet,image-classification,_tflite,_multiplier-1.0,_resolution-128,_v2,_fp32,script-artifact-ce46675a3ab249e4) 17) /home/susie.sun/CM/repos/local/cache/8efabbed7cf94346 (get,ml-model,mobilenet,raw,ml-model-mobilenet,image-classification,_tflite,_multiplier-1.0,_resolution-160,_v1,_fp32,script-artifact-ce46675a3ab249e4) 18) /home/susie.sun/CM/repos/local/cache/33cd02e665564162 (get,ml-model,mobilenet,raw,ml-model-mobilenet,image-classification,_tflite,_multiplier-1.0,_resolution-160,_v2,_fp32,script-artifact-ce46675a3ab249e4) 19) /home/susie.sun/CM/repos/local/cache/969a46662ba440bd (get,ml-model,mobilenet,raw,ml-model-mobilenet,image-classification,_tflite,_multiplier-1.0,_v1,_resolution-192,_fp32,script-artifact-ce46675a3ab249e4) 20) /home/susie.sun/CM/repos/local/cache/2690464abb6c4e2e (get,ml-model,mobilenet,raw,ml-model-mobilenet,image-classification,_tflite,_multiplier-1.0,_v2,_resolution-192,_fp32,script-artifact-ce46675a3ab249e4) 21) /home/susie.sun/CM/repos/local/cache/e3997b12d6974cd7 (get,ml-model,mobilenet,raw,ml-model-mobilenet,image-classification,_tflite,_multiplier-1.0,_resolution-224,_v1,_fp32,script-artifact-ce46675a3ab249e4) 22) /home/susie.sun/CM/repos/local/cache/ab88c32e22cf4bb6 (get,ml-model,mobilenet,raw,ml-model-mobilenet,image-classification,_tflite,_multiplier-1.0,_v2,_resolution-224,_fp32,script-artifact-ce46675a3ab249e4)
Make your selection or press Enter for 0 or use -1 to skip: -1
Skipped
Downloading from https://storage.googleapis.com/mobilenet_v3/checkpoints/v3-<<
CM error: 404 Client Error: Not Found for url: https://storage.googleapis.com/mobilenet_v3/checkpoints/v3-%3C%3C%3CCM_ML_MODEL_MOBILENET_KIND%3E%3E%3E_224_1.0_float.tgz! (mlperf) susie.sun@yizhu-R5300-G5:~$
(mlperf) susie.sun@yizhu-R5300-G5:~$ cmr "run mobilenet-models _tflite _accuracy-only" \
cm run script "run mobilenet-models _tflite _accuracy-only"
cm run script "generate-run-cmds mlperf inference _full _accuracy-only"
cm run script "detect os" ! cd /home/susie.sun ! call /home/susie.sun/CM/repos/mlcommons@cm4mlops/script/detect-os/run.sh from tmp-run.sh ! call "postprocess" from /home/susie.sun/CM/repos/mlcommons@cm4mlops/script/detect-os/customize.py
cm run script "detect cpu"
cm run script "detect os" ! cd /home/susie.sun ! call /home/susie.sun/CM/repos/mlcommons@cm4mlops/script/detect-os/run.sh from tmp-run.sh ! call "postprocess" from /home/susie.sun/CM/repos/mlcommons@cm4mlops/script/detect-os/customize.py ! cd /home/susie.sun ! call /home/susie.sun/CM/repos/mlcommons@cm4mlops/script/detect-cpu/run.sh from tmp-run.sh ! call "postprocess" from /home/susie.sun/CM/repos/mlcommons@cm4mlops/script/detect-cpu/customize.py
cm run script "get python3" ! load /home/susie.sun/CM/repos/local/cache/38444f88746d4e25/cm-cached-state.json
Path to Python: /home/susie.sun/anaconda3/envs/mlperf/bin/python3 Python version: 3.10.0
Path to the MLPerf inference benchmark configuration file: /home/susie.sun/CM/repos/local/cache/2c8c91d452654dd5/inference/mlperf.conf Path to MLPerf inference benchmark sources: /home/susie.sun/CM/repos/local/cache/2c8c91d452654dd5/inference
cm run script "get sut description" ! load /home/susie.sun/CM/repos/local/cache/1ddd481dde0648ee/cm-cached-state.json
cm run script "install pip-package for-cmind-python _package.tabulate" ! load /home/susie.sun/CM/repos/local/cache/1ed4900679914a1a/cm-cached-state.json
cm run script "get mlperf inference utils"
Path to the MLPerf inference benchmark configuration file: /home/susie.sun/CM/repos/local/cache/2c8c91d452654dd5/inference/mlperf.conf Path to MLPerf inference benchmark sources: /home/susie.sun/CM/repos/local/cache/2c8c91d452654dd5/inference
Using MLCommons Inference source from /home/susie.sun/CM/repos/local/cache/2c8c91d452654dd5/inference
Running loadgen scenario: SingleStream and mode: accuracy
cm run script "app mlperf inference generic _tflite-cpp _mobilenet _valid _fp32 _singlestream"
cm run script "detect os" ! cd /home/susie.sun ! call /home/susie.sun/CM/repos/mlcommons@cm4mlops/script/detect-os/run.sh from tmp-run.sh ! call "postprocess" from /home/susie.sun/CM/repos/mlcommons@cm4mlops/script/detect-os/customize.py
cm run script "get sys-utils-cm" ! load /home/susie.sun/CM/repos/local/cache/576bc766a772475d/cm-cached-state.json
cm run script "get python" ! load /home/susie.sun/CM/repos/local/cache/38444f88746d4e25/cm-cached-state.json
Path to Python: /home/susie.sun/anaconda3/envs/mlperf/bin/python3 Python version: 3.10.0
Path to the MLPerf inference benchmark configuration file: /home/susie.sun/CM/repos/local/cache/2c8c91d452654dd5/inference/mlperf.conf Path to MLPerf inference benchmark sources: /home/susie.sun/CM/repos/local/cache/2c8c91d452654dd5/inference
cm run script "get mlperf inference utils"
Path to the MLPerf inference benchmark configuration file: /home/susie.sun/CM/repos/local/cache/2c8c91d452654dd5/inference/mlperf.conf Path to MLPerf inference benchmark sources: /home/susie.sun/CM/repos/local/cache/2c8c91d452654dd5/inference
cm run script "get dataset-aux imagenet-aux" ! load /home/susie.sun/CM/repos/local/cache/cfc97b4769654b05/cm-cached-state.json
cm run script "app mlperf tflite-cpp inference _mobilenet _tflite _cpu _singlestream"
cm run script "detect os" ! cd /home/susie.sun ! call /home/susie.sun/CM/repos/mlcommons@cm4mlops/script/detect-os/run.sh from tmp-run.sh ! call "postprocess" from /home/susie.sun/CM/repos/mlcommons@cm4mlops/script/detect-os/customize.py
cm run script "detect cpu"
cm run script "get sys-utils-cm" ! load /home/susie.sun/CM/repos/local/cache/576bc766a772475d/cm-cached-state.json
cm run script "get loadgen" ! load /home/susie.sun/CM/repos/local/cache/782950bcd3fd474c/cm-cached-state.json
Path to the tool: /home/susie.sun/CM/repos/local/cache/782950bcd3fd474c/install
Path to the MLPerf inference benchmark configuration file: /home/susie.sun/CM/repos/local/cache/2c8c91d452654dd5/inference/mlperf.conf Path to MLPerf inference benchmark sources: /home/susie.sun/CM/repos/local/cache/2c8c91d452654dd5/inference
Downloading from http://download.tensorflow.org/models/mobilenet_v1_2018_08_02/mobilenet_v1_1.0_224.tgz Downloading to /home/susie.sun/CM/repos/local/cache/78749b76f6ee4674/mobilenet_v1_1.0_224.tgz
Downloaded: 100% ./ ./mobilenet_v1_1.0_224.tflite ./mobilenet_v1_1.0_224.ckpt.meta ./mobilenet_v1_1.0_224.ckpt.index ./mobilenet_v1_1.0_224.ckpt.data-00000-of-00001 ./mobilenet_v1_1.0_224_info.txt ./mobilenet_v1_1.0_224_frozen.pb ./mobilenet_v1_1.0_224_eval.pbtxt
Path to the ML model: /home/susie.sun/CM/repos/local/cache/78749b76f6ee4674/mobilenet_v1_1.0_224.tflite
Path to Python: /home/susie.sun/anaconda3/envs/mlperf/bin/python3 Python version: 3.10.0
Path to Python: /home/susie.sun/anaconda3/envs/mlperf/bin/python3 Python version: 3.10.0
/home/susie.sun/.local/bin/gcc --version /home/susie.sun/CM/repos/mlcommons@cm4mlops/script/get-gcc/run.sh: line 5: /home/susie.sun/.local/bin/gcc: No such file or directory
/usr/bin/gcc --version gcc (Ubuntu 7.5.0-6ubuntu2) 7.5.0 Copyright (C) 2017 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
/usr/bin/gcc-8 --version gcc-8 (Ubuntu 8.4.0-3ubuntu2) 8.4.0 Copyright (C) 2018 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
/usr/bin/gcc-9 --version gcc-9 (Ubuntu 9.4.0-1ubuntu1~20.04.2) 9.4.0 Copyright (C) 2019 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
/usr/bin/gcc-10 --version gcc-10 (Ubuntu 10.5.0-1ubuntu1~20.04) 10.5.0 Copyright (C) 2020 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
/usr/bin/gcc-11 --version gcc-11 (Ubuntu 11.4.0-2ubuntu1~20.04) 11.4.0 Copyright (C) 2021 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
/usr/bin/gcc-9 --version gcc-9 (Ubuntu 9.4.0-1ubuntu1~20.04.2) 9.4.0 Copyright (C) 2019 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
--2024-05-23 14:28:18-- https://github.com/bazelbuild/bazel/releases/download/6.5.0/bazel-6.5.0-installer-linux-x86_64.sh Connecting to 10.10.60.103:7890... connected. Proxy request sent, awaiting response... 302 Found Location: https://objects.githubusercontent.com/github-production-release-asset-2e65be/20773773/6f2926f6-5ad0-44ce-aeaf-071ad279408c?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=releaseassetproduction%2F20240523%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20240523T062819Z&X-Amz-Expires=300&X-Amz-Signature=f0ad97054bd8072e8b040f207ac6b5c66be71bf9688a61318052c4a681940752&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=20773773&response-content-disposition=attachment%3B%20filename%3Dbazel-6.5.0-installer-linux-x86_64.sh&response-content-type=application%2Foctet-stream [following] --2024-05-23 14:28:19-- https://objects.githubusercontent.com/github-production-release-asset-2e65be/20773773/6f2926f6-5ad0-44ce-aeaf-071ad279408c?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=releaseassetproduction%2F20240523%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20240523T062819Z&X-Amz-Expires=300&X-Amz-Signature=f0ad97054bd8072e8b040f207ac6b5c66be71bf9688a61318052c4a681940752&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=20773773&response-content-disposition=attachment%3B%20filename%3Dbazel-6.5.0-installer-linux-x86_64.sh&response-content-type=application%2Foctet-stream Connecting to 10.10.60.103:7890... connected. Proxy request sent, awaiting response... 200 OK Length: 51137599 (49M) [application/octet-stream] Saving to: ‘bazel-6.5.0-installer-linux-x86_64.sh’
bazel-6.5.0-installer-linux-x86_64.sh 100%[==============================================================================================================================================>] 48.77M 6.87MB/s in 7.1s
2024-05-23 14:28:27 (6.86 MB/s) - ‘bazel-6.5.0-installer-linux-x86_64.sh’ saved [51137599/51137599]
Bazel installer
Bazel is bundled with software licensed under the GPLv2 with Classpath exception. You can find the sources next to the installer on our release page: https://github.com/bazelbuild/bazel/releases
#
Build information
Bazel is now installed!
Make sure you have "/home/susie.sun/CM/repos/local/cache/708c52c4acb04d9d/bin" in your path.
For bash completion, add the following line to your ~/.bashrc: source /home/susie.sun/CM/repos/local/cache/708c52c4acb04d9d/install/bin/bazel-complete.bash
For fish shell completion, link this file into your /home/susie.sun/.config/fish/completions/ directory: ln -s /home/susie.sun/CM/repos/local/cache/708c52c4acb04d9d/install/bin/bazel.fish /home/susie.sun/.config/fish/completions/bazel.fish
See http://bazel.build/docs/getting-started.html to start a new project! Bazel is installed to /home/susie.sun/CM/repos/local/cache/708c52c4acb04d9d ... ! cd /home/susie.sun/CM/repos/local/cache/20c75dc531d14fdf ! call /home/susie.sun/CM/repos/mlcommons@cm4mlops/script/get-bazel/run.sh from tmp-run.sh ! call "postprocess" from /home/susie.sun/CM/repos/mlcommons@cm4mlops/script/get-bazel/customize.py Detected version: 6.5.0 ! cd /home/susie.sun/CM/repos/local/cache/441bbe5c385e4ad8 ! call /home/susie.sun/CM/repos/mlcommons@cm4mlops/script/install-tensorflow-from-src/run.sh from tmp-run.sh Cloning Tensorflow from https://github.com/tensorflow/tensorflow with branch master --depth 1... Cloning into 'src'... remote: Enumerating objects: 31950, done. remote: Counting objects: 100% (31950/31950), done. remote: Compressing objects: 100% (24447/24447), done. remote: Total 31950 (delta 10675), reused 17399 (delta 6937), pack-reused 0 Receiving objects: 100% (31950/31950), 79.92 MiB | 6.71 MiB/s, done. Resolving deltas: 100% (10675/10675), done. Updating files: 100% (31915/31915), done.
/home/susie.sun/CM/repos/local/cache/708c52c4acb04d9d/install/bin/bazel build -c opt --define tflite_with_xnnpack=true //tensorflow/lite:libtensorflowlite.so Starting local Bazel server and connecting to it... INFO: Reading 'startup' options from /home/susie.sun/CM/repos/local/cache/441bbe5c385e4ad8/src/.bazelrc: --windows_enable_symlinks INFO: Options provided by the client: Inherited 'common' options: --isatty=1 --terminal_columns=245 INFO: Reading rc options for 'build' from /home/susie.sun/CM/repos/local/cache/441bbe5c385e4ad8/src/.bazelrc: Inherited 'common' options: --experimental_repo_remote_exec INFO: Reading rc options for 'build' from /home/susie.sun/CM/repos/local/cache/441bbe5c385e4ad8/src/.bazelrc: 'build' options: --define framework_shared_object=true --define tsl_protobuf_header_only=true --define=use_fast_cpp_protos=true --define=allow_oversize_protos=true --spawn_strategy=standalone -c opt --announce_rc --define=grpc_no_ares=true --noincompatible_remove_legacy_whole_archive --features=-force_no_whole_archive --enable_platform_specific_config --define=with_xla_support=true --config=short_logs --config=v2 --define=no_aws_support=true --define=no_hdfs_support=true --experimental_cc_shared_library --experimental_link_static_libraries_once=false --incompatible_enforce_config_setting_visibility INFO: Found applicable config definition build:short_logs in file /home/susie.sun/CM/repos/local/cache/441bbe5c385e4ad8/src/.bazelrc: --output_filter=DONT_MATCH_ANYTHING INFO: Found applicable config definition build:v2 in file /home/susie.sun/CM/repos/local/cache/441bbe5c385e4ad8/src/.bazelrc: --define=tf_api_version=2 --action_env=TF2_BEHAVIOR=1 INFO: Found applicable config definition build:linux in file /home/susie.sun/CM/repos/local/cache/441bbe5c385e4ad8/src/.bazelrc: --host_copt=-w --copt=-Wno-all --copt=-Wno-extra --copt=-Wno-deprecated --copt=-Wno-deprecated-declarations --copt=-Wno-ignored-attributes --copt=-Wno-array-bounds --copt=-Wunused-result --copt=-Werror=unused-result --copt=-Wswitch --copt=-Werror=switch --copt=-Wno-error=unused-but-set-variable --define=PREFIX=/usr --define=LIBDIR=$(PREFIX)/lib --define=INCLUDEDIR=$(PREFIX)/include --define=PROTOBUF_INCLUDE_PATH=$(PREFIX)/include --cxxopt=-std=c++17 --host_cxxopt=-std=c++17 --config=dynamic_kernels --experimental_guard_against_concurrent_changes INFO: Found applicable config definition build:dynamic_kernels in file /home/susie.sun/CM/repos/local/cache/441bbe5c385e4ad8/src/.bazelrc: --define=dynamic_loaded_kernels=true --copt=-DAUTOLOAD_DYNAMIC_KERNELS DEBUG: /home/susie.sun/CM/repos/local/cache/441bbe5c385e4ad8/src/tensorflow/tools/toolchains/python/python_repo.bzl:32:14: TF_PYTHON_VERSION environment variable was not set correctly; using Python 3.11.
To set Python version, run: export TF_PYTHON_VERSION=3.11 INFO: Analyzed target //tensorflow/lite:libtensorflowlite.so (102 packages loaded, 2755 targets configured). INFO: Found 1 target... ERROR: /home/susie.sun/.cache/bazel/_bazel_susie.sun/c4ee57410c5fc2d20a0ef98cfbbf1bde/external/XNNPACK/BUILD.bazel:1887:19: Compiling src/amalgam/gen/avxvnni.c failed: (Exit 1): gcc failed: error executing command (from target @XNNPACK//:avxvnni_prod_microkernels) /usr/bin/gcc -U_FORTIFY_SOURCE -fstack-protector -Wall -Wunused-but-set-parameter -Wno-free-nonheap-object -fno-omit-frame-pointer -g0 -O2 '-D_FORTIFY_SOURCE=1' -DNDEBUG -ffunction-sections ... (remaining 48 arguments skipped) gcc: error: unrecognized command line option '-mavxvnni'; did you mean '-mavx'? Target //tensorflow/lite:libtensorflowlite.so failed to build Use --verbose_failures to see the command lines of failed build steps. INFO: Elapsed time: 615.500s, Critical Path: 0.41s INFO: 351 processes: 327 internal, 24 local. FAILED: Build did NOT complete successfully
CM error: Portable CM script failed (name = install-tensorflow-from-src, return code = 256)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Note that it is often a portability issue of a third-party tool or a native script wrapped and unified by this CM script (automation recipe). Please re-run this script with --repro flag and report this issue with the original command line, cm-repro directory and full log here:
https://github.com/mlcommons/cm4mlops/issues
The CM concept is to collaboratively fix such issues inside portable CM scripts to make existing tools and native scripts more portable, interoperable and deterministic. Thank you!