Collective Knowledge workflow for Caffe to automate installation across diverse platforms and to collaboratively evaluate and optimize Caffe-based workloads across diverse hardware, software and data sets (compilers, libraries, tools, models, inputs):
Updating deps based on selected command line key ...
Resolving software dependencies ...
*** Dependency 1 = caffemodel (Caffe model (net and weights)):
WARNING: no registered CK environment was found for "Caffe model (net and weights)" dependency with tags="caffemodel" and setup={"target_os_bits": "32", "host_os_uoa": "linux-32", "target_os_uoa": "linux-32"}
Trying to automatically detect required software ...
1) Checking if "Caffe model (net and weights)" (model.caffe / e2400aabc11d6cd1) is installed ...
Searching for Caffe model (net and weights) (deploy.prototxt) to automatically register in the CK - it may take some time, please wait ...
* Searching in /usr ...
* Searching in /opt ...
* Searching in /home/odroid ...
Search completed in 2.1 secs. Found 0 target files (may be pruned) ...
(warning during intermediate step: software was not automatically found on your system! Please, install it and re-try again!)
No software auto-detection scripts found for this software in CK :( ...
Checked following related CK soft entries:
* model.caffe
Searching and installing CK software packages ...
tags: caffemodel
or tags:
no tags:
More than one package or version found:
0) caffemodel-vgg19 Version deploy (70d16f21cc47d6a6)
1) caffemodel-vgg16 Version deploy (9511d37475281f84)
2) caffemodel-tidsp-jacintonet11-non-sparse Version trunk (90896a4ec3be86d7)
3) caffemodel-ssd-voc-512 Version trunk (01a873a305f644e7)
4) caffemodel-ssd-coco-512 Version trunk (110e9184758af0fd)
5) caffemodel-ssd-coco-300 Version trunk (7979c73da498fb1c)
6) caffemodel-ssd-512 Version trunk (b4e82b2a3da92441)
7) caffemodel-resnet50 Version fp32 (4b439b412770d1a6)
8) caffemodel-resnet152 Version deploy (947db9bba222a3a5)
9) caffemodel-resnet101 Version deploy (bba7bea86e5b9160)
10) caffemodel-mobilenet Version master (1e45deb4990b2f86)
11) caffemodel-inception-v3 Version fp32 (6aab061f186710ea)
12) caffemodel-deepscale-squeezenet-1.1 Version deepscale-squeezenet-1.1 (59414348b1bdedb5)
13) caffemodel-deepscale-squeezenet-1.0 Version deepscale-squeezenet-1.0 (5054a9abd7ee0db0)
14) caffemodel-bvlc-googlenet-fast-mirror Version trunk (b96fc6a959de0ad1)
15) caffemodel-bvlc-googlenet Version bvlc-googlenet (94ab0ab4f209a2fc)
16) caffemodel-bvlc-alexnet-fast-mirror Version trunk (29043e1b6a95bed6)
17) caffemodel-bvlc-alexnet Version bvlc-alexnet (b3a4a0c7bb464882)
Please select the package to install [ Enter -1 to use CK packages or hit return for "0" ]: 7
Calculating the hash of '/home/odroid/CK/local/env/f1227daa45e999e1/ResNet-50-model.caffemodel' ...
Validating the hash of '/home/odroid/CK/local/env/f1227daa45e999e1/ResNet-50-model.caffemodel' ...
Resolved. CK environment UID = f1227daa45e999e1 (version fp32)
*** Dependency 2 = lib-caffe (Caffe framework):
WARNING: no registered CK environment was found for "Caffe framework" dependency with tags="lib,caffe,vcpu" and setup={"target_os_bits": "32", "host_os_uoa": "linux-32", "target_os_uoa": "linux-32"}
Trying to automatically detect required software ...
No software auto-detection scripts found for this software in CK :( ...
Searching and installing CK software packages ...
tags: lib,caffe,vcpu
or tags:
no tags:
More than one package or version found:
0) lib-caffe-intel-request-cpu Version request (5fa2e216a5c46a8a)
1) lib-caffe-intel-master-cpu Version master (4f48b2e77da739cc)
2) lib-caffe-bvlc-opencl-cpu-universal Version trunk (outdated?, 9fa7bd782a74492d)
3) lib-caffe-bvlc-master-cpu-universal-20171013 Version master-20171013 (build stable Caffe CPU from master snapshot 20171013, af7cfba74b1ea1d6)
4) lib-caffe-bvlc-master-cpu-universal Version master (build Caffe CPU from master, b39f40abfd34f10b)
5) lib-caffe-ssd-cpu Version ssd (155b014d7af749ce)
Please select the package to install [ Enter -1 to use CK packages or hit return for "0" ]: 2
OS name: Ubuntu 16.04.7 LTS
Short OS name: Linux 4.14.5
Long OS name: Linux-4.14.5-92-armv7l-with-Ubuntu-16.04-xenial
OS bits: 32
OS ABI: armv7l
Platform init UOA: 34fe49aa49a49937
Number of logical processors: 8
CPU name: ODROID-XU4
CPU sub name: ARMv7 Processor rev 3 (v7l)
CPU ABI:
CPU features: half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt vfpd32 lpae
Resolved. CK environment UID = e4054a3f11728922 (detected version 3.10.2)
*** Dependency 2 = compiler (C++ compiler):
Resolved. CK environment UID = 7980dd11c2e84d5b (detected version 5.4.0)
*** Dependency 3 = lib-gflags (GFlags library):
Resolved. CK environment UID = 2a2fb9855cf64c02 (version 2.2.0)
*** Dependency 4 = lib-glog (GLog library):
Resolved. CK environment UID = 6ec0b477fa282753 (version 0.3.5)
*** Dependency 5 = lib-lmdb (LMDB library):
Resolved. CK environment UID = 6f58f2941571416e (version 0.9.21-60d5002)
*** Dependency 6 = lib-hdf5 (HDF5 library):
Resolved. CK environment UID = 4ede852c895d2f59 (version 1.10.1)
*** Dependency 7 = lib-opencv (OpenCV library):
Resolved. CK environment UID = 539cd5436187fbdb (version 3.3.0)
*** Dependency 8 = lib-boost (Boost library):
WARNING: no registered CK environment was found for "Boost library" dependency with tags="lib,boost" and setup={"target_os_bits": "32", "host_os_uoa": "linux-32", "target_os_uoa": "linux-32"} and version constraints ([1, 64, 0] <= v <= [])
Searching and installing CK software packages ...
tags: lib,boost
or tags:
no tags:
More than one package or version found:
0) lib-boost-1.67.0-min-for-caffe Version 1.67.0-min-for-caffe (fa18879866cd3e97)
1) lib-boost-1.67.0 Version 1.67.0 (21a32c9591557339)
2) lib-boost-1.66.0-min-for-caffe Version 1.66.0-min-for-caffe (04254b0e8fd053ef)
3) lib-boost-1.66.0 Version 1.66.0 (0417ba98aa5ca4ad)
4) lib-boost-1.65.1-min-for-caffe Version 1.65.1-min-for-caffe (e8e0c4ee4898b84b)
5) lib-boost-1.65.1 Version 1.65.1 (8bb8a5b19b48120c)
6) lib-boost-1.64.0-min-for-caffe Version 1.64.0-min-for-caffe (cb7aa7d5bbd74d94)
7) lib-boost-1.64.0 Version 1.64.0 (c6a6991d5c951d20)
Please select the package to install [ Enter -1 to use CK packages or hit return for "0" ]: 0
CK detected a PROBLEM in the third-party CK package:
CK package: lib-boost-1.67.0-min-for-caffe
Native package URL: https://dl.bintray.com/boostorg/release/1.67.0/source
CK repo: ck-math
CK repo URL: https://github.com/ctuning/ck-math
CK package URL: https://github.com/ctuning/ck-math/tree/master/package/lib-boost-1.67.0-min-for-caffe
Issues URL: https://github.com/ctuning/ck-math/issues
CK stable package URL: https://cKnowledge.io/c/package/lib-boost-1.67.0-min-for-caffe
Please, check that there is no discussion about this issue at https://cKnowledge.io/c/package/lib-boost-1.67.0-min-for-caffe
Please, submit the original comand line and the *full* log to:
* https://github.com/ctuning/ck/issues
* https://github.com/ctuning/ck-mlops/issues
CK detected a potential PROBLEM in the third-party CK program workflow:
CK component: program:caffe
CK repo: ck-caffe
CK repo URL: https://github.com/dividiti/ck-caffe
CK program URL: https://github.com/dividiti/ck-caffe/tree/master/program/caffe
Please, check that there is no related discussion about this issue at:
Command -sudo ck run program:caffe
OS CK UOA: linux-32 (fb525e34243f49c1)
OS name: Ubuntu 16.04.7 LTS Short OS name: Linux 4.14.5 Long OS name: Linux-4.14.5-92-armv7l-with-Ubuntu-16.04-xenial OS bits: 32 OS ABI: armv7l
Platform init UOA: 34fe49aa49a49937
Current directory: /home/odroid/CK/ck-caffe/program/caffe/tmp
More than one commmand line is found to run this program:
0) test_gpu_fp16 ($<>$ test --model=$<>$ --weights=$<>$ --iterations=$<>$ --gpu=$<>$)
1) query_gpu_cuda ($<>$ device_query --gpu=$<>$)
2) time_gpu ($<>$ time --model=$<>$ --gpu=$<>$ --iterations=1)
3) classify_ck_ai_api ($<>$ $<>$ $<>$ $#up_dir#$for_classification_demo/imagenet_mean.binaryproto $#up_dir#$for_classification_demo/synset_words.txt $<>$)
4) time_cpu ($<>$ time --model=$<>$ --iterations=1)
5) time_gpu_fp16 ($<>$ time --model=$<>$ --gpu=$<>$ --iterations=1)
6) query_gpu_opencl ($<>$ device_query)
7) test_gpu ($<>$ test --model=$<>$ --weights=$<>$ --iterations=$<>$ --gpu=$<>$)
8) classify ($<>$ $<>$ $<>$ $#up_dir#$for_classification_demo/imagenet_mean.binaryproto $#up_dir#$for_classification_demo/synset_words.txt $#dataset_path#$$#dataset_filename#$)
9) test_cpu ($<>$ test --model=$<>$ --weights=$<>$ --iterations=$<>$)
Select command line (or Enter to select 0): 4
Updating deps based on selected command line key ...
Resolving software dependencies ...
*** Dependency 1 = caffemodel (Caffe model (net and weights)):
WARNING: no registered CK environment was found for "Caffe model (net and weights)" dependency with tags="caffemodel" and setup={"target_os_bits": "32", "host_os_uoa": "linux-32", "target_os_uoa": "linux-32"}
Trying to automatically detect required software ...
1) Checking if "Caffe model (net and weights)" (model.caffe / e2400aabc11d6cd1) is installed ...
Searching for Caffe model (net and weights) (deploy.prototxt) to automatically register in the CK - it may take some time, please wait ...
Search completed in 2.1 secs. Found 0 target files (may be pruned) ... (warning during intermediate step: software was not automatically found on your system! Please, install it and re-try again!) No software auto-detection scripts found for this software in CK :( ...
Searching and installing CK software packages ...
More than one package or version found:
0) caffemodel-vgg19 Version deploy (70d16f21cc47d6a6) 1) caffemodel-vgg16 Version deploy (9511d37475281f84) 2) caffemodel-tidsp-jacintonet11-non-sparse Version trunk (90896a4ec3be86d7) 3) caffemodel-ssd-voc-512 Version trunk (01a873a305f644e7) 4) caffemodel-ssd-coco-512 Version trunk (110e9184758af0fd) 5) caffemodel-ssd-coco-300 Version trunk (7979c73da498fb1c) 6) caffemodel-ssd-512 Version trunk (b4e82b2a3da92441) 7) caffemodel-resnet50 Version fp32 (4b439b412770d1a6) 8) caffemodel-resnet152 Version deploy (947db9bba222a3a5) 9) caffemodel-resnet101 Version deploy (bba7bea86e5b9160) 10) caffemodel-mobilenet Version master (1e45deb4990b2f86) 11) caffemodel-inception-v3 Version fp32 (6aab061f186710ea) 12) caffemodel-deepscale-squeezenet-1.1 Version deepscale-squeezenet-1.1 (59414348b1bdedb5) 13) caffemodel-deepscale-squeezenet-1.0 Version deepscale-squeezenet-1.0 (5054a9abd7ee0db0) 14) caffemodel-bvlc-googlenet-fast-mirror Version trunk (b96fc6a959de0ad1) 15) caffemodel-bvlc-googlenet Version bvlc-googlenet (94ab0ab4f209a2fc) 16) caffemodel-bvlc-alexnet-fast-mirror Version trunk (29043e1b6a95bed6) 17) caffemodel-bvlc-alexnet Version bvlc-alexnet (b3a4a0c7bb464882)
Please select the package to install [ Enter -1 to use CK packages or hit return for "0" ]: 7
Package found: caffemodel-resnet50 (4b439b412770d1a6)
Searching if CK environment for this package already exists using:
CK environment not found for this package ...
Installing to /home/odroid/CK/local/env/f1227daa45e999e1
Copying net topology files to '/home/odroid/CK/local/env/f1227daa45e999e1' ...
Checking whether '/home/odroid/CK/local/env/f1227daa45e999e1/ResNet-50-model.caffemodel' already exists ...
Downloading the weights from 'https://deepdetect.com/models/resnet/ResNet-50-model.caffemodel' ... --2021-04-03 05:02:29-- https://deepdetect.com/models/resnet/ResNet-50-model.caffemodel Resolving deepdetect.com (deepdetect.com)... 163.172.82.48 Connecting to deepdetect.com (deepdetect.com)|163.172.82.48|:443... connected. WARNING: cannot verify deepdetect.com's certificate, issued by ‘CN=R3,O=Let's Encrypt,C=US’: Issued certificate not yet valid. HTTP request sent, awaiting response... 301 Moved Permanently Location: https://www.deepdetect.com/models/resnet/ResNet-50-model.caffemodel [following] --2021-04-03 05:02:31-- https://www.deepdetect.com/models/resnet/ResNet-50-model.caffemodel Resolving www.deepdetect.com (www.deepdetect.com)... 163.172.82.48 Connecting to www.deepdetect.com (www.deepdetect.com)|163.172.82.48|:443... connected. WARNING: cannot verify www.deepdetect.com's certificate, issued by ‘CN=R3,O=Let's Encrypt,C=US’: Issued certificate not yet valid. HTTP request sent, awaiting response... 200 OK Length: 102462397 (98M) [application/octet-stream] Saving to: ‘/home/odroid/CK/local/env/f1227daa45e999e1/ResNet-50-model.caffemodel’
/home/odroid/CK/local/env/f12 100%[=================================================>] 97.71M 644KB/s in 6m 20s
2021-04-03 05:08:52 (263 KB/s) - ‘/home/odroid/CK/local/env/f1227daa45e999e1/ResNet-50-model.caffemodel’ saved [102462397/102462397]
Calculating the hash of '/home/odroid/CK/local/env/f1227daa45e999e1/ResNet-50-model.caffemodel' ... Validating the hash of '/home/odroid/CK/local/env/f1227daa45e999e1/ResNet-50-model.caffemodel' ...
Installed =/home/odroid/CK/local/env/f1227daa45e999e1/ResNet-50-model.caffemodel
Setting up environment for installed package ... (full path = /home/odroid/CK/local/env/f1227daa45e999e1/ResNet-50-model.caffemodel)
Software entry found: model.caffe (e2400aabc11d6cd1)
Environment entry updated (f1227daa45e999e1)!
Recording CK configuration to /home/odroid/CK/local/env/f1227daa45e999e1/ck-install.json ...
Installation path: /home/odroid/CK/local/env/f1227daa45e999e1
Installation time: 388.637087107 sec.
*** Dependency 2 = lib-caffe (Caffe framework):
WARNING: no registered CK environment was found for "Caffe framework" dependency with tags="lib,caffe,vcpu" and setup={"target_os_bits": "32", "host_os_uoa": "linux-32", "target_os_uoa": "linux-32"}
Trying to automatically detect required software ... No software auto-detection scripts found for this software in CK :( ...
Searching and installing CK software packages ...
More than one package or version found:
0) lib-caffe-intel-request-cpu Version request (5fa2e216a5c46a8a) 1) lib-caffe-intel-master-cpu Version master (4f48b2e77da739cc) 2) lib-caffe-bvlc-opencl-cpu-universal Version trunk (outdated?, 9fa7bd782a74492d) 3) lib-caffe-bvlc-master-cpu-universal-20171013 Version master-20171013 (build stable Caffe CPU from master snapshot 20171013, af7cfba74b1ea1d6) 4) lib-caffe-bvlc-master-cpu-universal Version master (build Caffe CPU from master, b39f40abfd34f10b) 5) lib-caffe-ssd-cpu Version ssd (155b014d7af749ce)
Please select the package to install [ Enter -1 to use CK packages or hit return for "0" ]: 2
Package found: lib-caffe-bvlc-opencl-cpu-universal (9fa7bd782a74492d)
OS CK UOA: linux-32 (fb525e34243f49c1)
OS name: Ubuntu 16.04.7 LTS Short OS name: Linux 4.14.5 Long OS name: Linux-4.14.5-92-armv7l-with-Ubuntu-16.04-xenial OS bits: 32 OS ABI: armv7l
Platform init UOA: 34fe49aa49a49937
Number of logical processors: 8 CPU name: ODROID-XU4 CPU sub name: ARMv7 Processor rev 3 (v7l) CPU ABI: CPU features: half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt vfpd32 lpae
CPU frequency: CPU0 = 1400.0 MHz CPU1 = 1400.0 MHz CPU2 = 1400.0 MHz CPU3 = 1400.0 MHz CPU4 = 2000.0 MHz CPU5 = 2000.0 MHz CPU6 = 2000.0 MHz CPU7 = 2000.0 MHz CPU max frequency: CPU0 = 1400.0 MHz CPU1 = 1400.0 MHz CPU2 = 1400.0 MHz CPU3 = 1400.0 MHz CPU4 = 2000.0 MHz CPU5 = 2000.0 MHz CPU6 = 2000.0 MHz CPU7 = 2000.0 MHz CPU all frequencies (Hz): CPU0 = [200000, 300000, 400000, 500000, 600000, 700000, 800000, 900000, 1000000, 1100000, 1200000, 1300000, 1400000] CPU1 = [200000, 300000, 400000, 500000, 600000, 700000, 800000, 900000, 1000000, 1100000, 1200000, 1300000, 1400000] CPU2 = [200000, 300000, 400000, 500000, 600000, 700000, 800000, 900000, 1000000, 1100000, 1200000, 1300000, 1400000] CPU3 = [200000, 300000, 400000, 500000, 600000, 700000, 800000, 900000, 1000000, 1100000, 1200000, 1300000, 1400000] CPU4 = [200000, 300000, 400000, 500000, 600000, 700000, 800000, 900000, 1000000, 1100000, 1200000, 1300000, 1400000, 1500000, 1600000, 1700000, 1800000, 1900000, 2000000] CPU5 = [200000, 300000, 400000, 500000, 600000, 700000, 800000, 900000, 1000000, 1100000, 1200000, 1300000, 1400000, 1500000, 1600000, 1700000, 1800000, 1900000, 2000000] CPU6 = [200000, 300000, 400000, 500000, 600000, 700000, 800000, 900000, 1000000, 1100000, 1200000, 1300000, 1400000, 1500000, 1600000, 1700000, 1800000, 1900000, 2000000] CPU7 = [200000, 300000, 400000, 500000, 600000, 700000, 800000, 900000, 1000000, 1100000, 1200000, 1300000, 1400000, 1500000, 1600000, 1700000, 1800000, 1900000, 2000000]
Detecting GPGPU type: opencl
Features (properties) in JSON:
{ "gpgpu": { "vendor": "ARM", "type": "opencl", "name": "Mali-T628" }, "gpgpu_misc": { "address bits": "64", "unified memory": "yes", "- max work-item size #1": "256", "- max work-item size #0": "256", "opencl c version": "OpenCL C 1.2 v1.r17p0-01rel0.a881d28363cdb20f0017ed13c980967e", "- max work-item size #2": "256", "software (driver) version": "1.2", "work-item dimensions": "3", "hardware (device) version": "OpenCL 1.2 v1.r17p0-01rel0.a881d28363cdb20f0017ed13c980967e", "parallel compute units": "4" }, "gpgpu_id": { "gpgpu_device_id": "0", "gpgpu_platform_id": "0" } }
Features (properties) in JSON:
{ "gpgpu": { "vendor": "ARM", "type": "opencl", "name": "Mali-T628" }, "gpgpu_misc": { "address bits": "64", "unified memory": "yes", "- max work-item size #1": "256", "- max work-item size #0": "256", "opencl c version": "OpenCL C 1.2 v1.r17p0-01rel0.a881d28363cdb20f0017ed13c980967e", "- max work-item size #2": "256", "software (driver) version": "1.2", "work-item dimensions": "3", "hardware (device) version": "OpenCL 1.2 v1.r17p0-01rel0.a881d28363cdb20f0017ed13c980967e", "parallel compute units": "2" }, "gpgpu_id": { "gpgpu_device_id": "1", "gpgpu_platform_id": "0" } }
Resolving software dependencies ...
*** Dependency 1 = cmake (cmake):
*** Dependency 2 = compiler (C++ compiler):
*** Dependency 3 = lib-gflags (GFlags library):
*** Dependency 4 = lib-glog (GLog library):
*** Dependency 5 = lib-lmdb (LMDB library):
*** Dependency 6 = lib-hdf5 (HDF5 library):
*** Dependency 7 = lib-opencv (OpenCV library):
*** Dependency 8 = lib-boost (Boost library):
WARNING: no registered CK environment was found for "Boost library" dependency with tags="lib,boost" and setup={"target_os_bits": "32", "host_os_uoa": "linux-32", "target_os_uoa": "linux-32"} and version constraints ([1, 64, 0] <= v <= [])
Searching and installing CK software packages ...
More than one package or version found:
0) lib-boost-1.67.0-min-for-caffe Version 1.67.0-min-for-caffe (fa18879866cd3e97) 1) lib-boost-1.67.0 Version 1.67.0 (21a32c9591557339) 2) lib-boost-1.66.0-min-for-caffe Version 1.66.0-min-for-caffe (04254b0e8fd053ef) 3) lib-boost-1.66.0 Version 1.66.0 (0417ba98aa5ca4ad) 4) lib-boost-1.65.1-min-for-caffe Version 1.65.1-min-for-caffe (e8e0c4ee4898b84b) 5) lib-boost-1.65.1 Version 1.65.1 (8bb8a5b19b48120c) 6) lib-boost-1.64.0-min-for-caffe Version 1.64.0-min-for-caffe (cb7aa7d5bbd74d94) 7) lib-boost-1.64.0 Version 1.64.0 (c6a6991d5c951d20)
Please select the package to install [ Enter -1 to use CK packages or hit return for "0" ]: 0
Package found: lib-boost-1.67.0-min-for-caffe (fa18879866cd3e97)
Resolving software dependencies ...
*** Dependency 1 = cmake (cmake):
*** Dependency 2 = compiler (C++ compiler):
Searching if CK environment for this package already exists using:
CK environment found for this package: 3a44a68809320a99
Resolving software dependencies ...
*** Dependency 1 = cmake (cmake):
*** Dependency 2 = compiler (C++ compiler):
Installing to /home/odroid/CK/local/env/3a44a68809320a99
Using scripts from /home/odroid/CK/ck-math/package/lib-boost-1.62.0
Downloading package from 'https://dl.bintray.com/boostorg/release/1.67.0/source/boost_1_67_0.zip' ... --2021-04-03 05:13:17-- https://dl.bintray.com/boostorg/release/1.67.0/source/boost_1_67_0.zip Resolving dl.bintray.com (dl.bintray.com)... 54.148.26.147, 35.161.110.43 Connecting to dl.bintray.com (dl.bintray.com)|54.148.26.147|:443... connected. HTTP request sent, awaiting response... 403 Forbidden 2021-04-03 05:13:19 ERROR 403: Forbidden.
Error: downloading package failed!
Please, check that there is no related discussion about this issue at:
https://cKnowledge.io/c/program/caffe
Please, submit the original command line and the full log to:
https://github.com/dividiti/ck-caffe/issues
Make the CK community aware of this problem by referencing this ticket at:
CK error: [program] package installation failed!