search

dividiti / ck-caffe

Collective Knowledge workflow for Caffe to automate installation across diverse platforms and to collaboratively evaluate and optimize Caffe-based workloads across diverse hardware, software and data sets (compilers, libraries, tools, models, inputs):
http://cKnowledge.org
BSD 3-Clause "New" or "Revised" License
193 stars 40 forks source link

Missing shared object file when run caffe-classification #112

Open yzh89 opened 7 years ago

yzh89 commented 7 years ago

I followed the tutorial and finished installation of CK-caffe. compiling of classification has no errors, but when running, the system complain about missing .so file. The error file is attached below


export CK_DATASET_FILENAME=data.jpg
export CK_CAFFE_MODEL_WEIGHTS=/home/yzhou/CK_TOOLS/caffemodel-bvlc-googlenet-fast-mirror/bvlc_googlenet.caffemodel
export CK_CAFFE_BATCH_SIZE=1
export CK_ENV_MODEL_CAFFE_WEIGHTS=/home/yzhou/CK_TOOLS/caffemodel-bvlc-googlenet-fast-mirror/bvlc_googlenet.caffemodel
export CK_CAFFE_MODEL_MEAN_BIN=imagenet_mean.binaryproto

export CK_CAFFE_ITERATIONS=1
export CK_CAFFE_MODEL=/home/yzhou/CK_REPOS/ck-caffe/program/caffe-classification/tmp/tmp-DMbpcR.prototxt
export CK_CAFFE_MODEL_FILE=tmp-DMbpcR.prototxt

echo    executing code ...
 ./classification ${CK_CAFFE_MODEL_FILE} ${CK_CAFFE_MODEL_WEIGHTS} ../imagenet_mean.binaryproto ../synset_words.txt /home/yzhou/CK_REPOS/ctuning-datasets-min/dataset/image-jpeg-0001/data.jpg > tmp-output1.tmp 2> tmp-output2.tmp

***************************************************************************************
  (bash -c "chmod 755 ./tmp-agReEJ.sh; . ./tmp-agReEJ.sh")

  (sleep 0.5 sec ...)

  (run ...)
executing code ...

 (printing output files) 

    * tmp-output1.tmp

    * tmp-output2.tmp

      ./classification: error while loading shared libraries: libcaffe.so.1.0.0: cannot open shared object file: No such file or directory

  (post processing from script  /  ... )"

  (reading fine grain timers from tmp-ck-timer.json ...)

Program execution likely failed (can't find fine grain timers)!
psyhtest commented 7 years ago

yzh89 Thanks to our offline investigation, it is now clear what's going wrong.

CK sets up the path to the Caffe library as follows:

export CK_ENV_LIB_CAFFE_LIB=$CK_TOOLS/lib-caffe-bvlc-master-cpu-master-gcc-4.8.4-linux-64/install/lib

However, on your Ubuntu 14.04 platform the Caffe library gets installed under $CK_TOOLS/lib-caffe-bvlc-master-cpu-master-gcc-4.8.4-linux-64/install/lib/x86_64-linux, i.e. one level down in the x86_64-linux subdirectory.

This actually appears to be quite characteristic of your platform: even most system libraries are under /usr/lib/x86_64-linux-gnu, not under /usr/lib as is usual on other platforms. For example, here's a snippet from ldd $CK_REPOS/ck-caffe/program/caffe-classification/tmp/classification:

    linux-vdso.so.1 =>  (0x00007ffc9b5d5000)
    libcaffe.so.1.0.0 => /home/yzhou/CK_TOOLS/lib-caffe-bvlc-master-cpu-master-gcc-4.8.4-linux-64/install/lib/libcaffe.so.1.0.0 (0x00007f439c845000)
    libglog.so.0 => /usr/lib/x86_64-linux-gnu/libglog.so.0 (0x00007f439c60c000)
    libopencv_core.so.2.4 => /usr/lib/x86_64-linux-gnu/libopencv_core.so.2.4 (0x00007f439c1d5000)
    librtlxopenme.so => /home/yzhou/CK_TOOLS/lib-rtl-xopenme-0.3-gcc-4.8.4-linux-64/lib/librtlxopenme.so (0x00007f439bfd2000)
    libboost_thread.so.1.54.0 => /usr/lib/x86_64-linux-gnu/libboost_thread.so.1.54.0 (0x00007f439bdbb000)
    libboost_filesystem.so.1.54.0 => /usr/lib/x86_64-linux-gnu/libboost_filesystem.so.1.54.0 (0x00007f439bba5000)
    libboost_system.so.1.54.0 => /usr/lib/x86_64-linux-gnu/libboost_system.so.1.54.0 (0x00007f439b9a1000)
...

As I understand, the following quick workaround has worked for you:

$ cp $CK_TOOLS/lib-caffe-bvlc-master-cpu-master-gcc-4.8.4-linux-64/install/lib/x86_64-linux/* \
        $CK_TOOLS/lib-caffe-bvlc-master-cpu-master-gcc-4.8.4-linux-64/install/lib/

A proper fix is to make CK include correct paths into the environment script. We will look into this.

The next problem you encountered was with the system Boost 1.54.0 being used instead of the CK-installed Boost 1.62.0. This appears to be the same I've dealt with recently: https://github.com/dividiti/ck-caffe/issues/113. You need to reinstall Caffe e.g.

$ ck clean env --tags=lib,caffe,vcpu
$ ck pull all
$ ck install package:lib-caffe-bvlc-master-cpu-universal

(and apply the same workaround as above).