BVLC / caffe

Caffe: a fast open framework for deep learning.
http://caffe.berkeleyvision.org/
Other
34.12k stars 18.68k forks source link

Solvers not registered during building? #4010

Open san-bil opened 8 years ago

san-bil commented 8 years ago

This is a similar problem to #3712 - namely matcaffe complaining at runtime that it doesn't know any concrete Solver types.

Caffe builds successfully with CUDA and the matlab wrapper, but when I try to call caffe.get_solver(some_prototxt_file) from Matlab, I'm getting an exception thrown with the error text:

F0417 22:03:48.503923 11578 solver_factory.hpp:76] Check failed: registry.count(type) == 1 (0 vs. 1) Unknown solver type: SGD (known types: )

This is obviously related to the issues raised in:

https://groups.google.com/forum/#!topic/caffe-users/TMuS7A_mccI http://stackoverflow.com/questions/32416226/create-layer-error-at-layer-factory-hpp https://groups.google.com/forum/#!topic/caffe-users/Py6IwMQvtqo/discussion http://stackoverflow.com/questions/30325108/caffe-layer-creation-failure

where the common factor seems to be the usage of Caffe as a statically linked lib rather than a dynamically linked lib. This is not true in my case...as confirmed by ldd, which shows the second run time dependency of the compiled mex-file to be libcaffe.so.1.0.0-rc3. Furthermore, I don't even have any *.a files in the entire caffe project folder.

Also, I can't see a SolverFactory or anything like LayerFactory, where different types of layer would need registering as in http://stackoverflow.com/questions/32416226/create-layer-error-at-layer-factory-hpp

I call cmake in relative path ./build with:

cmake .. -DCMAKE_INSTALL_PREFIX=$MINICONDA_ENV -DCUDA_TOOLKIT_ROOT_DIR=/vol/cuda/7.5.18 -DUSE_LMDB=0 -DUSE_CUDNN=0 -DUSE_LEVELDB=0 -DCPU_ONLY=0 -DBUILD_python=0 -DBUILD_docs=0 -DBLAS=atlas -DBUILD_matlab=1 && make -j6

This is the only thing I'm doing different from the installation tutorial (rather than using the Makefile.config)

SergioRAgostinho commented 8 years ago

So, I just hit the same problem while invoking

caffe.run_tests()

in Matlab. The "offending" line is this one. The content of the file passed in solver_file is similar to this.

net: "/tmp/tp5f07d5a6_88ae_4365_b574_d20a69a66765"
test_iter: 10 test_interval: 10 base_lr: 0.01 momentum: 0.9
weight_decay: 0.0005 lr_policy: "inv" gamma: 0.0001 power: 0.75
display: 100 max_iter: 100 snapshot_after_train: false

The stacktrace in matlab is

K>> dbstack
> In caffe.get_solver (line 7)
  In caffe.Solver (line 21)
  In caffe.test.test_solver (line 22)
  In caffe.run_tests (line 12)

going through here and eventually dying here. The registry is empty and the thing aborts with

...
I0721 18:32:09.077687 25952 net.cpp:283] Network initialization done.
F0721 18:34:51.517117 25952 solver_factory.hpp:76] Check failed: registry.count(type) == 1 (0 vs. 1) Unknown solver type: SGD (known types: )
*** Check failure stack trace: ***

The build options are below.

-- ******************* Caffe Configuration Summary *******************
-- General:
--   Version           :   1.0.0-rc3 
--   Git               :   rc3-216-g42cd785-dirty
--   System            :   Linux
--   C++ compiler      :   /usr/bin/c++
--   Release CXX flags :   -O3 -DNDEBUG -fPIC -Wall -Wno-sign-compare -Wno-uninitialized
--   Debug CXX flags   :   -g -fPIC -Wall -Wno-sign-compare -Wno-uninitialized
--   Build type        :   Release
-- 
--   BUILD_SHARED_LIBS :   ON
--   BUILD_python      :   ON
--   BUILD_matlab      :   ON
--   BUILD_docs        :   ON
--   CPU_ONLY          :   OFF
--   USE_OPENCV        :   ON
--   USE_LEVELDB       :   ON
--   USE_LMDB          :   ON
--   ALLOW_LMDB_NOLOCK :   OFF
-- 
-- Dependencies:
--   BLAS              :   Yes (Open)
--   Boost             :   Yes (ver. 1.55)
--   glog              :   Yes
--   gflags            :   Yes
--   protobuf          :   Yes (ver. 2.5.0)
--   lmdb              :   Yes (ver. 0.9.10)
--   LevelDB           :   Yes (ver. 1.15)
--   Snappy            :   Yes (ver. 1.1.0)
--   OpenCV            :   Yes (ver. 2.4.8)
--   CUDA              :   Yes (ver. 7.5)
-- 
-- NVIDIA CUDA:
--   Target GPU(s)     :   Auto
--   GPU arch(s)       :   sm_52
--   cuDNN             :   Yes (ver. 5.0.5)
-- 
-- Matlab:
--   Matlab            :   Yes (/opt/MATLAB/R2015a/bin/mex, /opt/MATLAB/R2015a/bin/mexext
--   Octave            :   No
-- 
-- Documentaion:
--   Doxygen           :   /usr/bin/doxygen (1.8.6)
--   config_file       :   /<omitted>/caffe/.Doxyfile
-- 
-- Install:
--   Install path      :   /usr/local
-- 
-- Configuring done
-- Generating done
-- Build files have been written to: /<omitted>/caffe/build

I was trying to see if I could patch this but this registry mechanism is somewhat cryptic at first sight. Any quick pointers on how and when do the solvers get registered?

SergioRAgostinho commented 8 years ago

Forgot to add that all my tests with make runtests are green.

ghostcow commented 8 years ago

I had a similar issue, and when i re-built caffe with Makefile.config it worked. I don't know exactly what's different, but before- ldd showed libcaffe.so linked to the caffe mex file, and now it is not, and the problem was solved:

With Cmake (libraries sorted alphabetically for clarity):

$ ldd caffe_.mexa64 
    /lib64/ld-linux-x86-64.so.2 (0x00007f8ae9f92000)
    libaudio.so.2 => /usr/lib/x86_64-linux-gnu/libaudio.so.2 (0x00007f8acd216000)
    libavcodec.so.54 => /usr/lib/x86_64-linux-gnu/libavcodec.so.54 (0x00007f8ad8138000)
    libavformat.so.54 => /usr/lib/x86_64-linux-gnu/libavformat.so.54 (0x00007f8ad7e16000)
    libavutil.so.52 => /usr/lib/x86_64-linux-gnu/libavutil.so.52 (0x00007f8ad7bf1000)
    libboost_python-py27.so.1.54.0 => /usr/lib/x86_64-linux-gnu/libboost_python-py27.so.1.54.0 (0x00007f8adc47b000)
    libboost_system.so.1.54.0 => /usr/lib/x86_64-linux-gnu/libboost_system.so.1.54.0 (0x00007f8ae9477000)
    libboost_thread.so.1.54.0 => /usr/lib/x86_64-linux-gnu/libboost_thread.so.1.54.0 (0x00007f8ae8749000)
    libbz2.so.1.0 => /lib/x86_64-linux-gnu/libbz2.so.1.0 (0x00007f8acdb17000)
    libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f8ae895f000)
    libcaffe.so => /home/lioruzan/caffe_LocNet/build/lib/libcaffe.so (0x00007f8ae967b000)
    libcgmanager.so.0 => /lib/x86_64-linux-gnu/libcgmanager.so.0 (0x00007f8acadcb000)
    libcublas.so.7.5 => /usr/local/cuda/lib64/libcublas.so.7.5 (0x00007f8ae2357000)
    libcudart.so.7.5 => /usr/local/cuda/lib64/libcudart.so.7.5 (0x00007f8ae749e000)
    libcudnn.so.4 => /usr/local/cuda/lib64/libcudnn.so.4 (0x00007f8ade604000)
    libcurand.so.7.5 => /usr/local/cuda/lib64/libcurand.so.7.5 (0x00007f8ae3c36000)
    libdbus-1.so.3 => /lib/x86_64-linux-gnu/libdbus-1.so.3 (0x00007f8aca764000)
    libdc1394.so.22 => /usr/lib/x86_64-linux-gnu/libdc1394.so.22 (0x00007f8ad92a0000)
    libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f8adbb50000)
    libexpat.so.1 => /lib/x86_64-linux-gnu/libexpat.so.1 (0x00007f8acb85b000)
    libffi.so.6 => /usr/lib/x86_64-linux-gnu/libffi.so.6 (0x00007f8ad1a20000)
    libfontconfig.so.1 => /usr/lib/x86_64-linux-gnu/libfontconfig.so.1 (0x00007f8acd42e000)
    libfreetype.so.6 => /usr/lib/x86_64-linux-gnu/libfreetype.so.6 (0x00007f8acd874000)
    libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f8ae8d24000)
    libgcrypt.so.11 => /lib/x86_64-linux-gnu/libgcrypt.so.11 (0x00007f8acbedb000)
    libgfortran.so.3 => /usr/lib/x86_64-linux-gnu/libgfortran.so.3 (0x00007f8ad5fd5000)
    libGL.so.1 => /usr/lib/x86_64-linux-gnu/libGL.so.1 (0x00007f8adb61a000)
    libglib-2.0.so.0 => /lib/x86_64-linux-gnu/libglib-2.0.so.0 (0x00007f8ad9d6d000)
    libglog.so.0 => /usr/local/lib/libglog.so.0 (0x00007f8ae923e000)
    libgmodule-2.0.so.0 => /usr/lib/x86_64-linux-gnu/libgmodule-2.0.so.0 (0x00007f8ad1f8f000)
    libgnutls.so.26 => /usr/lib/x86_64-linux-gnu/libgnutls.so.26 (0x00007f8acdd27000)
    libgobject-2.0.so.0 => /usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0 (0x00007f8ada075000)
    libgomp.so.1 => /usr/lib/x86_64-linux-gnu/libgomp.so.1 (0x00007f8ad5dc6000)
    libgpg-error.so.0 => /lib/x86_64-linux-gnu/libgpg-error.so.0 (0x00007f8aca55f000)
    libgsm.so.1 => /usr/lib/x86_64-linux-gnu/libgsm.so.1 (0x00007f8ace415000)
    libgstapp-0.10.so.0 => /usr/lib/x86_64-linux-gnu/libgstapp-0.10.so.0 (0x00007f8ad9945000)
    libgstaudio-0.10.so.0 => /usr/lib/x86_64-linux-gnu/libgstaudio-0.10.so.0 (0x00007f8ad10d4000)
    libgstbase-0.10.so.0 => /usr/lib/x86_64-linux-gnu/libgstbase-0.10.so.0 (0x00007f8ad158e000)
    libgstinterfaces-0.10.so.0 => /usr/lib/x86_64-linux-gnu/libgstinterfaces-0.10.so.0 (0x00007f8acc575000)
    libgstpbutils-0.10.so.0 => /usr/lib/x86_64-linux-gnu/libgstpbutils-0.10.so.0 (0x00007f8ad9514000)
    libgstreamer-0.10.so.0 => /usr/lib/x86_64-linux-gnu/libgstreamer-0.10.so.0 (0x00007f8ada2c6000)
    libgstriff-0.10.so.0 => /usr/lib/x86_64-linux-gnu/libgstriff-0.10.so.0 (0x00007f8ad9738000)
    libgsttag-0.10.so.0 => /usr/lib/x86_64-linux-gnu/libgsttag-0.10.so.0 (0x00007f8ad0e9d000)
    libgstvideo-0.10.so.0 => /usr/lib/x86_64-linux-gnu/libgstvideo-0.10.so.0 (0x00007f8ad9b51000)
    libHalf.so.6 => /usr/lib/x86_64-linux-gnu/libHalf.so.6 (0x00007f8ada5b0000)
    libhdf5.so.7 => /usr/lib/x86_64-linux-gnu/libhdf5.so.7 (0x00007f8ae7b5d000)
    libhdf5_hl.so.7 => /usr/lib/x86_64-linux-gnu/libhdf5_hl.so.7 (0x00007f8ae7ff9000)
    libICE.so.6 => /usr/lib/x86_64-linux-gnu/libICE.so.6 (0x00007f8accdf2000)
    libIex.so.6 => /usr/lib/x86_64-linux-gnu/libIex.so.6 (0x00007f8ad2399000)
    libIlmImf.so.6 => /usr/lib/x86_64-linux-gnu/libIlmImf.so.6 (0x00007f8ada7f3000)
    libIlmThread.so.6 => /usr/lib/x86_64-linux-gnu/libIlmThread.so.6 (0x00007f8ad2193000)
    libjasper.so.1 => /usr/lib/x86_64-linux-gnu/libjasper.so.1 (0x00007f8adaaa2000)
    libjbig.so.0 => /usr/lib/x86_64-linux-gnu/libjbig.so.0 (0x00007f8ad25b7000)
    libjpeg.so.8 => /usr/lib/x86_64-linux-gnu/libjpeg.so.8 (0x00007f8adb191000)
    libleveldb.so.1 => /usr/lib/x86_64-linux-gnu/libleveldb.so.1 (0x00007f8ae76fc000)
    liblmdb.so => /usr/local/lib/liblmdb.so (0x00007f8ae7949000)
    liblzma.so.5 => /lib/x86_64-linux-gnu/liblzma.so.5 (0x00007f8ad27c5000)
    libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f8adc175000)
    libmex.so => not found
    libmp3lame.so.0 => /usr/lib/x86_64-linux-gnu/libmp3lame.so.0 (0x00007f8ace623000)
    libmx.so => not found
    libnih-dbus.so.1 => /lib/x86_64-linux-gnu/libnih-dbus.so.1 (0x00007f8aca9a9000)
    libnih.so.1 => /lib/x86_64-linux-gnu/libnih.so.1 (0x00007f8acabb3000)
    libnvidia-glcore.so.352.41 => /usr/lib/x86_64-linux-gnu/libnvidia-glcore.so.352.41 (0x00007f8ad2f2e000)
    libnvidia-tls.so.352.41 => /usr/lib/x86_64-linux-gnu/tls/libnvidia-tls.so.352.41 (0x00007f8ad59c0000)
    libogg.so.0 => /usr/lib/x86_64-linux-gnu/libogg.so.0 (0x00007f8acc15b000)
    libopenblas.so.0 => /opt/OpenBLAS/lib/libopenblas.so.0 (0x00007f8adcc2c000)
    libopencv_core.so.2.4 => /usr/local/lib/libopencv_core.so.2.4 (0x00007f8ade161000)
    libopencv_highgui.so.2.4 => /usr/local/lib/libopencv_highgui.so.2.4 (0x00007f8addecb000)
    libopencv_imgproc.so.2.4 => /usr/local/lib/libopencv_imgproc.so.2.4 (0x00007f8add9cd000)
    libopenjpeg.so.2 => /usr/lib/x86_64-linux-gnu/libopenjpeg.so.2 (0x00007f8ace8b0000)
    libopus.so.0 => /usr/lib/x86_64-linux-gnu/libopus.so.0 (0x00007f8acead2000)
    liborc-0.4.so.0 => /usr/lib/x86_64-linux-gnu/liborc-0.4.so.0 (0x00007f8ad130c000)
    libp11-kit.so.0 => /usr/lib/x86_64-linux-gnu/libp11-kit.so.0 (0x00007f8acba85000)
    libpcre.so.3 => /lib/x86_64-linux-gnu/libpcre.so.3 (0x00007f8ad17e2000)
    libpng12.so.0 => /lib/x86_64-linux-gnu/libpng12.so.0 (0x00007f8adaf6b000)
    libprotobuf.so.8 => /usr/lib/x86_64-linux-gnu/libprotobuf.so.8 (0x00007f8ae8229000)
    libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f8ae852b000)
    libpython2.7.so.1.0 => /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0 (0x00007f8adc6c8000)
    libQtCore.so.4 => /usr/lib/x86_64-linux-gnu/libQtCore.so.4 (0x00007f8ad62ef000)
    libQtGui.so.4 => /usr/lib/x86_64-linux-gnu/libQtGui.so.4 (0x00007f8ad69f9000)
    libQtOpenGL.so.4 => /usr/lib/x86_64-linux-gnu/libQtOpenGL.so.4 (0x00007f8ad76ac000)
    libQtTest.so.4 => /usr/lib/x86_64-linux-gnu/libQtTest.so.4 (0x00007f8ad67d4000)
    libquadmath.so.0 => /usr/lib/x86_64-linux-gnu/libquadmath.so.0 (0x00007f8acc9a6000)
    libraw1394.so.11 => /usr/lib/x86_64-linux-gnu/libraw1394.so.11 (0x00007f8ad0c8f000)
    librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f8adbf6d000)
    librtmp.so.0 => /usr/lib/x86_64-linux-gnu/librtmp.so.0 (0x00007f8acdfe5000)
    libschroedinger-1.0.so.0 => /usr/lib/x86_64-linux-gnu/libschroedinger-1.0.so.0 (0x00007f8aced1a000)
    libSM.so.6 => /usr/lib/x86_64-linux-gnu/libSM.so.6 (0x00007f8acd00e000)
    libsnappy.so.1 => /usr/lib/libsnappy.so.1 (0x00007f8adb94a000)
    libspeex.so.1 => /usr/lib/x86_64-linux-gnu/libspeex.so.1 (0x00007f8acefde000)
    libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f8ae8f3a000)
    libswscale.so.2 => /usr/lib/x86_64-linux-gnu/libswscale.so.2 (0x00007f8ad79aa000)
    libtasn1.so.6 => /usr/lib/x86_64-linux-gnu/libtasn1.so.6 (0x00007f8acbcc7000)
    libtbb.so.2 => /usr/lib/libtbb.so.2 (0x00007f8adb3e6000)
    libtheoradec.so.1 => /usr/lib/x86_64-linux-gnu/libtheoradec.so.1 (0x00007f8acf1f7000)
    libtheoraenc.so.1 => /usr/lib/x86_64-linux-gnu/libtheoraenc.so.1 (0x00007f8acf410000)
    libtiff.so.5 => /usr/lib/x86_64-linux-gnu/libtiff.so.5 (0x00007f8adacf9000)
    libudev.so.1 => /lib/x86_64-linux-gnu/libudev.so.1 (0x00007f8acc364000)
    libusb-1.0.so.0 => /lib/x86_64-linux-gnu/libusb-1.0.so.0 (0x00007f8ad0a78000)
    libutil.so.1 => /lib/x86_64-linux-gnu/libutil.so.1 (0x00007f8ad5bc3000)
    libuuid.so.1 => /lib/x86_64-linux-gnu/libuuid.so.1 (0x00007f8acb1ec000)
    libv4l1.so.0 => /usr/lib/x86_64-linux-gnu/libv4l1.so.0 (0x00007f8ad909a000)
    libv4l2.so.0 => /usr/lib/x86_64-linux-gnu/libv4l2.so.0 (0x00007f8ad8e8c000)
    libv4lconvert.so.0 => /usr/lib/x86_64-linux-gnu/libv4lconvert.so.0 (0x00007f8ad07ff000)
    libva.so.1 => /usr/lib/x86_64-linux-gnu/libva.so.1 (0x00007f8ace1ff000)
    libvorbis.so.0 => /usr/lib/x86_64-linux-gnu/libvorbis.so.0 (0x00007f8acf650000)
    libvorbisenc.so.2 => /usr/lib/x86_64-linux-gnu/libvorbisenc.so.2 (0x00007f8acf87d000)
    libvpx.so.1 => /usr/lib/x86_64-linux-gnu/libvpx.so.1 (0x00007f8acfd4c000)
    libX11.so.6 => /usr/lib/x86_64-linux-gnu/libX11.so.6 (0x00007f8ad2bf9000)
    libx264.so.142 => /usr/lib/x86_64-linux-gnu/libx264.so.142 (0x00007f8ad012b000)
    libXau.so.6 => /usr/lib/x86_64-linux-gnu/libXau.so.6 (0x00007f8acb3f1000)
    libxcb.so.1 => /usr/lib/x86_64-linux-gnu/libxcb.so.1 (0x00007f8acc787000)
    libXdmcp.so.6 => /usr/lib/x86_64-linux-gnu/libXdmcp.so.6 (0x00007f8acafe6000)
    libXext.so.6 => /usr/lib/x86_64-linux-gnu/libXext.so.6 (0x00007f8ad29e7000)
    libXi.so.6 => /usr/lib/x86_64-linux-gnu/libXi.so.6 (0x00007f8accbe2000)
    libxml2.so.2 => /usr/lib/x86_64-linux-gnu/libxml2.so.2 (0x00007f8ad1c28000)
    libXrender.so.1 => /usr/lib/x86_64-linux-gnu/libXrender.so.1 (0x00007f8acd66a000)
    libXt.so.6 => /usr/lib/x86_64-linux-gnu/libXt.so.6 (0x00007f8acb5f5000)
    libxvidcore.so.4 => /usr/lib/x86_64-linux-gnu/libxvidcore.so.4 (0x00007f8ad04c1000)
    libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f8adbd54000)
    linux-vdso.so.1 =>  (0x00007ffc801dd000)

Without Cmake:

$ ldd caffe_.mexa64 
    /lib64/ld-linux-x86-64.so.2 (0x00007fcc30a12000)
    libaudio.so.2 => /usr/lib/x86_64-linux-gnu/libaudio.so.2 (0x00007fcc13c6f000)
    libavcodec.so.54 => /usr/lib/x86_64-linux-gnu/libavcodec.so.54 (0x00007fcc1eb91000)
    libavformat.so.54 => /usr/lib/x86_64-linux-gnu/libavformat.so.54 (0x00007fcc1e86f000)
    libavutil.so.52 => /usr/lib/x86_64-linux-gnu/libavutil.so.52 (0x00007fcc1e64a000)
    libboost_python-py27.so.1.54.0 => /usr/lib/x86_64-linux-gnu/libboost_python-py27.so.1.54.0 (0x00007fcc249d2000)
    libboost_system.so.1.54.0 => /usr/lib/x86_64-linux-gnu/libboost_system.so.1.54.0 (0x00007fcc2a5f0000)
    libboost_thread.so.1.54.0 => /usr/lib/x86_64-linux-gnu/libboost_thread.so.1.54.0 (0x00007fcc28c76000)
    libbz2.so.1.0 => /lib/x86_64-linux-gnu/libbz2.so.1.0 (0x00007fcc14570000)
    libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fcc22bce000)
    libcgmanager.so.0 => /lib/x86_64-linux-gnu/libcgmanager.so.0 (0x00007fcc11824000)
    libcublas.so.7.5 => /usr/local/cuda/lib64/libcublas.so.7.5 (0x00007fcc2e597000)
    libcudart.so.7.5 => /usr/local/cuda/lib64/libcudart.so.7.5 (0x00007fcc2fe76000)
    libcudnn.so.4 => /usr/local/cuda/lib64/libcudnn.so.4 (0x00007fcc24c1f000)
    libcurand.so.7.5 => /usr/local/cuda/lib64/libcurand.so.7.5 (0x00007fcc2ad2f000)
    libdbus-1.so.3 => /lib/x86_64-linux-gnu/libdbus-1.so.3 (0x00007fcc111bd000)
    libdc1394.so.22 => /usr/lib/x86_64-linux-gnu/libdc1394.so.22 (0x00007fcc1fcf9000)
    libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fcc229ca000)
    libexpat.so.1 => /lib/x86_64-linux-gnu/libexpat.so.1 (0x00007fcc122b4000)
    libffi.so.6 => /usr/lib/x86_64-linux-gnu/libffi.so.6 (0x00007fcc18479000)
    libfontconfig.so.1 => /usr/lib/x86_64-linux-gnu/libfontconfig.so.1 (0x00007fcc13e87000)
    libfreetype.so.6 => /usr/lib/x86_64-linux-gnu/libfreetype.so.6 (0x00007fcc142cd000)
    libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007fcc231b1000)
    libgcrypt.so.11 => /lib/x86_64-linux-gnu/libgcrypt.so.11 (0x00007fcc12934000)
    libgfortran.so.3 => /usr/lib/x86_64-linux-gnu/libgfortran.so.3 (0x00007fcc1c82b000)
    libGL.so.1 => /usr/lib/x86_64-linux-gnu/libGL.so.1 (0x00007fcc22073000)
    libglib-2.0.so.0 => /lib/x86_64-linux-gnu/libglib-2.0.so.0 (0x00007fcc207c6000)
    libglog.so.0 => /usr/local/lib/libglog.so.0 (0x00007fcc2aaf6000)
    libgmodule-2.0.so.0 => /usr/lib/x86_64-linux-gnu/libgmodule-2.0.so.0 (0x00007fcc189e8000)
    libgnutls.so.26 => /usr/lib/x86_64-linux-gnu/libgnutls.so.26 (0x00007fcc14780000)
    libgobject-2.0.so.0 => /usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0 (0x00007fcc20ace000)
    libgomp.so.1 => /usr/lib/x86_64-linux-gnu/libgomp.so.1 (0x00007fcc1c61c000)
    libgpg-error.so.0 => /lib/x86_64-linux-gnu/libgpg-error.so.0 (0x00007fcc10fb8000)
    libgsm.so.1 => /usr/lib/x86_64-linux-gnu/libgsm.so.1 (0x00007fcc14e6e000)
    libgstapp-0.10.so.0 => /usr/lib/x86_64-linux-gnu/libgstapp-0.10.so.0 (0x00007fcc2039e000)
    libgstaudio-0.10.so.0 => /usr/lib/x86_64-linux-gnu/libgstaudio-0.10.so.0 (0x00007fcc17b2d000)
    libgstbase-0.10.so.0 => /usr/lib/x86_64-linux-gnu/libgstbase-0.10.so.0 (0x00007fcc17fe7000)
    libgstinterfaces-0.10.so.0 => /usr/lib/x86_64-linux-gnu/libgstinterfaces-0.10.so.0 (0x00007fcc12fce000)
    libgstpbutils-0.10.so.0 => /usr/lib/x86_64-linux-gnu/libgstpbutils-0.10.so.0 (0x00007fcc1ff6d000)
    libgstreamer-0.10.so.0 => /usr/lib/x86_64-linux-gnu/libgstreamer-0.10.so.0 (0x00007fcc20d1f000)
    libgstriff-0.10.so.0 => /usr/lib/x86_64-linux-gnu/libgstriff-0.10.so.0 (0x00007fcc20191000)
    libgsttag-0.10.so.0 => /usr/lib/x86_64-linux-gnu/libgsttag-0.10.so.0 (0x00007fcc178f6000)
    libgstvideo-0.10.so.0 => /usr/lib/x86_64-linux-gnu/libgstvideo-0.10.so.0 (0x00007fcc205aa000)
    libHalf.so.6 => /usr/lib/x86_64-linux-gnu/libHalf.so.6 (0x00007fcc21009000)
    libhdf5.so.7 => /usr/lib/x86_64-linux-gnu/libhdf5.so.7 (0x00007fcc29f24000)
    libhdf5_hl.so.7 => /usr/lib/x86_64-linux-gnu/libhdf5_hl.so.7 (0x00007fcc2a3c0000)
    libICE.so.6 => /usr/lib/x86_64-linux-gnu/libICE.so.6 (0x00007fcc1384b000)
    libIex.so.6 => /usr/lib/x86_64-linux-gnu/libIex.so.6 (0x00007fcc18df2000)
    libIlmImf.so.6 => /usr/lib/x86_64-linux-gnu/libIlmImf.so.6 (0x00007fcc2124c000)
    libIlmThread.so.6 => /usr/lib/x86_64-linux-gnu/libIlmThread.so.6 (0x00007fcc18bec000)
    libjasper.so.1 => /usr/lib/x86_64-linux-gnu/libjasper.so.1 (0x00007fcc214fb000)
    libjbig.so.0 => /usr/lib/x86_64-linux-gnu/libjbig.so.0 (0x00007fcc19010000)
    libjpeg.so.8 => /usr/lib/x86_64-linux-gnu/libjpeg.so.8 (0x00007fcc21bea000)
    libleveldb.so.1 => /usr/lib/x86_64-linux-gnu/libleveldb.so.1 (0x00007fcc29cd7000)
    liblmdb.so => /usr/local/lib/liblmdb.so (0x00007fcc29ac3000)
    liblzma.so.5 => /lib/x86_64-linux-gnu/liblzma.so.5 (0x00007fcc1921e000)
    libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fcc233c7000)
    libmex.so => not found
    libmp3lame.so.0 => /usr/lib/x86_64-linux-gnu/libmp3lame.so.0 (0x00007fcc1507c000)
    libmx.so => not found
    libnih-dbus.so.1 => /lib/x86_64-linux-gnu/libnih-dbus.so.1 (0x00007fcc11402000)
    libnih.so.1 => /lib/x86_64-linux-gnu/libnih.so.1 (0x00007fcc1160c000)
    libnvidia-glcore.so.352.41 => /usr/lib/x86_64-linux-gnu/libnvidia-glcore.so.352.41 (0x00007fcc19987000)
    libnvidia-tls.so.352.41 => /usr/lib/x86_64-linux-gnu/tls/libnvidia-tls.so.352.41 (0x00007fcc1c419000)
    libogg.so.0 => /usr/lib/x86_64-linux-gnu/libogg.so.0 (0x00007fcc12bb4000)
    libopenblas.so.0 => /opt/OpenBLAS/lib/libopenblas.so.0 (0x00007fcc236cd000)
    libopencv_core.so.2.4 => /usr/local/lib/libopencv_core.so.2.4 (0x00007fcc29620000)
    libopencv_highgui.so.2.4 => /usr/local/lib/libopencv_highgui.so.2.4 (0x00007fcc2938a000)
    libopencv_imgproc.so.2.4 => /usr/local/lib/libopencv_imgproc.so.2.4 (0x00007fcc28e8c000)
    libopenjpeg.so.2 => /usr/lib/x86_64-linux-gnu/libopenjpeg.so.2 (0x00007fcc15309000)
    libopus.so.0 => /usr/lib/x86_64-linux-gnu/libopus.so.0 (0x00007fcc1552b000)
    liborc-0.4.so.0 => /usr/lib/x86_64-linux-gnu/liborc-0.4.so.0 (0x00007fcc17d65000)
    libp11-kit.so.0 => /usr/lib/x86_64-linux-gnu/libp11-kit.so.0 (0x00007fcc124de000)
    libpcre.so.3 => /lib/x86_64-linux-gnu/libpcre.so.3 (0x00007fcc1823b000)
    libpng12.so.0 => /lib/x86_64-linux-gnu/libpng12.so.0 (0x00007fcc219c4000)
    libprotobuf.so.8 => /usr/lib/x86_64-linux-gnu/libprotobuf.so.8 (0x00007fcc2a7f4000)
    libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fcc22f93000)
    libpython2.7.so.1.0 => /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0 (0x00007fcc2446e000)
    libQtCore.so.4 => /usr/lib/x86_64-linux-gnu/libQtCore.so.4 (0x00007fcc1cd48000)
    libQtGui.so.4 => /usr/lib/x86_64-linux-gnu/libQtGui.so.4 (0x00007fcc1d452000)
    libQtOpenGL.so.4 => /usr/lib/x86_64-linux-gnu/libQtOpenGL.so.4 (0x00007fcc1e105000)
    libQtTest.so.4 => /usr/lib/x86_64-linux-gnu/libQtTest.so.4 (0x00007fcc1d22d000)
    libquadmath.so.0 => /usr/lib/x86_64-linux-gnu/libquadmath.so.0 (0x00007fcc133ff000)
    libraw1394.so.11 => /usr/lib/x86_64-linux-gnu/libraw1394.so.11 (0x00007fcc176e8000)
    librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007fcc227c2000)
    librtmp.so.0 => /usr/lib/x86_64-linux-gnu/librtmp.so.0 (0x00007fcc14a3e000)
    libschroedinger-1.0.so.0 => /usr/lib/x86_64-linux-gnu/libschroedinger-1.0.so.0 (0x00007fcc15773000)
    libSM.so.6 => /usr/lib/x86_64-linux-gnu/libSM.so.6 (0x00007fcc13a67000)
    libsnappy.so.1 => /usr/lib/libsnappy.so.1 (0x00007fcc223a3000)
    libspeex.so.1 => /usr/lib/x86_64-linux-gnu/libspeex.so.1 (0x00007fcc15a37000)
    libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007fcc28972000)
    libswscale.so.2 => /usr/lib/x86_64-linux-gnu/libswscale.so.2 (0x00007fcc1e403000)
    libtasn1.so.6 => /usr/lib/x86_64-linux-gnu/libtasn1.so.6 (0x00007fcc12720000)
    libtbb.so.2 => /usr/lib/libtbb.so.2 (0x00007fcc21e3f000)
    libtheoradec.so.1 => /usr/lib/x86_64-linux-gnu/libtheoradec.so.1 (0x00007fcc15c50000)
    libtheoraenc.so.1 => /usr/lib/x86_64-linux-gnu/libtheoraenc.so.1 (0x00007fcc15e69000)
    libtiff.so.5 => /usr/lib/x86_64-linux-gnu/libtiff.so.5 (0x00007fcc21752000)
    libudev.so.1 => /lib/x86_64-linux-gnu/libudev.so.1 (0x00007fcc12dbd000)
    libusb-1.0.so.0 => /lib/x86_64-linux-gnu/libusb-1.0.so.0 (0x00007fcc174d1000)
    libutil.so.1 => /lib/x86_64-linux-gnu/libutil.so.1 (0x00007fcc1cb45000)
    libuuid.so.1 => /lib/x86_64-linux-gnu/libuuid.so.1 (0x00007fcc11c45000)
    libv4l1.so.0 => /usr/lib/x86_64-linux-gnu/libv4l1.so.0 (0x00007fcc1faf3000)
    libv4l2.so.0 => /usr/lib/x86_64-linux-gnu/libv4l2.so.0 (0x00007fcc1f8e5000)
    libv4lconvert.so.0 => /usr/lib/x86_64-linux-gnu/libv4lconvert.so.0 (0x00007fcc17258000)
    libva.so.1 => /usr/lib/x86_64-linux-gnu/libva.so.1 (0x00007fcc14c58000)
    libvorbis.so.0 => /usr/lib/x86_64-linux-gnu/libvorbis.so.0 (0x00007fcc160a9000)
    libvorbisenc.so.2 => /usr/lib/x86_64-linux-gnu/libvorbisenc.so.2 (0x00007fcc162d6000)
    libvpx.so.1 => /usr/lib/x86_64-linux-gnu/libvpx.so.1 (0x00007fcc167a5000)
    libX11.so.6 => /usr/lib/x86_64-linux-gnu/libX11.so.6 (0x00007fcc19652000)
    libx264.so.142 => /usr/lib/x86_64-linux-gnu/libx264.so.142 (0x00007fcc16b84000)
    libXau.so.6 => /usr/lib/x86_64-linux-gnu/libXau.so.6 (0x00007fcc11e4a000)
    libxcb.so.1 => /usr/lib/x86_64-linux-gnu/libxcb.so.1 (0x00007fcc131e0000)
    libXdmcp.so.6 => /usr/lib/x86_64-linux-gnu/libXdmcp.so.6 (0x00007fcc11a3f000)
    libXext.so.6 => /usr/lib/x86_64-linux-gnu/libXext.so.6 (0x00007fcc19440000)
    libXi.so.6 => /usr/lib/x86_64-linux-gnu/libXi.so.6 (0x00007fcc1363b000)
    libxml2.so.2 => /usr/lib/x86_64-linux-gnu/libxml2.so.2 (0x00007fcc18681000)
    libXrender.so.1 => /usr/lib/x86_64-linux-gnu/libXrender.so.1 (0x00007fcc140c3000)
    libXt.so.6 => /usr/lib/x86_64-linux-gnu/libXt.so.6 (0x00007fcc1204e000)
    libxvidcore.so.4 => /usr/lib/x86_64-linux-gnu/libxvidcore.so.4 (0x00007fcc16f1a000)
    libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007fcc225a9000)
    linux-vdso.so.1 =>  (0x00007fff9a1c1000)
SergioRAgostinho commented 8 years ago

So I'm tempted to assume that you're statically linking in the Makefile.config (without CMake) route you've just tested.

Nevertheless, it's definitely something on the linking options passed to the mex compiler.

rokm commented 8 years ago

I've hit the same issue with MatCaffe failing to find any solver, and consequently aborting with

solver_factory.hpp:76] Check failed: registry.count(type) == 1 (0 vs. 1) Unknown solver type: SGD (known types: )

When trying to use the SolverRegistry within the MEX file, one ends up with an empty CreatorList (e.g., SolverRegistry::SolverTypeList() will return an empty vector). And the same actually goes for the CreatorList of the LayerFactory as well...

Instrumenting the SolverRegistry methods with trace messages shows that when the MEX is loaded (and before the mexFunction() is actually called for the first time), the solvers are registered via AddCreator(), but the CreatorRegistry object used is different than the one used in the subsequent SolverRegistry::CreateSolver() call by the MEX itself - which is actually empty.

The issue is that a large part of Caffe's public API, including the solver and layer factory, is implemented as header templates, which end up instantiated in the client's compilation units. Now, the linker typically ends up removing the multiple instantiations, which is why, for example, the "caffe" program does not exhibit this issue.

However, the "mex" program passes -Wl,--version-script,$MATLAB_ROOT/extern/lib/glnxa64/mexFunction.map linker option, which causes all symbols but "mexFunction" to become local. Hence, we end up with a global & unique SolverRegistry::gregistry in libcaffe.so, which is used by template instantiations from the library itself (e.g., when the layers are registered via static variable initialization), as well as a locally-defined SolverRegistry::gregistry in a caffe_.mexa64, which is used only by template instantiations from the mexFunction itself (i.e., the SolverRegistry::CreateSolver() call).

This can be confirmed by inspecting the symbols:

$ nm libcaffe.so | grep g_registry_
0000000000a6abb0 u _ZGVZN5caffe13LayerRegistryIdE8RegistryB5cxx11EvE11g_registry_
0000000000a6abc0 u _ZGVZN5caffe13LayerRegistryIfE8RegistryB5cxx11EvE11g_registry_
0000000000a6ad60 u _ZGVZN5caffe14SolverRegistryIdE8RegistryB5cxx11EvE11g_registry_
0000000000a6ad70 u _ZGVZN5caffe14SolverRegistryIfE8RegistryB5cxx11EvE11g_registry_
0000000000a6abb8 u _ZZN5caffe13LayerRegistryIdE8RegistryB5cxx11EvE11g_registry_
0000000000a6abc8 u _ZZN5caffe13LayerRegistryIfE8RegistryB5cxx11EvE11g_registry_
0000000000a6ad68 u _ZZN5caffe14SolverRegistryIdE8RegistryB5cxx11EvE11g_registry_
0000000000a6ad78 u _ZZN5caffe14SolverRegistryIfE8RegistryB5cxx11EvE11g_registry_

(Note the "u" -> unique global symbol.)

The "caffe" binary:

$ nm caffe | grep g_registry_
000000000061a7d0 u _ZGVZN5caffe14SolverRegistryIfE8RegistryB5cxx11EvE11g_registry_
000000000061a7d8 u _ZZN5caffe14SolverRegistryIfE8RegistryB5cxx11EvE11g_registry_

(Again, unique global symbol, which is good.)

The "caffe_.mexa64":

$ nm caffe_.mexa64 | grep g_registry_
000000000020d8e0 b _ZGVZN5caffe14SolverRegistryIfE8RegistryB5cxx11EvE11g_registry_
000000000020d8e8 b _ZZN5caffe14SolverRegistryIfE8RegistryB5cxx11EvE11g_registry_

(Note the "b" -> a local copy in BSS!)

Therefore, probably the best way to resolve this issue is to hide the implementation of SolverRegistry::Registry(), i.e., move it from header to a .cpp file, and instantiate the class there.

PS: as mentioned at the beginning, there is a similar issue with the LayerRegistry, but it actually never surfaces when creating a network, because caffe::Net properly hides most of its implementation from the header, and therefore LayerRegistry template instantiations from libcaffe.so are used.

willyd commented 8 years ago

Moving LayerRegistry and SolverRegistry to .cpp files would be a good thing for the windows build too. Otherwise, it is impossible to build a shared library correctly.

rcorbish commented 7 years ago

If the appropriate SolverRegistry is instantiated it seems to work. I built a .so and could not use it, without the following

Add the following to your library code to instantiate the appropriate class

template class caffe::SolverRegistry<float>;

I added it to tools/caffe.cpp for latest release

Hope it saves someone some time

spark123456789 commented 7 years ago

@san-bil Hi, did you solve these problems as I may have the same problem after build caffe with VS2010. The program prompted "Check failed: registry.count(type) == 1 (0 vs. 1) Unknown solver type: SGD (known types: )" Thank you for you reply.

rajasrijan commented 7 years ago

is there any workaround this. I too have this problem.

rcorbish commented 7 years ago

I did add the above line into my library code. That initialized the registry. The above only happened creating a .so normal executables worked fine.

soulslicer commented 6 years ago

Matlab caffe is a goddam disaster

lizhongguo commented 6 years ago

https://groups.google.com/forum/#!msg/caffe-users/S0mhGLXFhCM/WHim8ix1FAAJ It works for me