search

facebookarchive / caffe2

Caffe2 is a lightweight, modular, and scalable deep learning framework.
https://caffe2.ai
Apache License 2.0
8.42k stars 1.94k forks source link

caffe2 expects future glog version #1753

Closed AliengirlLiv closed 6 years ago

AliengirlLiv commented 6 years ago

I installed caffe2 without any errors, but when I tried to import caffe2 in python, I get this error:

CRITICAL:root:Cannot load caffe2.python. Error: dlopen(caffe2/python/caffe2_pybind11_state.so, 10): Library not loaded: /usr/local/opt/glog/lib/libglog.0.dylib
  Referenced from: /Users/cslaptop/Downloads/caffe2/build/caffe2/python/caffe2_pybind11_state.so
  Reason: Incompatible library version: caffe2_pybind11_state.so requires version 1.0.0 or later, but libglog.0.dylib provides version 0.3.5

However, glog does not have 1.0.0 version - 0.3.5 appears to be the most recent version.

System information

CMake summary output

******** Summary ********
-- Need to define long as a separate typeid.
-- Current compiler supports avx2 extention. Will build perfkernels.
-- The BLAS backend of choice:Eigen
-- Could NOT find NNPACK (missing: CPUINFO_LIBRARY) 
-- Brace yourself, we are building NNPACK
-- Found PythonInterp: /Users/cslaptop/miniconda3/bin/python (found version "3.6.3") 
-- Configuring done
-- Generating done
-- Build files have been written to: /Users/cslaptop/Downloads/caffe2/build/six-download
[100%] Built target six
-- Configuring done
-- Generating done
-- Build files have been written to: /Users/cslaptop/Downloads/caffe2/build/opcodes-download
[100%] Built target opcodes
-- Configuring done
-- Generating done
-- Build files have been written to: /Users/cslaptop/Downloads/caffe2/build/peachpy-download
[ 11%] Performing update step for 'peachpy'
Current branch master is up to date.
[ 22%] No configure step for 'peachpy'
[ 33%] No build step for 'peachpy'
[ 44%] No install step for 'peachpy'
[ 55%] No test step for 'peachpy'
[ 66%] Completed 'peachpy'
[100%] Built target peachpy
-- Configuring done
-- Generating done
-- Build files have been written to: /Users/cslaptop/Downloads/caffe2/build/fp16-download
[ 11%] Performing update step for 'fp16'
Current branch master is up to date.
[ 22%] No configure step for 'fp16'
[ 33%] No build step for 'fp16'
[ 44%] No install step for 'fp16'
[ 55%] No test step for 'fp16'
[ 66%] Completed 'fp16'
[100%] Built target fp16
-- Configuring done
-- Generating done
-- Build files have been written to: /Users/cslaptop/Downloads/caffe2/build/fxdiv-download
[ 11%] Performing update step for 'fxdiv'
Current branch master is up to date.
[ 22%] No configure step for 'fxdiv'
[ 33%] No build step for 'fxdiv'
[ 44%] No install step for 'fxdiv'
[ 55%] No test step for 'fxdiv'
[ 66%] Completed 'fxdiv'
[100%] Built target fxdiv
-- Configuring done
-- Generating done
-- Build files have been written to: /Users/cslaptop/Downloads/caffe2/build/psimd-download
[ 11%] Performing update step for 'psimd'
Current branch master is up to date.
[ 22%] No configure step for 'psimd'
[ 33%] No build step for 'psimd'
[ 44%] No install step for 'psimd'
[ 55%] No test step for 'psimd'
[ 66%] Completed 'psimd'
[100%] Built target psimd
-- Configuring done
-- Generating done
-- Build files have been written to: /Users/cslaptop/Downloads/caffe2/build/pthreadpool-download
[ 11%] Performing update step for 'pthreadpool'
Current branch master is up to date.
[ 22%] No configure step for 'pthreadpool'
[ 33%] No build step for 'pthreadpool'
[ 44%] No install step for 'pthreadpool'
[ 55%] No test step for 'pthreadpool'
[ 66%] Completed 'pthreadpool'
[100%] Built target pthreadpool
-- Configuring done
-- Generating done
-- Build files have been written to: /Users/cslaptop/Downloads/caffe2/build/googletest-download
[100%] Built target googletest
-- Configuring done
-- Generating done
-- Build files have been written to: /Users/cslaptop/Downloads/caffe2/build/fxdiv-download
[ 11%] Performing update step for 'fxdiv'
Current branch master is up to date.
[ 22%] No configure step for 'fxdiv'
[ 33%] No build step for 'fxdiv'
[ 44%] No install step for 'fxdiv'
[ 55%] No test step for 'fxdiv'
[ 66%] Completed 'fxdiv'
[100%] Built target fxdiv
-- Configuring done
-- Generating done
-- Build files have been written to: /Users/cslaptop/Downloads/caffe2/build/googletest-download
[100%] Built target googletest
-- Configuring done
-- Generating done
-- Build files have been written to: /Users/cslaptop/Downloads/caffe2/build/googlebenchmark-download
[100%] Built target googlebenchmark
-- Configuring done
-- Generating done
-- Build files have been written to: /Users/cslaptop/Downloads/caffe2/build/googletest-download
[100%] Built target googletest
-- Configuring done
-- Generating done
-- Build files have been written to: /Users/cslaptop/Downloads/caffe2/build/googlebenchmark-download
[100%] Built target googlebenchmark
-- Configuring done
-- Generating done
-- Build files have been written to: /Users/cslaptop/Downloads/caffe2/build/googletest-download
[100%] Built target googletest
-- Configuring done
-- Generating done
-- Build files have been written to: /Users/cslaptop/Downloads/caffe2/build/googlebenchmark-download
[100%] Built target googlebenchmark
-- Found gflags  (include: /usr/local/include, library: /usr/local/lib/libgflags.dylib)
-- Found glog    (include: /usr/local/include, library: /usr/local/lib/libglog.dylib)
-- git Version: v0.0.0
-- Version: 0.0.0
-- Performing Test HAVE_THREAD_SAFETY_ATTRIBUTES
-- Performing Test HAVE_THREAD_SAFETY_ATTRIBUTES
-- Performing Test HAVE_THREAD_SAFETY_ATTRIBUTES -- failed to compile
-- Performing Test HAVE_STD_REGEX
-- Performing Test HAVE_STD_REGEX
-- Performing Test HAVE_STD_REGEX -- success
-- Performing Test HAVE_GNU_POSIX_REGEX
-- Performing Test HAVE_GNU_POSIX_REGEX
-- Performing Test HAVE_GNU_POSIX_REGEX -- failed to compile
-- Performing Test HAVE_POSIX_REGEX
-- Performing Test HAVE_POSIX_REGEX
-- Performing Test HAVE_POSIX_REGEX -- success
-- Performing Test HAVE_STEADY_CLOCK
-- Performing Test HAVE_STEADY_CLOCK
-- Performing Test HAVE_STEADY_CLOCK -- success
-- Found lmdb    (include: /usr/local/include, library: /usr/local/lib/liblmdb.dylib)
-- Found LevelDB (include: /usr/local/include, library: /usr/local/lib/libleveldb.dylib)
-- Found Snappy  (include: /usr/local/include, library: /usr/local/lib/libsnappy.dylib)
-- Could NOT find RocksDB (missing: RocksDB_INCLUDE_DIR RocksDB_LIBRARIES) 
CMake Warning at cmake/Dependencies.cmake:177 (message):
  Not compiling with RocksDB.  Suppress this warning with -DUSE_ROCKSDB=OFF
Call Stack (most recent call first):
  CMakeLists.txt:79 (include)

-- OpenCV found (/usr/local/share/OpenCV)
-- Found system Eigen at /usr/local/include/eigen3
-- Found PythonInterp: /Users/cslaptop/miniconda3/bin/python (found suitable version "3.6.3", minimum required is "2.7") 
-- NumPy ver. 1.13.3 found (include: /Users/cslaptop/miniconda3/lib/python3.6/site-packages/numpy/core/include)
-- Could NOT find pybind11 (missing: pybind11_INCLUDE_DIR) 
-- Could NOT find MPI_C (missing: MPI_C_LIB_NAMES MPI_C_HEADER_DIR MPI_C_WORKS) 
-- Could NOT find MPI_CXX (missing: MPI_CXX_LIB_NAMES MPI_CXX_HEADER_DIR MPI_CXX_WORKS) 
-- Could NOT find MPI (missing: MPI_C_FOUND MPI_CXX_FOUND) 
CMake Warning at cmake/Dependencies.cmake:300 (message):
  Not compiling with MPI.  Suppress this warning with -DUSE_MPI=OFF
Call Stack (most recent call first):
  CMakeLists.txt:79 (include)

CMake Warning at cmake/Dependencies.cmake:368 (message):
  If not using cuda, one should not use NCCL either.
Call Stack (most recent call first):
  CMakeLists.txt:79 (include)

CMake Warning at cmake/Dependencies.cmake:393 (message):
  Gloo can only be used on Linux.
Call Stack (most recent call first):
  CMakeLists.txt:79 (include)

CMake Warning at cmake/Dependencies.cmake:453 (message):
  mobile opengl is only used in android or ios builds.
Call Stack (most recent call first):
  CMakeLists.txt:79 (include)

CMake Warning at cmake/Dependencies.cmake:473 (message):
  Metal is only used in ios builds.
Call Stack (most recent call first):
  CMakeLists.txt:79 (include)

-- NCCL operators skipped due to no CUDA support
-- CUDA RTC operators skipped due to no CUDA support
-- Including image processing operators
-- Excluding video processing operators due to no opencv
-- Excluding mkl operators as we are not using mkl
-- MPI operators skipped due to no MPI support
-- Include Observer library
-- Automatically generating missing __init__.py files.
-- 
-- ******** Summary ********
-- General:
--   CMake version         : 3.10.1
--   CMake command         : /usr/local/Cellar/cmake/3.10.1/bin/cmake
--   Git version           : v0.8.1-935-gd026348d
--   System                : Darwin
--   C++ compiler          : /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++
--   C++ compiler version  : 8.0.0.8000042
--   Protobuf compiler     : /Users/cslaptop/miniconda3/bin/protoc
--   Protobuf include path : /Users/cslaptop/miniconda3/include
--   Protobuf libraries    : /Users/cslaptop/miniconda3/lib/libprotobuf.dylib
--   CXX flags             :  -std=c++11 -O2 -fPIC -Wno-narrowing
--   Build type            : Release
--   Compile definitions   : 
-- 
--   BUILD_BINARY          : ON
--   BUILD_PYTHON          : ON
--     Python version      : 2.7.10
--     Python library      : /usr/lib/libpython2.7.dylib
--   BUILD_SHARED_LIBS     : ON
--   BUILD_TEST            : ON
--   USE_ATEN              : OFF
--   USE_ASAN              : OFF
--   USE_CUDA              : OFF
--   USE_EIGEN_FOR_BLAS    : 1
--   USE_FFMPEG            : OFF
--   USE_GFLAGS            : ON
--   USE_GLOG              : ON
--   USE_GLOO              : OFF
--   USE_LEVELDB           : ON
--     LevelDB version     : 1.20
--     Snappy version      : 1.1.7
--   USE_LITE_PROTO        : OFF
--   USE_LMDB              : ON
--     LMDB version        : 0.9.21
--   USE_METAL             : OFF
--   USE_MKL               : 
--   USE_MOBILE_OPENGL     : OFF
--   USE_MPI               : OFF
--   USE_NCCL              : OFF
--   USE_NERVANA_GPU       : OFF
--   USE_NNPACK            : ON
--   USE_OBSERVERS         : ON
--   USE_OPENCV            : ON
--     OpenCV version      : 3.4.0
--   USE_OPENMP            : OFF
--   USE_PROF              : OFF
--   USE_REDIS             : OFF
--   USE_ROCKSDB           : OFF
--   USE_THREADS           : ON
--   USE_ZMQ               : OFF
-- Configuring done
CMake Warning (dev):
  Policy CMP0042 is not set: MACOSX_RPATH is enabled by default.  Run "cmake
  --help-policy CMP0042" for policy details.  Use the cmake_policy command to
  set the policy and suppress this warning.

  MACOSX_RPATH is not specified for the following targets:

   caffe2
   caffe2_module_test_dynamic
   nnpack_reference_layers

This warning is for project developers.  Use -Wno-dev to suppress it.

-- Generating done
-- Build files have been written to: /Users/cslaptop/Downloads/caffe2/build
pjh5 commented 6 years ago

I am not able to reproduce this so far. Are you using anaconda? What commands did you use to install caffe2?

AliengirlLiv commented 6 years ago

I'm not using anaconda.

I installed caffe2 using the "MacOS X" instructions on this page using the "Brew and Pip Install Path" instructions.

pjh5 commented 6 years ago

Can you run

otool -L /usr/local/opt/glog/lib/libglog.0.dylib

otool -L /Users/cslaptop/Downloads/caffe2/build/caffe2/python/caffe2_pybind11_state.so and also find / -name libglog* 2>/dev/null

and paste the output here?

AliengirlLiv commented 6 years ago 
otool -L /usr/local/opt/glog/lib/libglog.0.dylib
/usr/local/opt/glog/lib/libglog.0.dylib:
    /usr/local/opt/glog/lib/libglog.0.3.5.dylib (compatibility version 0.3.5, current version 0.0.0)
    /usr/local/opt/gflags/lib/libgflags.2.2.dylib (compatibility version 2.2.0, current version 2.2.1)
    /usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 120.1.0)
    /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1226.10.1)
otool -L /Users/cslaptop/Downloads/caffe2/build/caffe2/python/caffe2_pybind11_state.so
/Users/cslaptop/Downloads/caffe2/build/caffe2/python/caffe2_pybind11_state.so:
    /Users/cslaptop/Downloads/caffe2/build/lib/libcaffe2.dylib (compatibility version 0.0.0, current version 0.0.0)
    @rpath/libprotobuf.14.dylib (compatibility version 15.0.0, current version 15.0.0)
    /usr/local/opt/gflags/lib/libgflags.2.2.dylib (compatibility version 2.2.0, current version 2.2.0)
    /usr/local/opt/glog/lib/libglog.0.dylib (compatibility version 1.0.0, current version 1.0.0)
    /usr/local/opt/lmdb/lib/liblmdb.dylib (compatibility version 0.0.0, current version 0.0.0)
    /usr/local/opt/leveldb/lib/libleveldb.1.20.dylib (compatibility version 0.0.0, current version 0.0.0)
    /usr/local/opt/snappy/lib/libsnappy.1.dylib (compatibility version 5.0.0, current version 5.1.0)
    /usr/local/opt/opencv/lib/libopencv_highgui.2.4.dylib (compatibility version 2.4.0, current version 2.4.13)
    /usr/local/opt/opencv/lib/libopencv_imgproc.2.4.dylib (compatibility version 2.4.0, current version 2.4.13)
    /usr/local/opt/opencv/lib/libopencv_core.2.4.dylib (compatibility version 2.4.0, current version 2.4.13)
    /usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 120.1.0)
    /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1226.10.1)
find / -name libglog* 2>/dev/null
/Users/cslaptop/Documents/Coding/AICamera/app/src/main/jniLibs/armeabi-v7a/libglog.so
/Users/cslaptop/Documents/Coding/MobileApp/node_modules/react-native/ReactAndroid/src/main/jni/prebuilt/lib/armeabi-v7a/libglog.so
/Users/cslaptop/Documents/Coding/MobileApp/node_modules/react-native/ReactAndroid/src/main/jni/prebuilt/lib/armeabi-v7a/libglog_init.so
/Users/cslaptop/Documents/Coding/MobileApp/node_modules/react-native/ReactAndroid/src/main/jni/prebuilt/lib/x86/libglog.so
/Users/cslaptop/Documents/Coding/MobileApp/node_modules/react-native/ReactAndroid/src/main/jni/prebuilt/lib/x86/libglog_init.so
/Users/cslaptop/Library/Caches/Homebrew/glog--git/libglog.pc.in
/Users/cslaptop/miniconda3/envs/python2/lib/libglog.0.dylib
/Users/cslaptop/miniconda3/envs/python2/lib/libglog.a
/Users/cslaptop/miniconda3/envs/python2/lib/libglog.dylib
/Users/cslaptop/miniconda3/envs/python2/lib/libglog.la
/Users/cslaptop/miniconda3/envs/python2/lib/pkgconfig/libglog.pc
/Users/cslaptop/miniconda3/pkgs/glog-0.3.5-0/lib/libglog.0.dylib
/Users/cslaptop/miniconda3/pkgs/glog-0.3.5-0/lib/libglog.a
/Users/cslaptop/miniconda3/pkgs/glog-0.3.5-0/lib/libglog.dylib
/Users/cslaptop/miniconda3/pkgs/glog-0.3.5-0/lib/libglog.la
/Users/cslaptop/miniconda3/pkgs/glog-0.3.5-0/lib/pkgconfig/libglog.pc
/usr/local/Cellar/glog/0.3.5_3/lib/libglog.0.3.5.dylib
/usr/local/Cellar/glog/0.3.5_3/lib/libglog.0.dylib
/usr/local/Cellar/glog/0.3.5_3/lib/libglog.dylib
/usr/local/Cellar/glog/0.3.5_3/lib/pkgconfig/libglog.pc
/usr/local/lib/libglog.0.3.5.dylib
/usr/local/lib/libglog.0.dylib
/usr/local/lib/libglog.dylib
/usr/local/lib/pkgconfig/libglog.pc

As a note, I'm surprised to see miniconda directories here, since I uninstalled miniconda a while ago. Is it possible that's somehow related to the error?

pjh5 commented 6 years ago

If you echo $PATH and miniconda isn't it, then this miniconda installation is not related to your problem.

As an aside, Miniconda is uninstalled by manually removing the miniconda directory, like sudo rm -rf ~/miniconda3 , and then by removing 'miniconda' from your $PATH (if you echo $PATH and miniconda is still in it somwhere, then check your ~/.bashrc and ~/.bash_profile files for a line that looks like export PATH=<miniconda somewhere in here> and delete it, then reopen all of your terminal windows).

Uninstalling your miniconda and building again might help. Uninstalling glog with brew and reinstalling with pip might help too (brew uninstall -y glog && pip install glog), but I'm not really sure what's going on here yet. I don't see any problems in the output you posted. @orionr do you know what's happening here?

Could you also paste the output for brew list glog and pip freeze ?

pjh5 commented 6 years ago

Also, if you don't care about glog and just want to get Caffe2 running then you can just turn glog off by passing -DUSE_GLOG=OFF to cmake

AliengirlLiv commented 6 years ago

Thanks for the suggestions! After deleting the miniconda directory, uninstalling glog with brew and reinstalling it with pip, I stop getting the glog error, but when I try to import caffe2 in python I instead get the error

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "caffe2/python/core.py", line 29, in <module>
    from caffe2.proto import caffe2_pb2
ImportError: cannot import name caffe2_pb2

Here are the outputs of the commands you requested:

brew list glog
Error: No such keg: /usr/local/Cellar/glog
pip freeze
altgraph==0.10.2
appdirs==1.4.3
appnope==0.1.0
backports.shutil-get-terminal-size==1.0.0
bdist-mpkg==0.5.0
bonjour-py==0.3
certifi==2017.4.17
chardet==3.0.4
click==6.7
decorator==4.1.2
enum34==1.1.6
Flask==0.12.2
future==0.16.0
glog==0.3.1
httpie==0.9.9
idna==2.5
ipython==5.5.0
ipython-genutils==0.2.0
itsdangerous==0.24
Jinja2==2.9.6
macholib==1.5.1
MarkupSafe==1.0
matplotlib==1.3.1
modulegraph==0.10.4
nbdiff==1.0.3
numpy==1.8.0rc1
olefile==0.44
packaging==16.8
pathlib2==2.3.0
pexpect==4.2.1
pickleshare==0.7.4
Pillow==4.3.0
prompt-toolkit==1.0.15
protobuf==3.5.1
ptyprocess==0.5.2
py2app==0.7.3
Pygments==2.2.0
pyobjc-core==2.5.1
pyobjc-framework-Accounts==2.5.1
pyobjc-framework-AddressBook==2.5.1
pyobjc-framework-AppleScriptKit==2.5.1
pyobjc-framework-AppleScriptObjC==2.5.1
pyobjc-framework-Automator==2.5.1
pyobjc-framework-CFNetwork==2.5.1
pyobjc-framework-Cocoa==2.5.1
pyobjc-framework-Collaboration==2.5.1
pyobjc-framework-CoreData==2.5.1
pyobjc-framework-CoreLocation==2.5.1
pyobjc-framework-CoreText==2.5.1
pyobjc-framework-DictionaryServices==2.5.1
pyobjc-framework-EventKit==2.5.1
pyobjc-framework-ExceptionHandling==2.5.1
pyobjc-framework-FSEvents==2.5.1
pyobjc-framework-InputMethodKit==2.5.1
pyobjc-framework-InstallerPlugins==2.5.1
pyobjc-framework-InstantMessage==2.5.1
pyobjc-framework-LatentSemanticMapping==2.5.1
pyobjc-framework-LaunchServices==2.5.1
pyobjc-framework-Message==2.5.1
pyobjc-framework-OpenDirectory==2.5.1
pyobjc-framework-PreferencePanes==2.5.1
pyobjc-framework-PubSub==2.5.1
pyobjc-framework-QTKit==2.5.1
pyobjc-framework-Quartz==2.5.1
pyobjc-framework-ScreenSaver==2.5.1
pyobjc-framework-ScriptingBridge==2.5.1
pyobjc-framework-SearchKit==2.5.1
pyobjc-framework-ServiceManagement==2.5.1
pyobjc-framework-Social==2.5.1
pyobjc-framework-SyncServices==2.5.1
pyobjc-framework-SystemConfiguration==2.5.1
pyobjc-framework-WebKit==2.5.1
pyOpenSSL==0.13.1
pyparsing==2.0.1
python-dateutil==1.5
python-gflags==3.1.2
python-Levenshtein==0.12.0
pytz==2013.7
requests==2.18.1
scandir==1.6
scipy==0.13.0b1
selenium==3.4.3
simplegeneric==0.8.1
six==1.4.1
traitlets==4.3.2
urllib3==1.21.1
virtualenv==15.1.0
wcwidth==0.1.7
Werkzeug==0.12.2
xattr==0.6.4
zope.interface==4.1.1
pjh5 commented 6 years ago

Since the original issue is fixed, I will close this. Please open a new issue with the new problem (can't find caffe2_pb2).

Okay, so my best guess is that the latest Caffe2 build picked up cached / already-installed binaries that are left over from a previous failed Caffe2 build (which picked up binaries from miniconda). Please delete all the old build stuffs; if you're using the pip and brew instructions then these are probably 1) caffe2/build directory that you created during the installation instructions and 2) /usr/local/caffe2 and /usr/local/caffe (unless you manually changed CMAKE_INSTALL_PREFIX to something other than /usr/local). Please delete all of these folders, then rebuild (starting from the mkdir build && cd build).

This might be related to protobuf now, so https://caffe2.ai/docs/getting-started.html?platform=mac&configuration=compile#null__protobuf-errors and https://caffe2.ai/docs/getting-started.html?platform=mac&configuration=compile#null__general-debugging-tips might be useful to you.

If the problem persists after a clean build, please paste the full output from the cmake command that you ran.