search

facebookarchive / caffe2

Caffe2 is a lightweight, modular, and scalable deep learning framework.
https://caffe2.ai
Apache License 2.0
8.42k stars 1.94k forks source link

Failing to make with 8 cores #2464

Closed rosewang2008 closed 6 years ago

rosewang2008 commented 6 years ago

I'm trying to install the dependencies for Detectron (FAIR) but I am running into difficulties with makeing Caffe2. I am following this guide under mattifrind. I've already looked at these posts, but had no success in resolving my similar issue.

Thanks!

System information

under mattifrind

`rose@rose-GS73VR-7RF:~/caffe2$ sudo make -j8 -- Does not need to define long separately. -- std::exception_ptr is supported. -- NUMA is available -- Current compiler supports avx2 extention. Will build perfkernels. -- Caffe2: Found protobuf with old-style protobuf targets. -- Caffe2 protobuf include directory: /usr/include -- The BLAS backend of choice:Eigen -- Could NOT find NNPACK (missing: NNPACK_INCLUDE_DIR NNPACK_LIBRARY PTHREADPOOL_LIBRARY CPUINFO_LIBRARY) -- Brace yourself, we are building NNPACK -- Found PythonInterp: /usr/bin/python (found version "2.7.12") -- Caffe2: Cannot find gflags automatically. Using legacy find. -- Caffe2: Found gflags (include: /usr/include, library: /usr/lib/x86_64-linux-gnu/libgflags.so) -- Caffe2: Cannot find glog automatically. Using legacy find. -- Caffe2: Found glog (include: /usr/include, library: /usr/lib/x86_64-linux-gnu/libglog.so) -- git Version: v0.0.0 -- Version: 0.0.0 -- Performing Test HAVE_STD_REGEX -- Performing Test HAVE_STD_REGEX -- Performing Test HAVE_STD_REGEX -- success -- Performing Test HAVE_GNU_POSIX_REGEX -- Performing Test HAVE_GNU_POSIX_REGEX -- Performing Test HAVE_GNU_POSIX_REGEX -- failed to compile -- Performing Test HAVE_POSIX_REGEX -- Performing Test HAVE_POSIX_REGEX -- Performing Test HAVE_POSIX_REGEX -- success -- Performing Test HAVE_STEADY_CLOCK -- Performing Test HAVE_STEADY_CLOCK -- Performing Test HAVE_STEADY_CLOCK -- success -- Found lmdb (include: /usr/include, library: /usr/lib/x86_64-linux-gnu/liblmdb.so) -- Found LevelDB (include: /usr/include, library: /usr/lib/x86_64-linux-gnu/libleveldb.so) -- Found Snappy (include: /usr/include, library: /usr/lib/x86_64-linux-gnu/libsnappy.so) -- Found Numa (include: /usr/include, library: /usr/lib/x86_64-linux-gnu/libnuma.so) -- OpenCV found (/usr/share/OpenCV) CMake Warning at cmake/Dependencies.cmake:280 (find_package): By not providing "FindEigen3.cmake" in CMAKE_MODULE_PATH this project has asked CMake to find a package configuration file provided by "Eigen3", but CMake did not find one.

Could not find a package configuration file provided by "Eigen3" with any of the following names:

Eigen3Config.cmake
eigen3-config.cmake

Add the installation prefix of "Eigen3" to CMAKE_PREFIX_PATH or set "Eigen3_DIR" to a directory containing one of the above files. If "Eigen3" provides a separate development package or SDK, be sure it has been installed. Call Stack (most recent call first): CMakeLists.txt:103 (include)

-- Did not find system Eigen. Using third party subdirectory. -- Found PythonInterp: /usr/bin/python (found suitable version "2.7.12", minimum required is "2.7") -- NumPy ver. 1.14.2 found (include: /usr/local/lib/python2.7/dist-packages/numpy/core/include) -- Could NOT find pybind11 (missing: pybind11_INCLUDE_DIR) -- MPI support found -- MPI compile flags: -- MPI include path: /usr/lib/openmpi/include/openmpi/opal/mca/event/libevent2021/libevent/usr/lib/openmpi/include/openmpi/opal/mca/event/libevent2021/libevent/include/usr/lib/openmpi/include/usr/lib/openmpi/include/openmpi -- MPI LINK flags path: -Wl,-rpath -Wl,/usr/lib/openmpi/lib -Wl,--enable-new-dtags -- MPI libraries: /usr/lib/openmpi/lib/libmpi_cxx.so/usr/lib/openmpi/lib/libmpi.so CMake Warning at cmake/Dependencies.cmake:334 (message): OpenMPI found, but it is not built with CUDA support. Call Stack (most recent call first): CMakeLists.txt:103 (include)

-- Could NOT find CUDNN (missing: CUDNN_INCLUDE_DIR CUDNN_LIBRARY) CMake Warning at cmake/public/cuda.cmake:28 (message): Caffe2: cudnn cannot be found. Caffe2 CUDA depends explicitly on cudnn so you should consider installing it. Call Stack (most recent call first): cmake/Dependencies.cmake:366 (include) CMakeLists.txt:103 (include)

CMake Warning at cmake/Dependencies.cmake:374 (message): Not compiling with CUDA. Suppress this warning with -DUSE_CUDA=OFF. Call Stack (most recent call first): CMakeLists.txt:103 (include)

CMake Warning at cmake/Dependencies.cmake:384 (message): If not using cuda, one should not use NCCL either. Call Stack (most recent call first): CMakeLists.txt:103 (include)

-- Could NOT find Gloo (missing: Gloo_INCLUDE_DIR Gloo_LIBRARY) -- MPI include path: /usr/lib/openmpi/include/openmpi/opal/mca/event/libevent2021/libevent/usr/lib/openmpi/include/openmpi/opal/mca/event/libevent2021/libevent/include/usr/lib/openmpi/include/usr/lib/openmpi/include/openmpi -- MPI libraries: /usr/lib/openmpi/lib/libmpi_cxx.so/usr/lib/openmpi/lib/libmpi.so CMake Warning at cmake/Dependencies.cmake:467 (message): mobile opengl is only used in android or ios builds. Call Stack (most recent call first): CMakeLists.txt:103 (include)

CMake Warning at cmake/Dependencies.cmake:543 (message): Metal is only used in ios builds. Call Stack (most recent call first): CMakeLists.txt:103 (include)

-- Could NOT find pybind11 (missing: pybind11_INCLUDE_DIR) -- GCC 5.4.0: Adding gcc and gcc_s libs to link line -- NCCL operators skipped due to no CUDA support -- CUDA RTC operators skipped due to no CUDA support -- Including image processing operators -- Excluding video processing operators due to no opencv -- Excluding mkl operators as we are not using mkl -- Include Observer library -- Using lib/python2.7/dist-packages as python relative installation path -- Automatically generating missing init.py files. -- -- **** Summary **** -- General: -- CMake version : 3.5.1 -- CMake command : /usr/bin/cmake -- Git version : v0.8.1-1483-g56bb251 -- System : Linux -- C++ compiler : /usr/bin/c++ -- C++ compiler version : 5.4.0 -- Protobuf compiler : /usr/bin/protoc -- Protobuf include path : /usr/include -- Protobuf libraries : optimized;/usr/lib/x86_64-linux-gnu/libprotobuf.so;debug;/usr/lib/x86_64-linux-gnu/libprotobuf.so;-pthread -- BLAS : Eigen -- CXX flags : -DONNX_NAMESPACE=onnx_c2 -O2 -fPIC -Wno-narrowing -Wno-invalid-partial-specialization -- Build type : Release -- Compile definitions : -- -- BUILD_BINARY : ON -- BUILD_CUSTOM_PROTOBUF : OFF -- BUILD_DOCS : OFF -- BUILD_PYTHON : ON -- Python version : 2.7.12 -- Python includes : /usr/include/python2.7 -- BUILD_SHARED_LIBS : ON -- BUILD_TEST : ON -- USE_ATEN : OFF -- USE_ASAN : OFF -- USE_CUDA : OFF -- USE_EIGEN_FOR_BLAS : 1 -- USE_FFMPEG : OFF -- USE_GFLAGS : ON -- USE_GLOG : ON -- USE_GLOO : ON -- USE_LEVELDB : ON -- LevelDB version : 1.18 -- Snappy version : 1.1.3 -- USE_LITE_PROTO : OFF -- USE_LMDB : ON -- LMDB version : 0.9.17 -- USE_METAL : OFF -- USE_MKL : -- USE_MOBILE_OPENGL : OFF -- USE_MPI : ON -- USE_NCCL : OFF -- USE_NERVANA_GPU : OFF -- USE_NNPACK : ON -- USE_OBSERVERS : ON -- USE_OPENCV : ON -- OpenCV version : 2.4.9.1 -- USE_OPENMP : OFF -- USE_PROF : OFF -- USE_REDIS : OFF -- USE_ROCKSDB : OFF -- USE_ZMQ : OFF -- Configuring done -- Generating done -- Build files have been written to: /home/rose/caffe2/build make[1]: Entering directory '/home/rose/caffe2/build' make[2]: Entering directory '/home/rose/caffe2/build' make[3]: Entering directory '/home/rose/caffe2/build' make[3]: Entering directory '/home/rose/caffe2/build' make[3]: Entering directory '/home/rose/caffe2/build' make[3]: Entering directory '/home/rose/caffe2/build' make[3]: Entering directory '/home/rose/caffe2/build' make[3]: Entering directory '/home/rose/caffe2/build' make[3]: Entering directory '/home/rose/caffe2/build' make[3]: Entering directory '/home/rose/caffe2/build' make[3]: Leaving directory '/home/rose/caffe2/build' make[3]: Leaving directory '/home/rose/caffe2/build' make[3]: Leaving directory '/home/rose/caffe2/build' make[3]: Leaving directory '/home/rose/caffe2/build' make[3]: Leaving directory '/home/rose/caffe2/build' make[3]: Leaving directory '/home/rose/caffe2/build' make[3]: Leaving directory '/home/rose/caffe2/build' make[3]: Entering directory '/home/rose/caffe2/build' make[3]: Leaving directory '/home/rose/caffe2/build' [ 1%] Built target gtest [ 1%] Built target pthreadpool [ 1%] Built target onnx_proto [ 1%] Built target Caffe_PROTO [ 4%] Built target benchmark make[3]: Entering directory '/home/rose/caffe2/build' [ 8%] Built target cpuinfo make[3]: Entering directory '/home/rose/caffe2/build' make[3]: Entering directory '/home/rose/caffe2/build' make[3]: Entering directory '/home/rose/caffe2/build' make[3]: Entering directory '/home/rose/caffe2/build' make[3]: Leaving directory '/home/rose/caffe2/build' make[3]: Leaving directory '/home/rose/caffe2/build' make[3]: Leaving directory '/home/rose/caffe2/build' [ 12%] Built target gloo make[3]: Leaving directory '/home/rose/caffe2/build' make[3]: Entering directory '/home/rose/caffe2/build' make[3]: Leaving directory '/home/rose/caffe2/build' [ 14%] Built target Caffe2_PROTO [ 16%] Built target gtest_main make[3]: Entering directory '/home/rose/caffe2/build' [ 16%] Built target nnpack_reference_layers make[3]: Entering directory '/home/rose/caffe2/build' make[3]: Entering directory '/home/rose/caffe2/build' make[3]: Entering directory '/home/rose/caffe2/build' [ 20%] Built target onnx make[3]: Leaving directory '/home/rose/caffe2/build' make[3]: Entering directory '/home/rose/caffe2/build' [ 25%] Built target nnpack make[3]: Leaving directory '/home/rose/caffe2/build' make[3]: Leaving directory '/home/rose/caffe2/build' make[3]: Leaving directory '/home/rose/caffe2/build' make[3]: Leaving directory '/home/rose/caffe2/build' make[3]: Leaving directory '/home/rose/caffe2/build' [ 25%] Built target gloo_builder [ 25%] Built target caffe2_protos [ 25%] Built target nomnigraph [ 26%] Built target onnx_pybind [ 26%] Built target Caffe2_perfkernels_avx [ 27%] Built target Caffe2_perfkernels_avx2 make[3]: Entering directory '/home/rose/caffe2/build' make[3]: Leaving directory '/home/rose/caffe2/build' [ 78%] Built target caffe2 make[3]: Entering directory '/home/rose/caffe2/build' make[3]: Entering directory '/home/rose/caffe2/build' make[3]: Entering directory '/home/rose/caffe2/build' make[3]: Entering directory '/home/rose/caffe2/build' make[3]: Entering directory '/home/rose/caffe2/build' make[3]: Entering directory '/home/rose/caffe2/build' make[3]: Entering directory '/home/rose/caffe2/build' make[3]: Leaving directory '/home/rose/caffe2/build' make[3]: Leaving directory '/home/rose/caffe2/build' make[3]: Leaving directory '/home/rose/caffe2/build' make[3]: Leaving directory '/home/rose/caffe2/build' make[3]: Leaving directory '/home/rose/caffe2/build' make[3]: Leaving directory '/home/rose/caffe2/build' make[3]: Leaving directory '/home/rose/caffe2/build' [ 78%] Built target common_test [ 79%] Built target graph_test [ 79%] Built target blob_test [ 79%] Built target registry_test [ 80%] Built target typeid_test [ 81%] Built target module_test make[3]: Entering directory '/home/rose/caffe2/build' make[3]: Entering directory '/home/rose/caffe2/build' make[3]: Entering directory '/home/rose/caffe2/build' make[3]: Entering directory '/home/rose/caffe2/build' make[3]: Entering directory '/home/rose/caffe2/build' make[3]: Entering directory '/home/rose/caffe2/build' [ 82%] Built target caffe2_pybind11_state make[3]: Leaving directory '/home/rose/caffe2/build' make[3]: Entering directory '/home/rose/caffe2/build' make[3]: Leaving directory '/home/rose/caffe2/build' make[3]: Leaving directory '/home/rose/caffe2/build' make[3]: Leaving directory '/home/rose/caffe2/build' make[3]: Leaving directory '/home/rose/caffe2/build' make[3]: Leaving directory '/home/rose/caffe2/build' make[3]: Leaving directory '/home/rose/caffe2/build' [ 82%] Built target stats_test [ 83%] Built target workspace_test [ 83%] Built target cpuid_test [ 84%] Built target parallel_net_test make[3]: Entering directory '/home/rose/caffe2/build' make[3]: Entering directory '/home/rose/caffe2/build' [ 84%] Built target fixed_divisor_test [ 84%] Built target fatal_signal_asan_no_sig_test make[3]: Entering directory '/home/rose/caffe2/build' make[3]: Entering directory '/home/rose/caffe2/build' make[3]: Entering directory '/home/rose/caffe2/build' make[3]: Entering directory '/home/rose/caffe2/build' [ 84%] Built target operator_test make[3]: Leaving directory '/home/rose/caffe2/build' make[3]: Leaving directory '/home/rose/caffe2/build' make[3]: Leaving directory '/home/rose/caffe2/build' make[3]: Leaving directory '/home/rose/caffe2/build' make[3]: Entering directory '/home/rose/caffe2/build' make[3]: Leaving directory '/home/rose/caffe2/build' make[3]: Leaving directory '/home/rose/caffe2/build' make[3]: Leaving directory '/home/rose/caffe2/build' [ 84%] Built target smart_tensor_printer_test [ 85%] Built target init_test [ 86%] Built target boolean_unmask_ops_test [ 87%] Built target event_test [ 88%] Built target timer_test [ 88%] Built target operator_schema_test make[3]: Entering directory '/home/rose/caffe2/build' make[3]: Entering directory '/home/rose/caffe2/build' make[3]: Entering directory '/home/rose/caffe2/build' make[3]: Entering directory '/home/rose/caffe2/build' make[3]: Entering directory '/home/rose/caffe2/build' make[3]: Entering directory '/home/rose/caffe2/build' make[3]: Leaving directory '/home/rose/caffe2/build' make[3]: Leaving directory '/home/rose/caffe2/build' make[3]: Leaving directory '/home/rose/caffe2/build' make[3]: Leaving directory '/home/rose/caffe2/build' make[3]: Leaving directory '/home/rose/caffe2/build' make[3]: Leaving directory '/home/rose/caffe2/build' [ 88%] Built target transform_test [ 88%] Built target logging_test make[3]: Entering directory '/home/rose/caffe2/build' make[3]: Entering directory '/home/rose/caffe2/build' [ 88%] Built target time_observer_test [ 88%] Built target conv_to_nnpack_transform_test [ 88%] Built target predictor_test [ 88%] Built target generate_proposals_op_util_boxes_test [ 89%] Built target string_ops_test make[3]: Entering directory '/home/rose/caffe2/build' make[3]: Entering directory '/home/rose/caffe2/build' make[3]: Entering directory '/home/rose/caffe2/build' make[3]: Entering directory '/home/rose/caffe2/build' make[3]: Entering directory '/home/rose/caffe2/build' make[3]: Leaving directory '/home/rose/caffe2/build' make[3]: Leaving directory '/home/rose/caffe2/build' make[3]: Leaving directory '/home/rose/caffe2/build' make[3]: Leaving directory '/home/rose/caffe2/build' make[3]: Leaving directory '/home/rose/caffe2/build' make[3]: Leaving directory '/home/rose/caffe2/build' make[3]: Leaving directory '/home/rose/caffe2/build' [ 89%] Built target net_test make[3]: Entering directory '/home/rose/caffe2/build' [ 89%] Built target simple_queue_test make[3]: Entering directory '/home/rose/caffe2/build' [ 90%] Built target generate_proposals_op_test [ 91%] Built target conv_transpose_op_mobile_test [ 91%] Built target elementwise_op_test [ 91%] Built target generate_proposals_op_util_nms_test make[3]: Entering directory '/home/rose/caffe2/build' make[3]: Entering directory '/home/rose/caffe2/build' make[3]: Leaving directory '/home/rose/caffe2/build' [ 91%] Built target utility_ops_test make[3]: Entering directory '/home/rose/caffe2/build' make[3]: Entering directory '/home/rose/caffe2/build' make[3]: Entering directory '/home/rose/caffe2/build' make[3]: Leaving directory '/home/rose/caffe2/build' make[3]: Leaving directory '/home/rose/caffe2/build' make[3]: Leaving directory '/home/rose/caffe2/build' make[3]: Leaving directory '/home/rose/caffe2/build' make[3]: Leaving directory '/home/rose/caffe2/build' make[3]: Leaving directory '/home/rose/caffe2/build' make[3]: Entering directory '/home/rose/caffe2/build' [ 91%] Built target text_file_reader_utils_test make[3]: Entering directory '/home/rose/caffe2/build' [ 91%] Built target pattern_net_transform_test [ 92%] Built target observer_test [ 91%] Linking CXX executable ../bin/mpi_test [ 92%] Built target batch_matmul_op_test make[3]: Entering directory '/home/rose/caffe2/build' [ 93%] Built target context_test make[3]: Entering directory '/home/rose/caffe2/build' make[3]: Entering directory '/home/rose/caffe2/build' make[3]: Entering directory '/home/rose/caffe2/build' [ 93%] Built target conv_op_test make[3]: Leaving directory '/home/rose/caffe2/build' make[3]: Leaving directory '/home/rose/caffe2/build' make[3]: Leaving directory '/home/rose/caffe2/build' make[3]: Leaving directory '/home/rose/caffe2/build' make[3]: Leaving directory '/home/rose/caffe2/build' [ 93%] Built target common_subexpression_elimination_test [ 93%] Built target proto_utils_test [ 94%] Built target caffe2_observers [ 94%] Built target caffe2_module_test_dynamic make[3]: Entering directory '/home/rose/caffe2/build' make[3]: Entering directory '/home/rose/caffe2/build' make[3]: Entering directory '/home/rose/caffe2/build' make[3]: Entering directory '/home/rose/caffe2/build' make[3]: Entering directory '/home/rose/caffe2/build' [ 94%] Built target math_test make[3]: Entering directory '/home/rose/caffe2/build' make[3]: Leaving directory '/home/rose/caffe2/build' make[3]: Leaving directory '/home/rose/caffe2/build' make[3]: Leaving directory '/home/rose/caffe2/build' make[3]: Leaving directory '/home/rose/caffe2/build' make[3]: Leaving directory '/home/rose/caffe2/build' make[3]: Leaving directory '/home/rose/caffe2/build' [ 95%] Built target make_image_db [ 95%] Built target make_mnist_db [ 95%] Built target tutorial_blob make[3]: Entering directory '/home/rose/caffe2/build' make[3]: Entering directory '/home/rose/caffe2/build' make[3]: Entering directory '/home/rose/caffe2/build' [ 96%] Built target convert_encoded_to_raw_leveldb make[3]: Entering directory '/home/rose/caffe2/build' [ 97%] Built target caffe2_benchmark [ 97%] Built target run_plan make[3]: Entering directory '/home/rose/caffe2/build' make[3]: Leaving directory '/home/rose/caffe2/build' make[3]: Leaving directory '/home/rose/caffe2/build' make[3]: Entering directory '/home/rose/caffe2/build' make[3]: Leaving directory '/home/rose/caffe2/build' make[3]: Leaving directory '/home/rose/caffe2/build' make[3]: Leaving directory '/home/rose/caffe2/build' make[3]: Leaving directory '/home/rose/caffe2/build' [ 97%] Built target convert_db [ 97%] Built target print_registered_core_operators [ 98%] Built target predictor_verifier make[3]: Entering directory '/home/rose/caffe2/build' make[3]: Entering directory '/home/rose/caffe2/build' make[3]: Entering directory '/home/rose/caffe2/build' [ 99%] Built target run_plan_mpi [ 99%] Built target make_cifar_db [100%] Built target speed_benchmark make[3]: Leaving directory '/home/rose/caffe2/build' make[3]: Leaving directory '/home/rose/caffe2/build' make[3]: Leaving directory '/home/rose/caffe2/build' [100%] Built target split_db [100%] Built target db_throughput [100%] Built target convert_caffe_image_db /usr/bin/ld: CMakeFiles/mpi_test.dir/mpi/mpi_test.cc.o: undefined reference to symbol '_ZN3MPI8Datatype4FreeEv' //usr/lib/libmpi_cxx.so.1: error adding symbols: DSO missing from command line collect2: error: ld returned 1 exit status caffe2/CMakeFiles/mpi_test.dir/build.make:101: recipe for target 'bin/mpi_test' failed make[3]: [bin/mpi_test] Error 1 make[3]: Leaving directory '/home/rose/caffe2/build' CMakeFiles/Makefile2:2738: recipe for target 'caffe2/CMakeFiles/mpi_test.dir/all' failed make[2]: [caffe2/CMakeFiles/mpi_test.dir/all] Error 2 make[2]: Waiting for unfinished jobs.... make[3]: Leaving directory '/home/rose/caffe2/build' [100%] Built target python_copy_files make[2]: Leaving directory '/home/rose/caffe2/build' Makefile:138: recipe for target 'all' failed make[1]: [all] Error 2 make[1]: Leaving directory '/home/rose/caffe2/build' Makefile:4: recipe for target 'all' failed make: *** [all] Error 2 `

pjh5 commented 6 years ago

This is a duplicate of #2144 ; it's related to openmpi and not to how many cores you use to build. I will close this issue as a duplicate, but you can please post the output of mpirun --version and ldd //usr/lib/libmpi_cxx.so.1 and nm /usr/lib/libmpi_cxx.so.1 | grep -E 'Datatype.*Free' on #2144?