Segmentation fault (core dumped) when run tests pass "test_spatial_narrow_as_op.py"

[cvpr]

System information

• Operating system: ubuntu16.04
• Compiler version: gcc
• CUDA version: cuda 8.0
• cuDNN version: 6.021
• GPU models (for all devices if they are not all the same): GTX1080
• PYTHONPATH environment variable: caffe2/build
• python --version output: 2.7.12

I compiled successful, when I run test pass: "test_spatial_narrow_as_op.py" python Detectron/tests/test_spatial_narrow_as_op.py it output following: E0224 14:21:15.808567 5540 init_intrinsics_check.cc:59] CPU feature avx is present on your machine, but the Caffe2 binary is not compiled with it. It means you may not get the full speed of your CPU. E0224 14:21:15.808586 5540 init_intrinsics_check.cc:59] CPU feature avx2 is present on your machine, but the Caffe2 binary is not compiled with it. It means you may not get the full speed of your CPU. E0224 14:21:15.808593 5540 init_intrinsics_check.cc:59] CPU feature fma is present on your machine, but the Caffe2 binary is not compiled with it. It means you may not get the full speed of your CPU. .. Aborted at 1519453276 (unix time) try "date -d @1519453276" if you are using GNU date PC: @ 0x7f23ba9a05e6 google::protobuf::MessageLite::AppendToString() SIGSEGV (@0x29) received by PID 5540 (TID 0x7f23bfb24700) from PID 41; stack trace: @ 0x7f23bf3794b0 (unknown) @ 0x7f23ba9a05e6 google::protobuf::MessageLite::AppendToString() @ 0x7f23ba9a0925 _ZNK6google8protobuf11MessageLite17SerializeAsStringB5cxx11Ev @ 0x7f23bacab1cd _ZZN8pybind1112cpp_function10initializeIZN6caffe26python16addObjectMethodsERNS_6moduleEEUlNS_5bytesESt6vectorINS2_15GradientWrapperESaIS8_EEE28_St4pairIS7_IS6_SaIS6_EESA_EJS6_SA_EJNS_4nameENS_5scopeENS_7siblingENS_19return_value_policyEEEEvOT_PFT0_DpT1_EDpRKT2_ENUlRNS_6detail13function_callEE1_4FUNESX @ 0x7f23baccc3d3 pybind11::cpp_function::dispatcher() @ 0x4bc3fa PyEval_EvalFrameEx @ 0x4b9ab6 PyEval_EvalCodeEx @ 0x4c1e6f PyEval_EvalFrameEx @ 0x4b9ab6 PyEval_EvalCodeEx @ 0x4c1e6f PyEval_EvalFrameEx @ 0x4b9ab6 PyEval_EvalCodeEx @ 0x4c16e7 PyEval_EvalFrameEx @ 0x4b9ab6 PyEval_EvalCodeEx @ 0x4c16e7 PyEval_EvalFrameEx @ 0x4b9ab6 PyEval_EvalCodeEx @ 0x4c1e6f PyEval_EvalFrameEx @ 0x4b9ab6 PyEval_EvalCodeEx @ 0x4d55f3 (unknown) @ 0x4a577e PyObject_Call @ 0x4bed3d PyEval_EvalFrameEx @ 0x4b9ab6 PyEval_EvalCodeEx @ 0x4d54b9 (unknown) @ 0x4eebee (unknown) @ 0x4a577e PyObject_Call @ 0x548253 (unknown) @ 0x4c15bf PyEval_EvalFrameEx @ 0x4b9ab6 PyEval_EvalCodeEx @ 0x4d55f3 (unknown) @ 0x4a577e PyObject_Call @ 0x4bed3d PyEval_EvalFrameEx @ 0x4b9ab6 PyEval_EvalCodeEx @ 0x4d54b9 (unknown) Segmentation fault (core dumped)

Please help to solve it. thanks.

[ir413]

Hi @cvpr, we are unable to reproduce this issue on our side.

One shot in the dark is to check that the prtobuf versions are not getting mixed. In particular, when running cmake for Caffe2 double-check that the protobuf compiler, headers and libraries are from the same version. Here are the relevant fields from the Caffe2 CMake summary to look at to check this:

--   Protobuf compiler     :
--   Protobuf include path :
--   Protobuf libraries    :

If that seems to be the issue, you can look at this troubleshooting point for a a way to resolve it.

[andongchen]

@cvpr are you fixed it? I also encountered this Segmentation fault! when I run test pass: "test_spatial_narrow_as_op.py

chenandong@titanXp:~/detectron/lib$ python2 ../tests/test_spatial_narrow_as_op.py E0313 10:29:22.601534 11181 init_intrinsics_check.cc:54] CPU feature avx is present on your machine, but the Caffe2 binary is not compiled with it. It means you may not get the full speed of your CPU. E0313 10:29:22.601588 11181 init_intrinsics_check.cc:54] CPU feature avx2 is present on your machine, but the Caffe2 binary is not compiled with it. It means you may not get the full speed of your CPU. E0313 10:29:22.601600 11181 init_intrinsics_check.cc:54] CPU feature fma is present on your machine, but the Caffe2 binary is not compiled with it. It means you may not get the full speed of your CPU. Found Detectron ops lib: /usr/local/lib/libcaffe2_detectron_ops_gpu.so .. Aborted at 1520908169 (unix time) try "date -d @1520908169" if you are using GNU date PC: @ 0x7f2fc662809c caffe2::GradientMakerBase::Get() SIGSEGV (@0xc981078) received by PID 11181 (TID 0x7f2fda52b740) from PID 211292280; stack trace: @ 0x7f2fda128330 (unknown) @ 0x7f2fc662809c caffe2::GradientMakerBase::Get() @ 0x7f2fcfee21e7 caffe2::GetGradientForOp() @ 0x7f2fd53733be _ZZN8pybind1112cpp_function10initializeIZN6caffe26python16addObjectMethodsERNS_6moduleEEUlNS_5bytesESt6vectorINS2_15GradientWrapperESaIS8_EEE28_St4pairIS7_IS6_SaIS6_EESA_EJS6_SA_EJNS_4nameENS_5scopeENS_7siblingENS_19return_value_policyEEEEvOT_PFT0_DpT1_EDpRKT2_ENUlRNS_6detail13function_callEE1_4FUNESX @ 0x7f2fd5394f90 pybind11::cpp_function::dispatcher() @ 0x4c7f54 (unknown) @ 0x4cfedc (unknown) @ 0x4c9419 (unknown) @ 0x4cfedc (unknown) @ 0x4c9419 (unknown) @ 0x4cfedc (unknown) @ 0x4c8314 (unknown) @ 0x4cfedc (unknown) @ 0x4c8314 (unknown) @ 0x4cfedc (unknown) @ 0x4c9419 (unknown) @ 0x4704ea (unknown) @ 0x4c9aa5 (unknown) @ 0x4704ea (unknown) @ 0x4d8194 (unknown) @ 0x50e9af (unknown) @ 0x4c84a5 (unknown) @ 0x4704ea (unknown) @ 0x4c9aa5 (unknown) @ 0x4704ea (unknown) @ 0x4d8194 (unknown) @ 0x50e9af (unknown) @ 0x4c84a5 (unknown) @ 0x4704ea (unknown) @ 0x4c9aa5 (unknown) @ 0x4704ea (unknown) @ 0x4d8194 (unknown) Segmentation fault

[LonelyHunter]

@andongchen I have the same problem and I remake using 'sudo make install'. It works for me, you can try it.

[khanhha]

I suffer from the same problem. E.terminate called without an active exception Aborted at 1535039318 (unix time) try "date -d @1535039318" if you are using GNU date PC: @ 0x7f65d1b3b428 gsignal SIGABRT (@0x3e800004e7c) received by PID 20092 (TID 0x7f65d2dda700) from PID 20092; stack trace: @ 0x7f65d25f1390 (unknown) @ 0x7f65d1b3b428 gsignal @ 0x7f65d1b3d02a abort @ 0x7f65ad5b23df __gnu_cxx::verbose_terminate_handler() @ 0x7f65ad5b0b16 cxxabiv1::__terminate() @ 0x7f65ad5b0b4c std::terminate() @ 0x7f65568630c0 caffe2::SpatialNarrowAsOp<>::~SpatialNarrowAsOp() @ 0x7f65ab855a85 caffe2::Workspace::RunOperatorOnce() @ 0x7f65ac7ee8d8 _ZZN6caffe26python16addGlobalMethodsERN8pybind116moduleEENKUlRKNS1_5bytesEE26clES6.isra.3091.constprop.3151 @ 0x7f65ac7eea84 _ZZN8pybind1112cpp_function10initializeIZN6caffe26python16addGlobalMethodsERNS_6moduleEEUlRKNS_5bytesEE26_bJS8_EJNS_4nameENS_5scopeENS_7siblingEEEEvOT_PFT0_DpT1_EDpRKT2_ENUlRNS_6detail13function_callEE1_4FUNESQ @ 0x7f65ac822460 pybind11::cpp_function::dispatcher() @ 0x7f65d28dfcc4 PyEval_EvalFrameEx @ 0x7f65d28e14e9 PyEval_EvalCodeEx @ 0x7f65d28de9b8 PyEval_EvalFrameEx @ 0x7f65d28e14e9 PyEval_EvalCodeEx @ 0x7f65d28de9b8 PyEval_EvalFrameEx @ 0x7f65d28e14e9 PyEval_EvalCodeEx @ 0x7f65d28de9b8 PyEval_EvalFrameEx @ 0x7f65d28e14e9 PyEval_EvalCodeEx @ 0x7f65d286a377 function_call @ 0x7f65d28457a3 PyObject_Call @ 0x7f65d28da4be PyEval_EvalFrameEx @ 0x7f65d28e14e9 PyEval_EvalCodeEx @ 0x7f65d286a28a function_call @ 0x7f65d28457a3 PyObject_Call @ 0x7f65d285463d instancemethod_call @ 0x7f65d28457a3 PyObject_Call @ 0x7f65d289e8a4 slot_tp_call @ 0x7f65d28457a3 PyObject_Call @ 0x7f65d28dbb69 PyEval_EvalFrameEx @ 0x7f65d28e14e9 PyEval_EvalCodeEx @ 0x7f65d286a377 function_call Aborted (core dumped)

[azmathmoosa]

Yea same here. I am trying to run on python3. I compiled for python 3, CUDA 9.2, cuDNN 7.1.4, gcc 6.4, Ubuntu 18.04.

[E init_intrinsics_check.cc:43] CPU feature avx is present on your machine, but the Caffe2 binary is not compiled with it. It means you may not get the full speed of your CPU.
[E init_intrinsics_check.cc:43] CPU feature avx2 is present on your machine, but the Caffe2 binary is not compiled with it. It means you may not get the full speed of your CPU.
[E init_intrinsics_check.cc:43] CPU feature fma is present on your machine, but the Caffe2 binary is not compiled with it. It means you may not get the full speed of your CPU.
Found Detectron ops lib: /usr/local/lib/libcaffe2_detectron_ops_gpu.so
Segmentation fault (core dumped)