search

BVLC / caffe

Caffe: a fast open framework for deep learning.
http://caffe.berkeleyvision.org/
Other
34.03k stars 18.7k forks source link

make runtest error #6829

Closed kli017 closed 1 year ago

kli017 commented 5 years ago

Important - read before submitting

Please read the guidelines for contributing before submitting this issue!

Please do not post installation, build, usage, or modeling questions, or other requests for help to Issues. Use the caffe-users list instead. This helps developers maintain a clear, uncluttered, and efficient view of the state of Caffe.

Issue summary

Steps to reproduce

Tried solutions

System configuration

Issue checklist

kli017 commented 5 years ago

Ubuntu16.04 CUDA 9.2 cundd7.6.3 python3.6.5

.build_release/tools/caffe caffe: command line brew usage: caffe

commands: train train or finetune a model test score a model device_query show GPU diagnostic information time benchmark model execution time

Flags from tools/caffe.cpp: -gpu (Optional; run in GPU mode on given device IDs separated by ','.Use '-gpu all' to run on all available GPUs. The effective training batch size is multiplied by the number of devices.) type: string default: "" -iterations (The number of iterations to run.) type: int32 default: 50 -level (Optional; network level.) type: int32 default: 0 -model (The model definition protocol buffer text file.) type: string default: "" -phase (Optional; network phase (TRAIN or TEST). Only used for 'time'.) type: string default: "" -sighup_effect (Optional; action to take when a SIGHUP signal is received: snapshot, stop or none.) type: string default: "snapshot" -sigint_effect (Optional; action to take when a SIGINT signal is received: snapshot, stop or none.) type: string default: "stop" -snapshot (Optional; the snapshot solver state to resume training.) type: string default: "" -solver (The solver definition protocol buffer text file.) type: string default: "" -stage (Optional; network stages (not to be confused with phase), separated by ','.) type: string default: "" -weights (Optional; the pretrained weights to initialize finetuning, separated by ','. Cannot be set simultaneously with snapshot.) type: string default: "" .build_release/test/test_all.testbin 0 --gtest_shuffle Cuda number of devices: 1 Setting to use device 0 Current device id: 0 Current device name: GeForce GTX 1080 Ti Note: Randomizing tests' orders with a seed of 62195 . [==========] Running 2207 tests from 285 test cases. [----------] Global test environment set-up. [----------] 5 tests from BlobSimpleTest/0, where TypeParam = float [ RUN ] BlobSimpleTest/0.TestInitialization [ OK ] BlobSimpleTest/0.TestInitialization (158 ms) [ RUN ] BlobSimpleTest/0.TestLegacyBlobProtoShapeEquals [ OK ] BlobSimpleTest/0.TestLegacyBlobProtoShapeEquals (0 ms) [ RUN ] BlobSimpleTest/0.TestReshape [ OK ] BlobSimpleTest/0.TestReshape (0 ms) [ RUN ] BlobSimpleTest/0.TestReshapeZero [ OK ] BlobSimpleTest/0.TestReshapeZero (0 ms) [ RUN ] BlobSimpleTest/0.TestPointersCPUGPU [ OK ] BlobSimpleTest/0.TestPointersCPUGPU (6 ms) [----------] 5 tests from BlobSimpleTest/0 (164 ms total)

[----------] 5 tests from BenchmarkTest/2, where TypeParam = caffe::GPUDevice [ RUN ] BenchmarkTest/2.TestTimerStop [ OK ] BenchmarkTest/2.TestTimerStop (0 ms) [ RUN ] BenchmarkTest/2.TestTimerMilliSeconds [ OK ] BenchmarkTest/2.TestTimerMilliSeconds (300 ms) [ RUN ] BenchmarkTest/2.TestTimerSeconds [ OK ] BenchmarkTest/2.TestTimerSeconds (301 ms) [ RUN ] BenchmarkTest/2.TestTimerStart [ OK ] BenchmarkTest/2.TestTimerStart (0 ms) [ RUN ] BenchmarkTest/2.TestTimerConstructor [ OK ] BenchmarkTest/2.TestTimerConstructor (0 ms) [----------] 5 tests from BenchmarkTest/2 (601 ms total)

[----------] 6 tests from CuDNNConvolutionLayerTest/0, where TypeParam = float [ RUN ] CuDNNConvolutionLayerTest/0.TestGradientCuDNN [ OK ] CuDNNConvolutionLayerTest/0.TestGradientCuDNN (1168 ms) [ RUN ] CuDNNConvolutionLayerTest/0.TestSobelConvolutionCuDNN [ OK ] CuDNNConvolutionLayerTest/0.TestSobelConvolutionCuDNN (4 ms) [ RUN ] CuDNNConvolutionLayerTest/0.TestSetupCuDNN [ OK ] CuDNNConvolutionLayerTest/0.TestSetupCuDNN (5 ms) [ RUN ] CuDNNConvolutionLayerTest/0.TestGradientGroupCuDNN [ OK ] CuDNNConvolutionLayerTest/0.TestGradientGroupCuDNN (227 ms) [ RUN ] CuDNNConvolutionLayerTest/0.TestSimpleConvolutionCuDNN [ OK ] CuDNNConvolutionLayerTest/0.TestSimpleConvolutionCuDNN (2 ms) [ RUN ] CuDNNConvolutionLayerTest/0.TestSimpleConvolutionGroupCuDNN [ OK ] CuDNNConvolutionLayerTest/0.TestSimpleConvolutionGroupCuDNN (4 ms) [----------] 6 tests from CuDNNConvolutionLayerTest/0 (1410 ms total)

[----------] 6 tests from MSRAFillerTest/0, where TypeParam = float [ RUN ] MSRAFillerTest/0.TestFill5D [ OK ] MSRAFillerTest/0.TestFill5D (0 ms) [ RUN ] MSRAFillerTest/0.TestFill1D [ OK ] MSRAFillerTest/0.TestFill1D (1 ms) [ RUN ] MSRAFillerTest/0.TestFillFanOut [ OK ] MSRAFillerTest/0.TestFillFanOut (24 ms) [ RUN ] MSRAFillerTest/0.TestFillFanIn [ OK ] MSRAFillerTest/0.TestFillFanIn (24 ms) [ RUN ] MSRAFillerTest/0.TestFill2D [ OK ] MSRAFillerTest/0.TestFill2D (1 ms) [ RUN ] MSRAFillerTest/0.TestFillAverage [ OK ] MSRAFillerTest/0.TestFillAverage (24 ms) [----------] 6 tests from MSRAFillerTest/0 (74 ms total)

[----------] 58 tests from NeuronLayerTest/1, where TypeParam = caffe::CPUDevice [ RUN ] NeuronLayerTest/1.TestAbsGradient [ OK ] NeuronLayerTest/1.TestAbsGradient (1 ms) [ RUN ] NeuronLayerTest/1.TestPReLUInPlace [ OK ] NeuronLayerTest/1.TestPReLUInPlace (0 ms) [ RUN ] NeuronLayerTest/1.TestTanH [ OK ] NeuronLayerTest/1.TestTanH (0 ms) [ RUN ] NeuronLayerTest/1.TestLogGradientBase2Shift1Scale3 [ OK ] NeuronLayerTest/1.TestLogGradientBase2Shift1Scale3 (2 ms) [ RUN ] NeuronLayerTest/1.TestClipGradient [ OK ] NeuronLayerTest/1.TestClipGradient (1 ms) [ RUN ] NeuronLayerTest/1.TestReLUGradientWithNegativeSlope [ OK ] NeuronLayerTest/1.TestReLUGradientWithNegativeSlope (1 ms) [ RUN ] NeuronLayerTest/1.TestBNLL [ OK ] NeuronLayerTest/1.TestBNLL (0 ms) [ RUN ] NeuronLayerTest/1.TestExpGradient [ OK ] NeuronLayerTest/1.TestExpGradient (1 ms) [ RUN ] NeuronLayerTest/1.TestReLU [ OK ] NeuronLayerTest/1.TestReLU (0 ms) [ RUN ] NeuronLayerTest/1.TestELUasReLUGradient [ OK ] NeuronLayerTest/1.TestELUasReLUGradient (1 ms) [ RUN ] NeuronLayerTest/1.TestDropoutHalf [ OK ] NeuronLayerTest/1.TestDropoutHalf (0 ms) [ RUN ] NeuronLayerTest/1.TestExpLayerBase2Shift1 [ OK ] NeuronLayerTest/1.TestExpLayerBase2Shift1 (0 ms) [ RUN ] NeuronLayerTest/1.TestLogGradientBase2 [ OK ] NeuronLayerTest/1.TestLogGradientBase2 (1 ms) [ RUN ] NeuronLayerTest/1.TestPReLUConsistencyReLU [ OK ] NeuronLayerTest/1.TestPReLUConsistencyReLU (0 ms) [ RUN ] NeuronLayerTest/1.TestSwishAsLinear [ OK ] NeuronLayerTest/1.TestSwishAsLinear (0 ms) [ RUN ] NeuronLayerTest/1.TestSwishGradient [ OK ] NeuronLayerTest/1.TestSwishGradient (2 ms) [ RUN ] NeuronLayerTest/1.TestSigmoidGradient [ OK ] NeuronLayerTest/1.TestSigmoidGradient (1 ms) [ RUN ] NeuronLayerTest/1.TestExpGradientBase2Shift1 [ OK ] NeuronLayerTest/1.TestExpGradientBase2Shift1 (1 ms) [ RUN ] NeuronLayerTest/1.TestReLUGradient [ OK ] NeuronLayerTest/1.TestReLUGradient (1 ms) [ RUN ] NeuronLayerTest/1.TestExpLayerWithShift [ OK ] NeuronLayerTest/1.TestExpLayerWithShift (0 ms) [ RUN ] NeuronLayerTest/1.TestAbsVal [ OK ] NeuronLayerTest/1.TestAbsVal (0 ms) [ RUN ] NeuronLayerTest/1.TestPReLUForward [ OK ] NeuronLayerTest/1.TestPReLUForward (0 ms) [ RUN ] NeuronLayerTest/1.TestLogLayerBase2Shift1Scale3 [ OK ] NeuronLayerTest/1.TestLogLayerBase2Shift1Scale3 (0 ms) [ RUN ] NeuronLayerTest/1.TestLogGradient [ OK ] NeuronLayerTest/1.TestLogGradient (1 ms) [ RUN ] NeuronLayerTest/1.TestBNLLGradient [ OK ] NeuronLayerTest/1.TestBNLLGradient (3 ms) [ RUN ] NeuronLayerTest/1.TestSwishWithBetaGradient [ OK ] NeuronLayerTest/1.TestSwishWithBetaGradient (1 ms) [ RUN ] NeuronLayerTest/1.TestExpGradientBase2 [ OK ] NeuronLayerTest/1.TestExpGradientBase2 (1 ms) [ RUN ] NeuronLayerTest/1.TestPReLUParam [ OK ] NeuronLayerTest/1.TestPReLUParam (0 ms) [ RUN ] NeuronLayerTest/1.TestSwishWithBeta [ OK ] NeuronLayerTest/1.TestSwishWithBeta (0 ms) [ RUN ] NeuronLayerTest/1.TestExpGradientBase2Shift1Scale3 [ OK ] NeuronLayerTest/1.TestExpGradientBase2Shift1Scale3 (2 ms) [ RUN ] NeuronLayerTest/1.TestLogLayerBase2Scale3 [ OK ] NeuronLayerTest/1.TestLogLayerBase2Scale3 (0 ms) [ RUN ] NeuronLayerTest/1.TestDropoutGradient [ OK ] NeuronLayerTest/1.TestDropoutGradient (1 ms) [ RUN ] NeuronLayerTest/1.TestDropoutGradientTest [ OK ] NeuronLayerTest/1.TestDropoutGradientTest (0 ms) [ RUN ] NeuronLayerTest/1.TestLogGradientBase2Scale3 [ OK ] NeuronLayerTest/1.TestLogGradientBase2Scale3 (2 ms) [ RUN ] NeuronLayerTest/1.TestPReLUForwardChannelShared [ OK ] NeuronLayerTest/1.TestPReLUForwardChannelShared (0 ms) [ RUN ] NeuronLayerTest/1.TestLogGradientBase2Shift1 [ OK ] NeuronLayerTest/1.TestLogGradientBase2Shift1 (1 ms) [ RUN ] NeuronLayerTest/1.TestExpGradientBase2Scale3 [ OK ] NeuronLayerTest/1.TestExpGradientBase2Scale3 (2 ms) [ RUN ] NeuronLayerTest/1.TestSigmoid [ OK ] NeuronLayerTest/1.TestSigmoid (0 ms) [ RUN ] NeuronLayerTest/1.TestDropoutTestPhase [ OK ] NeuronLayerTest/1.TestDropoutTestPhase (0 ms) [ RUN ] NeuronLayerTest/1.TestDropoutThreeQuarters [ OK ] NeuronLayerTest/1.TestDropoutThreeQuarters (0 ms) [ RUN ] NeuronLayerTest/1.TestExpGradientWithShift [ OK ] NeuronLayerTest/1.TestExpGradientWithShift (1 ms) [ RUN ] NeuronLayerTest/1.TestSwish [ OK ] NeuronLayerTest/1.TestSwish (0 ms) [ RUN ] NeuronLayerTest/1.TestELUasReLU [ OK ] NeuronLayerTest/1.TestELUasReLU (0 ms) [ RUN ] NeuronLayerTest/1.TestExpLayerBase2 [ OK ] NeuronLayerTest/1.TestExpLayerBase2 (0 ms) [ RUN ] NeuronLayerTest/1.TestSwishAsLinearGradient [ OK ] NeuronLayerTest/1.TestSwishAsLinearGradient (1 ms) [ RUN ] NeuronLayerTest/1.TestPReLUGradientChannelShared [ OK ] NeuronLayerTest/1.TestPReLUGradientChannelShared (55 ms) [ RUN ] NeuronLayerTest/1.TestELU [ OK ] NeuronLayerTest/1.TestELU (0 ms) [ RUN ] NeuronLayerTest/1.TestExpLayerBase2Scale3 [ OK ] NeuronLayerTest/1.TestExpLayerBase2Scale3 (0 ms) [ RUN ] NeuronLayerTest/1.TestELUGradient [ OK ] NeuronLayerTest/1.TestELUGradient (1 ms) [ RUN ] NeuronLayerTest/1.TestLogLayerBase2Shift1 [ OK ] NeuronLayerTest/1.TestLogLayerBase2Shift1 (0 ms) [ RUN ] NeuronLayerTest/1.TestExpLayerBase2Shift1Scale3 [ OK ] NeuronLayerTest/1.TestExpLayerBase2Shift1Scale3 (0 ms) [ RUN ] NeuronLayerTest/1.TestClip [ OK ] NeuronLayerTest/1.TestClip (0 ms) [ RUN ] NeuronLayerTest/1.TestReLUWithNegativeSlope [ OK ] NeuronLayerTest/1.TestReLUWithNegativeSlope (0 ms) [ RUN ] NeuronLayerTest/1.TestLogLayer [ OK ] NeuronLayerTest/1.TestLogLayer (0 ms) [ RUN ] NeuronLayerTest/1.TestTanHGradient [ OK ] NeuronLayerTest/1.TestTanHGradient (1 ms) [ RUN ] NeuronLayerTest/1.TestExpLayer [ OK ] NeuronLayerTest/1.TestExpLayer (0 ms) [ RUN ] NeuronLayerTest/1.TestPReLUGradient [ OK ] NeuronLayerTest/1.TestPReLUGradient (56 ms) [ RUN ] NeuronLayerTest/1.TestLogLayerBase2 [ OK ] NeuronLayerTest/1.TestLogLayerBase2 (0 ms) [----------] 58 tests from NeuronLayerTest/1 (142 ms total)

[----------] 14 tests from DataLayerTest/1, where TypeParam = caffe::CPUDevice [ RUN ] DataLayerTest/1.TestSkipLevelDB Aborted at 1567851757 (unix time) try "date -d @1567851757" if you are using GNU date PC: @ 0x7f50f2c90931 (unknown) SIGSEGV (@0x8b43fc17e) received by PID 3141 (TID 0x7f50b08e1700) from PID 18446744072438661502; stack trace: @ 0x7f50e8f3f390 (unknown) @ 0x7f50f2c90931 (unknown) @ 0x7f50ea3dc81e std::_Rb_tree<>::_M_insert_unique() @ 0x7f50ea3d49f8 boost::detail::add_new_tss_node() @ 0x7f50ea3d4c43 boost::detail::set_tss_data() @ 0x7f50e95ed3bd caffe::Caffe::Get() @ 0x7f50e9649cc0 caffe::InternalThread::entry() @ 0x7f50e964aade boost::detail::thread_data<>::run() @ 0x7f50ea3d25d5 (unknown) @ 0x7f50e8f356ba start_thread @ 0x7f50e8c6b41d clone @ 0x0 (unknown) Makefile:542: recipe for target 'runtest' failed

darcyzhc commented 4 years ago

Hello, I have the same problem for a few days, but I haven't solved it yet. Have you solved it? May you share something?

kli017 commented 4 years ago

@Darcyzh I have no idea whats going wrong.. Maybe is the verson of libboost doesnt match. Finally I just reinstall the ubuntu and solved.