IntelLabs / SkimCaffe

Caffe for Sparse Convolutional Neural Network
Other
238 stars 64 forks source link

Errorless compilation but some tests do not pass #22

Open nadir199 opened 4 years ago

nadir199 commented 4 years ago

Issue summary

When I compile Caffe cloned from the official repository, the compilation is successful, but when trying to compile SkimCaffe, I don't get any errors while compiling (only some warnings), but when I run make runtest, some tests pass, but some don't.

nadd@nadd-GL552VW:~/Tools/Compile/SkimCaffe$ make runtest make -C src/libxsmm AVX=2 OPT=3 DBG=0 FC= make[1]: Entering directory '/home/nadd/Tools/Compile/SkimCaffe/src/libxsmm' make[1]: Nothing to be done for 'libxsmm'. make[1]: Leaving directory '/home/nadd/Tools/Compile/SkimCaffe/src/libxsmm' make -C src/SpMP DBG=0 make[1]: Entering directory '/home/nadd/Tools/Compile/SkimCaffe/src/SpMP' make[1]: 'libspmp.a' is up to date. make[1]: Leaving directory '/home/nadd/Tools/Compile/SkimCaffe/src/SpMP' LD -o .build_release/lib/libcaffe.so.1.0.0-rc3 icpc -shared -o .build_release/lib/libcaffe.so.1.0.0-rc3 .build_release/src/caffe/proto/caffe.pb.o .build_release/src/caffe/data_transformer.o .build_release/src/caffe/layer_factory.o .build_release/src/caffe/net.o .build_release/src/caffe/data_reader.o .build_release/src/caffe/internal_thread.o .build_release/src/caffe/util/math_functions.o .build_release/src/caffe/util/math_functions_intel.o .build_release/src/caffe/util/im2col.o .build_release/src/caffe/util/signal_handler.o .build_release/src/caffe/util/benchmark.o .build_release/src/caffe/util/db_lmdb.o .build_release/src/caffe/util/cudnn.o .build_release/src/caffe/util/upgrade_proto.o .build_release/src/caffe/util/db.o .build_release/src/caffe/util/hdf5.o .build_release/src/caffe/util/mmio.o .build_release/src/caffe/util/blocking_queue.o .build_release/src/caffe/util/libxsmm_spmv.o .build_release/src/caffe/util/io.o .build_release/src/caffe/util/db_leveldb.o .build_release/src/caffe/util/cpu_info.o .build_release/src/caffe/util/insert_splits.o .build_release/src/caffe/solvers/adagrad_solver.o .build_release/src/caffe/solvers/nesterov_solver.o .build_release/src/caffe/solvers/rmsprop_solver.o .build_release/src/caffe/solvers/adadelta_solver.o .build_release/src/caffe/solvers/sgd_solver.o .build_release/src/caffe/solvers/adam_solver.o .build_release/src/caffe/syncedmem.o .build_release/src/caffe/layer.o .build_release/src/caffe/common.o .build_release/src/caffe/layers/mkl_relu_layer.o .build_release/src/caffe/layers/softmax_loss_layer.o .build_release/src/caffe/layers/bnll_layer.o .build_release/src/caffe/layers/euclidean_loss_layer.o .build_release/src/caffe/layers/hinge_loss_layer.o .build_release/src/caffe/layers/cudnn_lrn_layer.o .build_release/src/caffe/layers/cudnn_lcn_layer.o .build_release/src/caffe/layers/im2col_layer.o .build_release/src/caffe/layers/window_data_layer.o .build_release/src/caffe/layers/base_conv_layer.o .build_release/src/caffe/layers/mkl_convolution_layer.o .build_release/src/caffe/layers/cudnn_pooling_layer.o .build_release/src/caffe/layers/sigmoid_layer.o .build_release/src/caffe/layers/inner_product_layer.o .build_release/src/caffe/layers/silence_layer.o .build_release/src/caffe/layers/pooling_layer.o .build_release/src/caffe/layers/deconv_layer.o .build_release/src/caffe/layers/embed_layer.o .build_release/src/caffe/layers/eltwise_layer.o .build_release/src/caffe/layers/parameter_layer.o .build_release/src/caffe/layers/conv_relu_pool_layer.o .build_release/src/caffe/layers/relu_layer.o .build_release/src/caffe/layers/mkl_lrn_layer.o .build_release/src/caffe/layers/sigmoid_cross_entropy_loss_layer.o .build_release/src/caffe/layers/filter_layer.o .build_release/src/caffe/layers/conv_relu_pool_lrn_layer.o .build_release/src/caffe/layers/conv_relu_layer.o .build_release/src/caffe/layers/elu_layer.o .build_release/src/caffe/layers/inner_product_relu_dropout_layer.o .build_release/src/caffe/layers/contrastive_loss_layer.o .build_release/src/caffe/layers/reduction_layer.o .build_release/src/caffe/layers/input_layer.o .build_release/src/caffe/layers/concat_layer.o .build_release/src/caffe/layers/cudnn_conv_layer.o .build_release/src/caffe/layers/bias_layer.o .build_release/src/caffe/layers/absval_layer.o .build_release/src/caffe/layers/mkl_pooling_layer.o .build_release/src/caffe/layers/recurrent_layer.o .build_release/src/caffe/layers/multinomial_logistic_loss_layer.o .build_release/src/caffe/layers/data_layer.o .build_release/src/caffe/layers/image_data_layer.o .build_release/src/caffe/layers/mvn_layer.o .build_release/src/caffe/layers/memory_data_layer.o .build_release/src/caffe/layers/lstm_unit_layer.o .build_release/src/caffe/layers/dropout_layer.o .build_release/src/caffe/layers/mkl_eltwise_layer.o .build_release/src/caffe/layers/split_layer.o .build_release/src/caffe/layers/argmax_layer.o .build_release/src/caffe/layers/mkl_batch_norm_layer.o .build_release/src/caffe/layers/conv_layer.o .build_release/src/caffe/layers/reshape_layer.o .build_release/src/caffe/layers/mkl_concat_layer.o .build_release/src/caffe/layers/spp_layer.o .build_release/src/caffe/layers/accuracy_layer.o .build_release/src/caffe/layers/exp_layer.o .build_release/src/caffe/layers/crop_layer.o .build_release/src/caffe/layers/cudnn_sigmoid_layer.o .build_release/src/caffe/layers/power_layer.o .build_release/src/caffe/layers/tanh_layer.o .build_release/src/caffe/layers/softmax_layer.o .build_release/src/caffe/layers/cudnn_softmax_layer.o .build_release/src/caffe/layers/prelu_layer.o .build_release/src/caffe/layers/infogain_loss_layer.o .build_release/src/caffe/layers/cudnn_tanh_layer.o .build_release/src/caffe/layers/lstm_layer.o .build_release/src/caffe/layers/threshold_layer.o .build_release/src/caffe/layers/flatten_layer.o .build_release/src/caffe/layers/rnn_layer.o .build_release/src/caffe/layers/scale_layer.o .build_release/src/caffe/layers/cudnn_relu_layer.o .build_release/src/caffe/layers/batch_norm_layer.o .build_release/src/caffe/layers/slice_layer.o .build_release/src/caffe/layers/dummy_data_layer.o .build_release/src/caffe/layers/winograd_layer.o .build_release/src/caffe/layers/hdf5_data_layer.o .build_release/src/caffe/layers/hdf5_output_layer.o .build_release/src/caffe/layers/mkl_split_layer.o .build_release/src/caffe/layers/relu_pooling_layer.o .build_release/src/caffe/layers/base_data_layer.o .build_release/src/caffe/layers/log_layer.o .build_release/src/caffe/layers/neuron_layer.o .build_release/src/caffe/layers/batch_reindex_layer.o .build_release/src/caffe/layers/loss_layer.o .build_release/src/caffe/layers/lrn_layer.o .build_release/src/caffe/layers/tile_layer.o .build_release/src/caffe/parallel.o .build_release/src/caffe/solver.o .build_release/src/caffe/mkl_memory.o .build_release/src/caffe/blob.o -Wl,-soname,libcaffe.so.1.0.0-rc3 -Wl,-rpath,\$ORIGIN/../lib -qopenmp -pthread -fPIC -DCAFFE_VERSION=1.0.0-rc3 -DNDEBUG -O3 -DUSE_LEVELDB -DUSE_LMDB -DMKL2017_SUPPORTED -DCPU_ONLY -DUSE_MKL -I/usr/include/python2.7 -I/usr/lib/python2.7/dist-packages/numpy/core/include -I/usr/local/include -Isrc -Isrc/libxsmm/include -I/usr/local/cuda/include -I/usr/include/hdf5/serial/ -I.build_release/src -I./src -I./include -I/opt/intel/mkl/include -I/opt/intel/mkl/include/fftw -Wall -Wno-sign-compare -L/usr/lib -L/usr/local/lib -L/usr/lib -Lsrc/SpMP -Lsrc/libxsmm/lib -L/usr/local/cuda/lib64 -L/usr/lib/x86_64-linux-gnu/hdf5/serial -L/opt/intel/mkl/lib -L/opt/intel/mkl/lib/intel64 -L.build_release/lib -lglog -lgflags -lprotobuf -lboost_system -lboost_filesystem -lm -lhdf5_hl -lhdf5 -lspmp -lxsmm -lleveldb -lsnappy -llmdb -lboost_thread -lstdc++ -lmkl_rt .build_release/tools/caffe freq = 2.592e+09 caffe: command line brew usage: caffe

commands: train train or finetune a model test score a model device_query show GPU diagnostic information time benchmark model execution time

Flags from tools/caffe.cpp: -forward_only (Optional; Execute only forward pass) type: bool default: false -gpu (Optional; run in GPU mode on given device IDs separated by ','.Use '-gpu all' to run on all available GPUs. The effective training batch size is multiplied by the number of devices.) type: string default: "" -iterations (The number of iterations to run.) type: int32 default: 50 -level (Optional; network level.) type: int32 default: 0 -model (The model definition protocol buffer text file.) type: string default: "" -phase (Optional; network phase (TRAIN or TEST). Only used for 'time'.) type: string default: "" -sighup_effect (Optional; action to take when a SIGHUP signal is received: snapshot, stop or none.) type: string default: "snapshot" -sigint_effect (Optional; action to take when a SIGINT signal is received: snapshot, stop or none.) type: string default: "stop" -snapshot (Optional; the snapshot solver state to resume training.) type: string default: "" -solver (The solver definition protocol buffer text file.) type: string default: "" -stage (Optional; network stages (not to be confused with phase), separated by ','.) type: string default: "" -weights (Optional; the pretrained weights to initialize finetuning, separated by ','. Cannot be set simultaneously with snapshot.) type: string default: "" .build_release/test/test_all.testbin 0 --gtest_shuffle --gtest_filter="-GPU" Note: Google Test filter = -GPU Note: Randomizing tests' orders with a seed of 12730 .
[==========] Running 998 tests from 141 test cases. [----------] Global test environment set-up. [----------] 5 tests from EmbedLayerTest/1, where TypeParam = caffe::CPUDevice [ RUN ] EmbedLayerTest/1.TestForward [ OK ] EmbedLayerTest/1.TestForward (2 ms) [ RUN ] EmbedLayerTest/1.TestGradient [ OK ] EmbedLayerTest/1.TestGradient (30 ms) [ RUN ] EmbedLayerTest/1.TestForwardWithBias [ OK ] EmbedLayerTest/1.TestForwardWithBias (10 ms) [ RUN ] EmbedLayerTest/1.TestGradientWithBias [ OK ] EmbedLayerTest/1.TestGradientWithBias (51 ms) [ RUN ] EmbedLayerTest/1.TestSetUp [ OK ] EmbedLayerTest/1.TestSetUp (0 ms) [----------] 5 tests from EmbedLayerTest/1 (93 ms total)

[----------] 1 test from CPUStochasticPoolingLayerTest/0, where TypeParam = float [ RUN ] CPUStochasticPoolingLayerTest/0.TestSetup [ OK ] CPUStochasticPoolingLayerTest/0.TestSetup (0 ms) [----------] 1 test from CPUStochasticPoolingLayerTest/0 (0 ms total)

[----------] 1 test from MultinomialLogisticLossLayerTest/0, where TypeParam = float [ RUN ] MultinomialLogisticLossLayerTest/0.TestGradientCPU [ OK ] MultinomialLogisticLossLayerTest/0.TestGradientCPU (1 ms) [----------] 1 test from MultinomialLogisticLossLayerTest/0 (1 ms total)

[----------] 11 tests from RandomNumberGeneratorTest/1, where TypeParam = double [ RUN ] RandomNumberGeneratorTest/1.TestRngGaussianTimesBernoulli [ OK ] RandomNumberGeneratorTest/1.TestRngGaussianTimesBernoulli (0 ms) [ RUN ] RandomNumberGeneratorTest/1.TestRngBernoulli [ OK ] RandomNumberGeneratorTest/1.TestRngBernoulli (0 ms) [ RUN ] RandomNumberGeneratorTest/1.TestRngUniform2 [ OK ] RandomNumberGeneratorTest/1.TestRngUniform2 (0 ms) [ RUN ] RandomNumberGeneratorTest/1.TestRngBernoulli2 [ OK ] RandomNumberGeneratorTest/1.TestRngBernoulli2 (0 ms) [ RUN ] RandomNumberGeneratorTest/1.TestRngGaussian [ OK ] RandomNumberGeneratorTest/1.TestRngGaussian (0 ms) [ RUN ] RandomNumberGeneratorTest/1.TestRngUniformTimesBernoulli [ OK ] RandomNumberGeneratorTest/1.TestRngUniformTimesBernoulli (0 ms) [ RUN ] RandomNumberGeneratorTest/1.TestRngUniformTimesUniform [ OK ] RandomNumberGeneratorTest/1.TestRngUniformTimesUniform (1 ms) [ RUN ] RandomNumberGeneratorTest/1.TestRngGaussianTimesGaussian [ OK ] RandomNumberGeneratorTest/1.TestRngGaussianTimesGaussian (0 ms) [ RUN ] RandomNumberGeneratorTest/1.TestRngBernoulliTimesBernoulli [ OK ] RandomNumberGeneratorTest/1.TestRngBernoulliTimesBernoulli (0 ms) [ RUN ] RandomNumberGeneratorTest/1.TestRngUniform [ OK ] RandomNumberGeneratorTest/1.TestRngUniform (0 ms) [ RUN ] RandomNumberGeneratorTest/1.TestRngGaussian2 [ OK ] RandomNumberGeneratorTest/1.TestRngGaussian2 (0 ms) [----------] 11 tests from RandomNumberGeneratorTest/1 (1 ms total)

[----------] 26 tests from NetTest/1, where TypeParam = caffe::CPUDevice [ RUN ] NetTest/1.TestHasBlob [ OK ] NetTest/1.TestHasBlob (2 ms) [ RUN ] NetTest/1.TestHasLayer [ OK ] NetTest/1.TestHasLayer (1 ms) [ RUN ] NetTest/1.TestComboLossWeight F1029 04:22:19.551991 1528 inner_product_layer.cpp:215] Not Implemented Yet Check failure stack trace: @ 0x7f2cd181b0cd google::LogMessage::Fail() @ 0x7f2cd181cf33 google::LogMessage::SendToLog() @ 0x7f2cd181ac28 google::LogMessage::Flush() @ 0x7f2cd181d999 google::LogMessageFatal::~LogMessageFatal() @ 0x7f2cce80cb2a caffe::InnerProductLayer<>::Forward_cpu() @ 0x7f2cce67f43a caffe::Net<>::Forward() @ 0x89edc1 caffe::Net<>::ForwardBackward() @ 0x89a819 caffe::NetTest_TestComboLossWeight_Test<>::TestBody() @ 0xd17eee testing::Test::Run() @ 0xd17aa0 testing::TestInfo::Run() @ 0xd17772 testing::TestCase::Run() @ 0xd16bfc testing::internal::UnitTestImpl::RunAllTests() @ 0xd13466 testing::UnitTest::Run() @ 0x45bdbd main @ 0x7f2ccd86db97 __libc_start_main @ 0x45bc6a _start Aborted (core dumped) Makefile:561: recipe for target 'runtest' failed make: *** [runtest] Error 134

Steps to reproduce

Try to compile SkimCaffe: In SkimCaffe directory, execute : make libxsmm make all make test make runtest

Your system configuration

Operating system: Ubuntu 18 Compiler: icpc CUDA version (if applicable): No (CPU ONLY) CUDNN version (if applicable): BLAS: mkl Python or MATLAB version (for pycaffe and matcaffe respectively):

Thank you.