xw-hu / SINet

IEEE Transactions on Intelligent Transportation Systems (TITS), 2019
Other
80 stars 22 forks source link

train_first_stage #8

Closed zq130320339 closed 6 years ago

zq130320339 commented 6 years ago

I0602 16:14:22.743806 1117 layer_factory.hpp:76] Creating layer data I0602 16:14:22.743829 1117 net.cpp:106] Creating Layer data I0602 16:14:22.743836 1117 net.cpp:411] data -> data I0602 16:14:22.743855 1117 net.cpp:411] data -> label_1_5x5 I0602 16:14:22.743863 1117 net.cpp:411] data -> label_1_7x7 I0602 16:14:22.743870 1117 net.cpp:411] data -> label_2_5x5 I0602 16:14:22.743873 1117 net.cpp:411] data -> label_2_7x7 I0602 16:14:22.743880 1117 net.cpp:411] data -> label_3_5x5 I0602 16:14:22.743885 1117 net.cpp:411] data -> label_3_7x7 I0602 16:14:22.743890 1117 net.cpp:411] data -> label_4_5x5 I0602 16:14:22.743893 1117 net.cpp:411] data -> label_4_7x7 I0602 16:14:22.743914 1117 image_gt_data_layer.cpp:55] Window data layer: batch size: 1 cache_images: 0 root_folder: I0602 16:14:22.743960 1117 image_gt_data_layer.cpp:160] num: 0 /home/zq/SINet/data/car_data/ImageSets/train_images/e3d5c101-6bb9-48af-ac80-08c73304456b.jpg 3 500 1069 windows to process: 2, RONI windows: 0 I0602 16:14:22.743965 1117 image_gt_data_layer.cpp:170] Number of images: 1 I0602 16:14:22.743968 1117 image_gt_data_layer.cpp:171] Number of colleting images: 1 I0602 16:14:22.743970 1117 image_gt_data_layer.cpp:177] class 0 has 0 samples I0602 16:14:22.743974 1117 image_gt_data_layer.cpp:177] class 1 has 2 samples I0602 16:14:22.743976 1117 image_gt_data_layer.cpp:183] Shuffling data I0602 16:14:22.751696 1117 image_gt_data_layer.cpp:216] output data size: 1,3,768,1024 I0602 16:14:22.751745 1117 image_gt_data_layer.cpp:238] output label size 0 : 1,6,96,128 I0602 16:14:22.751780 1117 image_gt_data_layer.cpp:238] output label size 1 : 1,6,96,128 I0602 16:14:22.751816 1117 image_gt_data_layer.cpp:238] output label size 2 : 1,6,48,64 I0602 16:14:22.751848 1117 image_gt_data_layer.cpp:238] output label size 3 : 1,6,48,64 I0602 16:14:22.751880 1117 image_gt_data_layer.cpp:238] output label size 4 : 1,6,24,32 I0602 16:14:22.751912 1117 image_gt_data_layer.cpp:238] output label size 5 : 1,6,24,32 I0602 16:14:22.751946 1117 image_gt_data_layer.cpp:238] output label size 6 : 1,6,12,16 I0602 16:14:22.751979 1117 image_gt_data_layer.cpp:238] output label size 7 : 1,6,12,16 I0602 16:14:22.769740 1117 net.cpp:150] Setting up data I0602 16:14:22.769767 1117 net.cpp:157] Top shape: 1 3 768 1024 (2359296) I0602 16:14:22.769771 1117 net.cpp:157] Top shape: 1 6 96 128 (73728) I0602 16:14:22.769774 1117 net.cpp:157] Top shape: 1 6 96 128 (73728) I0602 16:14:22.769778 1117 net.cpp:157] Top shape: 1 6 48 64 (18432) I0602 16:14:22.769781 1117 net.cpp:157] Top shape: 1 6 48 64 (18432) I0602 16:14:22.769784 1117 net.cpp:157] Top shape: 1 6 24 32 (4608) I0602 16:14:22.769788 1117 net.cpp:157] Top shape: 1 6 24 32 (4608) I0602 16:14:22.769793 1117 net.cpp:157] Top shape: 1 6 12 16 (1152) I0602 16:14:22.769795 1117 net.cpp:157] Top shape: 1 6 12 16 (1152) I0602 16:14:22.769798 1117 net.cpp:165] Memory required for data: 10220544 I0602 16:14:22.769804 1117 layer_factory.hpp:76] Creating layer label_1_5x5_data_1_split I0602 16:14:22.769815 1117 net.cpp:106] Creating Layer label_1_5x5_data_1_split I0602 16:14:22.769822 1117 net.cpp:454] label_1_5x5_data_1_split <- label_1_5x5 I0602 16:14:22.769835 1117 net.cpp:411] label_1_5x5_data_1_split -> label_1_5x5_data_1_split_0 I0602 16:14:22.769843 1117 net.cpp:411] label_1_5x5_data_1_split -> label_1_5x5_data_1_split_1 I0602 16:14:22.769912 1117 net.cpp:150] Setting up label_1_5x5_data_1_split I0602 16:14:22.769917 1117 net.cpp:157] Top shape: 1 6 96 128 (73728) I0602 16:14:22.769922 1117 net.cpp:157] Top shape: 1 6 96 128 (73728) I0602 16:14:22.769923 1117 net.cpp:165] Memory required for data: 10810368 I0602 16:14:22.769925 1117 layer_factory.hpp:76] Creating layer label_1_7x7_data_2_split I0602 16:14:22.769932 1117 net.cpp:106] Creating Layer label_1_7x7_data_2_split I0602 16:14:22.769933 1117 net.cpp:454] label_1_7x7_data_2_split <- label_1_7x7 I0602 16:14:22.769939 1117 net.cpp:411] label_1_7x7_data_2_split -> label_1_7x7_data_2_split_0 I0602 16:14:22.769944 1117 net.cpp:411] label_1_7x7_data_2_split -> label_1_7x7_data_2_split_1 I0602 16:14:22.769973 1117 net.cpp:150] Setting up label_1_7x7_data_2_split I0602 16:14:22.769979 1117 net.cpp:157] Top shape: 1 6 96 128 (73728) I0602 16:14:22.769982 1117 net.cpp:157] Top shape: 1 6 96 128 (73728) I0602 16:14:22.769984 1117 net.cpp:165] Memory required for data: 11400192 I0602 16:14:22.769987 1117 layer_factory.hpp:76] Creating layer label_2_5x5_data_3_split I0602 16:14:22.769991 1117 net.cpp:106] Creating Layer label_2_5x5_data_3_split I0602 16:14:22.769994 1117 net.cpp:454] label_2_5x5_data_3_split <- label_2_5x5 I0602 16:14:22.770000 1117 net.cpp:411] label_2_5x5_data_3_split -> label_2_5x5_data_3_split_0 I0602 16:14:22.770006 1117 net.cpp:411] label_2_5x5_data_3_split -> label_2_5x5_data_3_split_1 I0602 16:14:22.770035 1117 net.cpp:150] Setting up label_2_5x5_data_3_split I0602 16:14:22.770040 1117 net.cpp:157] Top shape: 1 6 48 64 (18432) I0602 16:14:22.770045 1117 net.cpp:157] Top shape: 1 6 48 64 (18432) I0602 16:14:22.770046 1117 net.cpp:165] Memory required for data: 11547648 I0602 16:14:22.770050 1117 layer_factory.hpp:76] Creating layer label_2_7x7_data_4_split I0602 16:14:22.770054 1117 net.cpp:106] Creating Layer label_2_7x7_data_4_split I0602 16:14:22.770057 1117 net.cpp:454] label_2_7x7_data_4_split <- label_2_7x7 I0602 16:14:22.770063 1117 net.cpp:411] label_2_7x7_data_4_split -> label_2_7x7_data_4_split_0 I0602 16:14:22.770069 1117 net.cpp:411] label_2_7x7_data_4_split -> label_2_7x7_data_4_split_1 I0602 16:14:22.770097 1117 net.cpp:150] Setting up label_2_7x7_data_4_split I0602 16:14:22.770102 1117 net.cpp:157] Top shape: 1 6 48 64 (18432) I0602 16:14:22.770104 1117 net.cpp:157] Top shape: 1 6 48 64 (18432) I0602 16:14:22.770107 1117 net.cpp:165] Memory required for data: 11695104 I0602 16:14:22.770109 1117 layer_factory.hpp:76] Creating layer label_3_5x5_data_5_split I0602 16:14:22.770114 1117 net.cpp:106] Creating Layer label_3_5x5_data_5_split I0602 16:14:22.770117 1117 net.cpp:454] label_3_5x5_data_5_split <- label_3_5x5 I0602 16:14:22.770121 1117 net.cpp:411] label_3_5x5_data_5_split -> label_3_5x5_data_5_split_0 I0602 16:14:22.770129 1117 net.cpp:411] label_3_5x5_data_5_split -> label_3_5x5_data_5_split_1 I0602 16:14:22.770150 1117 net.cpp:150] Setting up label_3_5x5_data_5_split I0602 16:14:22.770156 1117 net.cpp:157] Top shape: 1 6 24 32 (4608) I0602 16:14:22.770159 1117 net.cpp:157] Top shape: 1 6 24 32 (4608) I0602 16:14:22.770161 1117 net.cpp:165] Memory required for data: 11731968 I0602 16:14:22.770164 1117 layer_factory.hpp:76] Creating layer label_3_7x7_data_6_split I0602 16:14:22.770170 1117 net.cpp:106] Creating Layer label_3_7x7_data_6_split I0602 16:14:22.770172 1117 net.cpp:454] label_3_7x7_data_6_split <- label_3_7x7 I0602 16:14:22.770177 1117 net.cpp:411] label_3_7x7_data_6_split -> label_3_7x7_data_6_split_0 I0602 16:14:22.770184 1117 net.cpp:411] label_3_7x7_data_6_split -> label_3_7x7_data_6_split_1 I0602 16:14:22.770206 1117 net.cpp:150] Setting up label_3_7x7_data_6_split I0602 16:14:22.770211 1117 net.cpp:157] Top shape: 1 6 24 32 (4608) I0602 16:14:22.770215 1117 net.cpp:157] Top shape: 1 6 24 32 (4608) I0602 16:14:22.770217 1117 net.cpp:165] Memory required for data: 11768832 I0602 16:14:22.770220 1117 layer_factory.hpp:76] Creating layer label_4_5x5_data_7_split I0602 16:14:22.770225 1117 net.cpp:106] Creating Layer label_4_5x5_data_7_split I0602 16:14:22.770226 1117 net.cpp:454] label_4_5x5_data_7_split <- label_4_5x5 I0602 16:14:22.770236 1117 net.cpp:411] label_4_5x5_data_7_split -> label_4_5x5_data_7_split_0 I0602 16:14:22.770242 1117 net.cpp:411] label_4_5x5_data_7_split -> label_4_5x5_data_7_split_1 I0602 16:14:22.770262 1117 net.cpp:150] Setting up label_4_5x5_data_7_split I0602 16:14:22.770267 1117 net.cpp:157] Top shape: 1 6 12 16 (1152) I0602 16:14:22.770270 1117 net.cpp:157] Top shape: 1 6 12 16 (1152) I0602 16:14:22.770273 1117 net.cpp:165] Memory required for data: 11778048 I0602 16:14:22.770275 1117 layer_factory.hpp:76] Creating layer label_4_7x7_data_8_split I0602 16:14:22.770283 1117 net.cpp:106] Creating Layer label_4_7x7_data_8_split I0602 16:14:22.770287 1117 net.cpp:454] label_4_7x7_data_8_split <- label_4_7x7 I0602 16:14:22.770292 1117 net.cpp:411] label_4_7x7_data_8_split -> label_4_7x7_data_8_split_0 I0602 16:14:22.770303 1117 net.cpp:411] label_4_7x7_data_8_split -> label_4_7x7_data_8_split_1 I0602 16:14:22.770324 1117 net.cpp:150] Setting up label_4_7x7_data_8_split I0602 16:14:22.770329 1117 net.cpp:157] Top shape: 1 6 12 16 (1152) I0602 16:14:22.770334 1117 net.cpp:157] Top shape: 1 6 12 16 (1152) I0602 16:14:22.770335 1117 net.cpp:165] Memory required for data: 11787264 I0602 16:14:22.770337 1117 layer_factory.hpp:76] Creating layer conv1_1 I0602 16:14:22.770345 1117 net.cpp:106] Creating Layer conv1_1 I0602 16:14:22.770349 1117 net.cpp:454] conv1_1 <- data I0602 16:14:22.770352 1117 net.cpp:411] conv1_1 -> conv1_1 Aborted at 1527927262 (unix time) try "date -d @1527927262" if you are using GNU date PC: @ 0x7f2f507ef215 caffe::CuDNNConvolutionLayer<>::LayerSetUp() SIGFPE (@0x7f2f507ef215) received by PID 1117 (TID 0x7f2f513f9b00) from PID 1350496789; stack trace: @ 0x7f2f4eae34b0 (unknown) @ 0x7f2f507ef215 caffe::CuDNNConvolutionLayer<>::LayerSetUp() @ 0x7f2f506c42dc caffe::Net<>::Init() @ 0x7f2f506c5f31 caffe::Net<>::Net() @ 0x7f2f506d012a caffe::Solver<>::InitTrainNet() @ 0x7f2f506d06e7 caffe::Solver<>::Init() @ 0x7f2f506d0a59 caffe::Solver<>::Solver() @ 0x7f2f50711313 caffe::Creator_SGDSolver<>() @ 0x40ae30 train() @ 0x4076bc main @ 0x7f2f4eace830 __libc_start_main @ 0x407e69 _start @ 0x0 (unknown) Floating point exception (core dumped) 这是什么原因引起的?

xw-hu commented 6 years ago

I guess because there is zero sample for class 0. "I0602 16:14:22.743970 1117 image_gt_data_layer.cpp:177] class 0 has 0 samples"

zq130320339 commented 6 years ago

找到解决方法了,把cudnn注释掉就行了