BVLC / caffe

Caffe: a fast open framework for deep learning.
http://caffe.berkeleyvision.org/
Other
33.96k stars 18.72k forks source link

*** Error in `./build/tools/caffe': double free or corruption (!prev): 0x00007f5e15685ae0 *** #6963

Open GeneralSong opened 3 years ago

GeneralSong commented 3 years ago

I0819 11:42:18.866453 4075 sgd_solver.cpp:121] Iteration 40, lr = 0.005 I0819 11:42:19.737269 4075 yolov3_layer.cpp:764] noobj: 0.00100563 obj: 0.032264 iou: 0.339615 cat: 0.932463 recall: 0 recall75: 0 count: 2 I0819 11:42:19.755069 4075 yolov3_layer.cpp:764] noobj: 0.00128897 obj: 0.104052 iou: 0.599589 cat: 0.938093 recall: 0.889011 recall75: 0.144872 count: 6 I0819 11:42:19.766554 4075 yolov3_layer.cpp:764] noobj: 0.00146171 obj: 0.0252739 iou: 0.454691 cat: 0.951491 recall: 0.415456 recall75: 0.0173611 count: 6 I0819 11:42:20.863711 4075 yolov3_layer.cpp:764] noobj: 0.00110726 obj: 0.0443788 iou: 0.468613 cat: 0.935797 recall: 0.527778 recall75: 0 count: 2 I0819 11:42:20.875820 4075 yolov3_layer.cpp:764] noobj: 0.00122865 obj: 0.11299 iou: 0.601507 cat: 0.935109 recall: 0.717857 recall75: 0.119079 count: 4 I0819 11:42:20.886536 4075 yolov3_layer.cpp:764] noobj: 0.00135975 obj: 0.0603908 iou: 0.422262 cat: 0.92223 recall: 0.239321 recall75: 0 count: 7 I0819 11:42:22.064728 4075 solver.cpp:253] Iteration 50 (3.1267 iter/s, 3.19826s/10 iters), loss = 20.4152 I0819 11:42:22.064775 4075 solver.cpp:272] Train net output #0: det_loss1 = 0.0010248 ( 1 = 0.0010248 loss) I0819 11:42:22.064782 4075 solver.cpp:272] Train net output #1: det_loss2 = 0.0069714 ( 1 = 0.0069714 loss) I0819 11:42:22.064790 4075 solver.cpp:272] Train net output #2: det_loss3 = 22.4805 ( 1 = 22.4805 loss) I0819 11:42:22.064823 4075 sgd_solver.cpp:121] Iteration 50, lr = 0.005 I0819 11:42:22.329406 4075 yolov3_layer.cpp:764] noobj: 0.000891113 obj: 0.0156935 iou: 0.36022 cat: 0.954072 recall: 0.26431 recall75: 0.030303 count: 4 I0819 11:42:22.342959 4075 yolov3_layer.cpp:764] noobj: 0.000768706 obj: 0.0704814 iou: 0.58861 cat: 0.955218 recall: 0.813741 recall75: 0.0503846 count: 7 I0819 11:42:22.352453 4075 yolov3_layer.cpp:764] noobj: 0.00126887 obj: 0.0329924 iou: 0.463708 cat: 0.950573 recall: 0.456349 recall75: 0.0305556 count: 5 I0819 11:42:23.698518 4075 yolov3_layer.cpp:764] noobj: 0.00132654 obj: 0.0216623 iou: 0.479753 cat: 0.943866 recall: 0.490741 recall75: 0 count: 3 I0819 11:42:23.716898 4075 yolov3_layer.cpp:764] noobj: 0.00127563 obj: 0.117333 iou: 0.613226 cat: 0.923023 recall: 0.807949 recall75: 0.196296 count: 5 I0819 11:42:23.732429 4075 yolov3_layer.cpp:764] noobj: 0.00223707 obj: 0.0832536 iou: 0.460457 cat: 0.91354 recall: 0.334127 recall75: 0.0833333 count: 4 I0819 11:42:25.227735 4075 yolov3_layer.cpp:764] noobj: 0.00134623 obj: 0.052429 iou: 0.521597 cat: 0.931675 recall: 0.585714 recall75: 0.1 count: 3 I0819 11:42:25.251714 4075 yolov3_layer.cpp:764] noobj: 0.00116677 obj: 0.0696168 iou: 0.58971 cat: 0.951298 recall: 0.758939 recall75: 0.06443 count: 8 I0819 11:42:25.274422 4075 yolov3_layer.cpp:764] noobj: 0.00116826 obj: 0.0406312 iou: 0.465481 cat: 0.94833 recall: 0.468553 recall75: 0.0459184 count: 7 I0819 11:42:25.830940 4075 solver.cpp:253] Iteration 60 (2.65525 iter/s, 3.76612s/10 iters), loss = 63.9877 I0819 11:42:25.830992 4075 solver.cpp:272] Train net output #0: det_loss1 = 17.3835 ( 1 = 17.3835 loss) I0819 11:42:25.831001 4075 solver.cpp:272] Train net output #1: det_loss2 = 15.6254 ( 1 = 15.6254 loss) I0819 11:42:25.831005 4075 solver.cpp:272] Train net output #2: det_loss3 = 5.14256 ( 1 = 5.14256 loss) I0819 11:42:25.831012 4075 sgd_solver.cpp:121] Iteration 60, lr = 0.005 I0819 11:42:26.874212 4075 yolov3_layer.cpp:764] noobj: 0.00144508 obj: 0.0429786 iou: 0.569429 cat: 0.945507 recall: 0.738248 recall75: 0.221154 count: 3 I0819 11:42:26.883137 4075 yolov3_layer.cpp:764] noobj: 0.00129891 obj: 0.0936807 iou: 0.586956 cat: 0.937779 recall: 0.761095 recall75: 0.0905185 count: 7 I0819 11:42:26.892017 4075 yolov3_layer.cpp:764] noobj: 0.00141043 obj: 0.0409192 iou: 0.486543 cat: 0.941867 recall: 0.436012 recall75: 0.0208333 count: 4 I0819 11:42:28.375890 4075 yolov3_layer.cpp:764] noobj: 0.00140399 obj: 0.0357858 iou: 0.629574 cat: 0.949481 recall: 0.866667 recall75: 0.133333 count: 2 I0819 11:42:28.389731 4075 yolov3_layer.cpp:764] noobj: 0.00152678 obj: 0.111072 iou: 0.642737 cat: 0.935812 recall: 0.848599 recall75: 0.246954 count: 8 I0819 11:42:28.399421 4075 yolov3_layer.cpp:764] noobj: 0.00168546 obj: 0.0775423 iou: 0.485011 cat: 0.917492 recall: 0.459246 recall75: 0.0461538 count: 6 I0819 11:42:29.278738 4075 solver.cpp:253] Iteration 70 (2.90048 iter/s, 3.44771s/10 iters), loss = 23.8302 I0819 11:42:29.278791 4075 solver.cpp:272] Train net output #0: det_loss1 = 0.00127567 ( 1 = 0.00127567 loss) I0819 11:42:29.278800 4075 solver.cpp:272] Train net output #1: det_loss2 = 0.00180707 ( 1 = 0.00180707 loss) I0819 11:42:29.278805 4075 solver.cpp:272] Train net output #2: det_loss3 = 27.2362 ( 1 = 27.2362 loss) I0819 11:42:29.278812 4075 sgd_solver.cpp:121] Iteration 70, lr = 0.005 I0819 11:42:29.477385 4075 yolov3_layer.cpp:764] noobj: 0.000934527 obj: 0.0325627 iou: 0.580151 cat: 0.876965 recall: 0.666667 recall75: 0.142857 count: 2 I0819 11:42:29.483287 4075 yolov3_layer.cpp:764] noobj: 0.00103876 obj: 0.074003 iou: 0.62788 cat: 0.946094 recall: 0.887151 recall75: 0.200079 count: 6 I0819 11:42:29.487725 4075 yolov3_layer.cpp:764] noobj: 0.00201401 obj: 0.0635966 iou: 0.47776 cat: 0.931495 recall: 0.431551 recall75: 0.030303 count: 8 I0819 11:42:30.541354 4075 yolov3_layer.cpp:764] noobj: 0.0006573 obj: 0.0292746 iou: 0.223909 cat: 0.933381 recall: 0 recall75: 0 count: 2 I0819 11:42:30.562857 4075 yolov3_layer.cpp:764] noobj: 0.00164987 obj: 0.138198 iou: 0.612398 cat: 0.939353 recall: 0.809802 recall75: 0.153525 count: 5 I0819 11:42:30.576962 4075 yolov3_layer.cpp:764] noobj: 0.00315994 obj: 0.096369 iou: 0.477158 cat: 0.927696 recall: 0.356693 recall75: 0 count: 4 I0819 11:42:31.635396 4075 yolov3_layer.cpp:764] noobj: 0.00119918 obj: 0.0804991 iou: 0.583229 cat: 0.948968 recall: 0.777778 recall75: 0.148148 count: 2 I0819 11:42:31.644361 4075 yolov3_layer.cpp:764] noobj: 0.00231864 obj: 0.179783 iou: 0.630225 cat: 0.93122 recall: 0.895633 recall75: 0.132862 count: 8 I0819 11:42:31.653267 4075 yolov3_layer.cpp:764] noobj: 0.0025058 obj: 0.111694 iou: 0.524703 cat: 0.928534 recall: 0.580927 recall75: 0.0333333 count: 6 I0819 11:42:31.895534 4075 solver.cpp:253] Iteration 80 (3.8216 iter/s, 2.61671s/10 iters), loss = 21.6013 I0819 11:42:31.895578 4075 solver.cpp:272] Train net output #0: det_loss1 = 5.41036 ( 1 = 5.41036 loss) I0819 11:42:31.895586 4075 solver.cpp:272] Train net output #1: det_loss2 = 7.68542 ( 1 = 7.68542 loss) I0819 11:42:31.895591 4075 solver.cpp:272] Train net output #2: det_loss3 = 8.79476 ( 1 = 8.79476 loss) I0819 11:42:31.895597 4075 sgd_solver.cpp:121] Iteration 80, lr = 0.005 I0819 11:42:32.630692 4075 yolov3_layer.cpp:764] noobj: 0.00068461 obj: 0.0680269 iou: 0.377373 cat: 0.922178 recall: 0 recall75: 0 count: 2 I0819 11:42:32.633198 4075 yolov3_layer.cpp:764] noobj: 0.00175152 obj: 0.109701 iou: 0.582671 cat: 0.947679 recall: 0.756216 recall75: 0.133283 count: 6 I0819 11:42:32.635646 4075 yolov3_layer.cpp:764] noobj: 0.0026309 obj: 0.104385 iou: 0.480191 cat: 0.935891 recall: 0.451786 recall75: 0.0357143 count: 5 I0819 11:42:34.170811 4075 yolov3_layer.cpp:764] noobj: 0.00142523 obj: 0.0502862 iou: 0.522537 cat: 0.955546 recall: 0.620833 recall75: 0 count: 4 I0819 11:42:34.175158 4075 yolov3_layer.cpp:764] noobj: 0.0021638 obj: 0.18605 iou: 0.616084 cat: 0.930202 recall: 0.873038 recall75: 0.0235043 count: 8 I0819 11:42:34.179402 4075 yolov3_layer.cpp:764] noobj: 0.00270334 obj: 0.0973229 iou: 0.436797 cat: 0.936816 recall: 0.323481 recall75: 0.0421245 count: 7 I0819 11:42:35.073659 4075 solver.cpp:253] Iteration 90 (3.14659 iter/s, 3.17804s/10 iters), loss = 36.1134 I0819 11:42:35.073915 4075 solver.cpp:272] Train net output #0: det_loss1 = 0.0288001 ( 1 = 0.0288001 loss) I0819 11:42:35.073925 4075 solver.cpp:272] Train net output #1: det_loss2 = 4.42082 ( 1 = 4.42082 loss) I0819 11:42:35.073931 4075 solver.cpp:272] Train net output #2: det_loss3 = 14.3548 ( 1 = 14.3548 loss) I0819 11:42:35.073940 4075 sgd_solver.cpp:121] Iteration 90, lr = 0.005 I0819 11:42:35.320313 4075 yolov3_layer.cpp:764] noobj: 0.00146726 obj: 0.0597758 iou: 0.458683 cat: 0.962602 recall: 0.5 recall75: 0 count: 2 I0819 11:42:35.353832 4075 yolov3_layer.cpp:764] noobj: 0.00174725 obj: 0.120951 iou: 0.627141 cat: 0.944155 recall: 0.92161 recall75: 0.0640443 count: 8 I0819 11:42:35.387277 4075 yolov3_layer.cpp:764] noobj: 0.00172872 obj: 0.117688 iou: 0.487386 cat: 0.93654 recall: 0.547619 recall75: 0.025641 count: 6 I0819 11:42:37.023175 4075 yolov3_layer.cpp:764] noobj: 0.00178251 obj: 0.12897 iou: 0.32148 cat: 0.935032 recall: 0.141389 recall75: 0.0125 count: 4 I0819 11:42:37.030468 4075 yolov3_layer.cpp:764] noobj: 0.00163471 obj: 0.154741 iou: 0.630344 cat: 0.936312 recall: 0.847985 recall75: 0.155594 count: 8 I0819 11:42:37.037722 4075 yolov3_layer.cpp:764] noobj: 0.00118618 obj: 0.072498 iou: 0.488071 cat: 0.954717 recall: 0.4475 recall75: 0.0591667 count: 5 I0819 11:42:37.982605 4075 yolov3_layer.cpp:764] noobj: 0.00112142 obj: 0.0468273 iou: 0.535802 cat: 0.96545 recall: 0.533333 recall75: 0.2 count: 1 I0819 11:42:37.989929 4075 yolov3_layer.cpp:764] noobj: 0.00182985 obj: 0.114852 iou: 0.649605 cat: 0.946337 recall: 0.886111 recall75: 0.211458 count: 11 I0819 11:42:37.997181 4075 yolov3_layer.cpp:764] noobj: 0.00270029 obj: 0.0713871 iou: 0.407763 cat: 0.943458 recall: 0.370971 recall75: 0.00649351 count: 8 I0819 11:42:38.505374 4075 solver.cpp:253] Iteration 100 (2.91425 iter/s, 3.43142s/10 iters), loss = 63.0081 I0819 11:42:38.505429 4075 solver.cpp:272] Train net output #0: det_loss1 = 6.79009 ( 1 = 6.79009 loss) I0819 11:42:38.505435 4075 solver.cpp:272] Train net output #1: det_loss2 = 20.5059 ( 1 = 20.5059 loss) I0819 11:42:38.505442 4075 solver.cpp:272] Train net output #2: det_loss3 = 4.30649 ( 1 = 4.30649 loss) I0819 11:42:38.505450 4075 sgd_solver.cpp:121] Iteration 100, lr = 0.005 I0819 11:42:39.856153 4075 yolov3_layer.cpp:764] noobj: 0.00205681 obj: 0.135334 iou: 0.47676 cat: 0.932523 recall: 0.472727 recall75: 0.0272727 count: 3 I0819 11:42:39.878238 4075 yolov3_layer.cpp:764] noobj: 0.00277819 obj: 0.188572 iou: 0.630207 cat: 0.93234 recall: 0.8525 recall75: 0.168187 count: 11 I0819 11:42:39.899009 4075 yolov3_layer.cpp:764] noobj: 0.00315983 obj: 0.151978 iou: 0.471547 cat: 0.918079 recall: 0.439385 recall75: 0.0583333 count: 6 I0819 11:42:40.964285 4075 yolov3_layer.cpp:764] noobj: 0.00120604 obj: 0.122853 iou: 0.61514 cat: 0.956074 recall: 0.76 recall75: 0 count: 4 I0819 11:42:40.976162 4075 yolov3_layer.cpp:764] noobj: 0.00150233 obj: 0.15587 iou: 0.649348 cat: 0.952183 recall: 0.880449 recall75: 0.0957265 count: 7 I0819 11:42:40.988202 4075 yolov3_layer.cpp:764] noobj: 0.00157249 obj: 0.122949 iou: 0.481219 cat: 0.93284 recall: 0.499893 recall75: 0.0128205 count: 5 I0819 11:42:41.852511 4075 solver.cpp:253] Iteration 110 (2.98771 iter/s, 3.34704s/10 iters), loss = 27.4577 I0819 11:42:41.852556 4075 solver.cpp:272] Train net output #0: det_loss1 = 0.0158869 ( 1 = 0.0158869 loss) I0819 11:42:41.852563 4075 solver.cpp:272] Train net output #1: det_loss2 = 0.0170665 ( 1 = 0.0170665 loss) I0819 11:42:41.852568 4075 solver.cpp:272] Train net output #2: det_loss3 = 0.0222671 ( 1 = 0.0222671 loss) I0819 11:42:41.852576 4075 sgd_solver.cpp:121] Iteration 110, lr = 0.005 I0819 11:42:41.980340 4075 yolov3_layer.cpp:764] noobj: 0.00105913 obj: 0.0953398 iou: 0.508055 cat: 0.953485 recall: 0.595238 recall75: 0.0277778 count: 3 I0819 11:42:41.987910 4075 yolov3_layer.cpp:764] noobj: 0.00186722 obj: 0.152729 iou: 0.656379 cat: 0.944857 recall: 0.916861 recall75: 0.226116 count: 8 I0819 11:42:41.995223 4075 yolov3_layer.cpp:764] noobj: 0.00206614 obj: 0.0874918 iou: 0.436661 cat: 0.936148 recall: 0.41936 recall75: 0.0138889 count: 6 I0819 11:42:43.419327 4075 yolov3_layer.cpp:764] noobj: 0.00142951 obj: 0.0202146 iou: 0.394845 cat: 0.920546 recall: 0 recall75: 0 count: 2 I0819 11:42:43.450188 4075 yolov3_layer.cpp:764] noobj: 0.00157001 obj: 0.177976 iou: 0.683359 cat: 0.945793 recall: 0.924074 recall75: 0.333069 count: 5 I0819 11:42:43.479710 4075 yolov3_layer.cpp:764] noobj: 0.00260429 obj: 0.144902 iou: 0.477485 cat: 0.939793 recall: 0.503226 recall75: 0.0256944 count: 8 I0819 11:42:45.273149 4075 yolov3_layer.cpp:764] noobj: 0.00161606 obj: 0.117947 iou: 0.581935 cat: 0.941798 recall: 0.771429 recall75: 0.142857 count: 3 I0819 11:42:45.279295 4075 yolov3_layer.cpp:764] noobj: 0.00139028 obj: 0.158777 iou: 0.639101 cat: 0.944455 recall: 0.960631 recall75: 0.0980413 count: 9 I0819 11:42:45.285356 4075 yolov3_layer.cpp:764] noobj: 0.00139923 obj: 0.121996 iou: 0.455745 cat: 0.952231 recall: 0.384982 recall75: 0.010989 count: 5 I0819 11:42:45.793957 4075 solver.cpp:253] Iteration 120 (2.5372 iter/s, 3.94135s/10 iters), loss = 45.6905 I0819 11:42:45.794008 4075 solver.cpp:272] Train net output #0: det_loss1 = 0.183283 ( 1 = 0.183283 loss) I0819 11:42:45.794015 4075 solver.cpp:272] Train net output #1: det_loss2 = 16.2518 ( 1 = 16.2518 loss) I0819 11:42:45.794021 4075 solver.cpp:272] Train net output #2: det_loss3 = 11.3677 ( 1 = 11.3677 loss) I0819 11:42:45.794028 4075 sgd_solver.cpp:121] Iteration 120, lr = 0.005 I0819 11:42:46.817067 4075 yolov3_layer.cpp:764] noobj: 0.00102834 obj: 0.039228 iou: 0.496394 cat: 0.938405 recall: 0.375 recall75: 0.1875 count: 5 I0819 11:42:46.834674 4075 yolov3_layer.cpp:764] noobj: 0.00142745 obj: 0.19146 iou: 0.659532 cat: 0.945991 recall: 0.982351 recall75: 0.129056 count: 9 I0819 11:42:46.849304 4075 yolov3_layer.cpp:764] noobj: 0.00128746 obj: 0.0889695 iou: 0.533092 cat: 0.949509 recall: 0.649306 recall75: 0.0527778 count: 6 I0819 11:42:48.008698 4075 yolov3_layer.cpp:764] noobj: 0.00112133 obj: 0.0917656 iou: 0.587505 cat: 0.944788 recall: 0.791667 recall75: 0.169048 count: 3 I0819 11:42:48.023217 4075 yolov3_layer.cpp:764] noobj: 0.00129531 obj: 0.126619 iou: 0.650881 cat: 0.955609 recall: 0.955556 recall75: 0.172222 count: 6 I0819 11:42:48.032644 4075 yolov3_layer.cpp:764] noobj: 0.00245534 obj: 0.118864 iou: 0.429939 cat: 0.918746 recall: 0.375992 recall75: 0 count: 4 I0819 11:42:49.077349 4075 solver.cpp:253] Iteration 130 (3.04572 iter/s, 3.2833s/10 iters), loss = 25.8584 I0819 11:42:49.077395 4075 solver.cpp:272] Train net output #0: det_loss1 = 0.0195454 ( 1 = 0.0195454 loss) I0819 11:42:49.077402 4075 solver.cpp:272] Train net output #1: det_loss2 = 6.06656 ( 1 = 6.06656 loss) I0819 11:42:49.077409 4075 solver.cpp:272] Train net output #2: det_loss3 = 0.0624069 ( 1 = 0.0624069 loss) I0819 11:42:49.077415 4075 sgd_solver.cpp:121] Iteration 130, lr = 0.005 I0819 11:42:49.291328 4075 yolov3_layer.cpp:764] noobj: 0.000724645 obj: 0.05987 iou: 0.565091 cat: 0.961278 recall: 0.8 recall75: 0.2 count: 1 I0819 11:42:49.310643 4075 yolov3_layer.cpp:764] noobj: 0.00162486 obj: 0.193509 iou: 0.683781 cat: 0.95098 recall: 0.959184 recall75: 0.240023 count: 7 I0819 11:42:49.340662 4075 yolov3_layer.cpp:764] noobj: 0.00192007 obj: 0.129091 iou: 0.496307 cat: 0.940556 recall: 0.585847 recall75: 0.0376984 count: 7 I0819 11:42:51.123641 4075 yolov3_layer.cpp:764] noobj: 0.00125715 obj: 0.101457 iou: 0.540699 cat: 0.935908 recall: 0.65 recall75: 0.285185 count: 3 I0819 11:42:51.128455 4075 yolov3_layer.cpp:764] noobj: 0.00165974 obj: 0.177178 iou: 0.659341 cat: 0.948165 recall: 0.960259 recall75: 0.240668 count: 9 I0819 11:42:51.132661 4075 yolov3_layer.cpp:764] noobj: 0.00161846 obj: 0.101618 iou: 0.489052 cat: 0.956137 recall: 0.378788 recall75: 0.0818182 count: 4 I0819 11:42:52.650727 4075 yolov3_layer.cpp:764] noobj: 0.000884098 obj: 0.150811 iou: 0.674956 cat: 0.954091 recall: 0.857143 recall75: 0.357143 count: 2 I0819 11:42:52.667630 4075 yolov3_layer.cpp:764] noobj: 0.0011528 obj: 0.17494 iou: 0.671705 cat: 0.943958 recall: 0.884632 recall75: 0.34094 count: 6 I0819 11:42:52.684650 4075 yolov3_layer.cpp:764] noobj: 0.000969724 obj: 0.0867856 iou: 0.462527 cat: 0.957531 recall: 0.284965 recall75: 0.00854701 count: 5 I0819 11:42:53.124300 4075 solver.cpp:253] Iteration 140 (2.47106 iter/s, 4.04685s/10 iters), loss = 17.522 I0819 11:42:53.124353 4075 solver.cpp:272] Train net output #0: det_loss1 = 0.0147064 ( 1 = 0.0147064 loss) I0819 11:42:53.124361 4075 solver.cpp:272] Train net output #1: det_loss2 = 10.534 ( 1 = 10.534 loss) I0819 11:42:53.124367 4075 solver.cpp:272] Train net output #2: det_loss3 = 0.0275316 ( 1 = 0.0275316 loss) I0819 11:42:53.124375 4075 sgd_solver.cpp:121] Iteration 140, lr = 0.005 I0819 11:42:54.234261 4075 yolov3_layer.cpp:764] noobj: 0.00146271 obj: 0.0784978 iou: 0.551061 cat: 0.927739 recall: 0.791667 recall75: 0.1875 count: 4 I0819 11:42:54.255779 4075 yolov3_layer.cpp:764] noobj: 0.00135748 obj: 0.173769 iou: 0.69321 cat: 0.945866 recall: 0.935897 recall75: 0.218942 count: 8 I0819 11:42:54.275810 4075 yolov3_layer.cpp:764] noobj: 0.000699549 obj: 0.0860297 iou: 0.494293 cat: 0.955075 recall: 0.67963 recall75: 0 count: 3 I0819 11:42:55.542800 4075 yolov3_layer.cpp:764] noobj: 0.00136111 obj: 0.0887574 iou: 0.488839 cat: 0.960572 recall: 0.392857 recall75: 0 count: 2 I0819 11:42:55.573961 4075 yolov3_layer.cpp:764] noobj: 0.00122951 obj: 0.1588 iou: 0.694562 cat: 0.946831 recall: 0.956377 recall75: 0.35974 count: 6 I0819 11:42:55.605846 4075 yolov3_layer.cpp:764] noobj: 0.00159448 obj: 0.104533 iou: 0.470692 cat: 0.951598 recall: 0.547112 recall75: 0.0153846 count: 8 I0819 11:42:56.426056 4075 solver.cpp:253] Iteration 150 (3.02878 iter/s, 3.30166s/10 iters), loss = 20.8859 I0819 11:42:56.426107 4075 solver.cpp:272] Train net output #0: det_loss1 = 0.0271772 ( 1 = 0.0271772 loss) I0819 11:42:56.426115 4075 solver.cpp:272] Train net output #1: det_loss2 = 7.51331 ( 1 = 7.51331 loss) I0819 11:42:56.426120 4075 solver.cpp:272] Train net output #2: det_loss3 = 2.69053 (* 1 = 2.69053 loss) I0819 11:42:56.426128 4075 sgd_solver.cpp:121] Iteration 150, lr = 0.005 I0819 11:42:56.814664 4075 yolov3_layer.cpp:764] noobj: 0.00125685 obj: 0.170043 iou: 0.564878 cat: 0.942557 recall: 0.511111 recall75: 0 count: 3 I0819 11:42:56.836457 4075 yolov3_layer.cpp:764] noobj: 0.00175241 obj: 0.24072 iou: 0.655551 cat: 0.944426 recall: 0.943826 recall75: 0.193758 count: 8 I0819 11:42:56.857069 4075 yolov3_layer.cpp:764] noobj: 0.00258412 obj: 0.174224 iou: 0.509138 cat: 0.917276 recall: 0.513287 recall75: 0.0454545 count: 4 Error in `./build/tools/caffe': double free or corruption (!prev): 0x00007f5e15685ae0 ======= Backtrace: ========= /lib/x86_64-linux-gnu/libc.so.6(+0x777e5)[0x7f607d2707e5] /lib/x86_64-linux-gnu/libc.so.6(+0x8037a)[0x7f607d27937a] /lib/x86_64-linux-gnu/libc.so.6(cfree+0x4c)[0x7f607d27d53c] /usr/local/lib/libopencv_core.so.3.2(_ZN2cv3Mat10deallocateEv+0xc9)[0x7f60701537d9] /home/sx/worksoft/MobileNet-YOLO/build/lib/libcaffe.so.1.0.0(_ZN2cv3MatD1Ev+0x19)[0x7f607e92b019] /home/sx/worksoft/MobileNet-YOLO/build/lib/libcaffe.so.1.0.0(_ZN5caffe15DataTransformerIfE9CropImageERKNS_5DatumERKNS14NormalizedBBoxEPS2+0x205)[0x7f607eb5ae55] /home/sx/worksoft/MobileNet-YOLO/build/lib/libcaffe.so.1.0.0(_ZN5caffe15DataTransformerIfE9CropImageERKNS_14AnnotatedDatumERKNS_14NormalizedBBoxEPS2_b+0x5d)[0x7f607eb5cbdd] /home/sx/worksoft/MobileNet-YOLO/build/lib/libcaffe.so.1.0.0(_ZN5caffe18AnnotatedDataLayerIfE10load_batchEPNS_5BatchIfEE+0x127c)[0x7f607ea6308c] /home/sx/worksoft/MobileNet-YOLO/build/lib/libcaffe.so.1.0.0(_ZN5caffe24BasePrefetchingDataLayerIfE19InternalThreadEntryEv+0xc6)[0x7f607ea21436] /home/sx/worksoft/MobileNet-YOLO/build/lib/libcaffe.so.1.0.0(_ZN5caffe14InternalThread5entryEiNS_5Caffe4BrewEiiib+0xb5)[0x7f607e8db985] /home/sx/worksoft/MobileNet-YOLO/build/lib/libcaffe.so.1.0.0(_ZN5boost6detail11thread_dataINS_3_bi6bind_tIvNS_4_mfi3mf6IvN5caffe14InternalThreadEiNS6_5Caffe4BrewEiiibEENS2_5list7INS2_5valueIPS7_EENSC_IiEENSC_IS9_EESF_SF_SF_NSC_IbEEEEEEE3runEv+0x5e)[0x7f607e8dca7e] /usr/lib/x86_64-linux-gnu/libboost_thread.so.1.58.0(+0x115d5)[0x7f607cfe45d5] /lib/x86_64-linux-gnu/libpthread.so.0(+0x76ba)[0x7f607cba56ba] /lib/x86_64-linux-gnu/libc.so.6(clone+0x6d)[0x7f607d30041d] ======= Memory map: ======== 00400000-00419000 r-xp 00000000 08:04 53741120 /home/sx/worksoft/MobileNet-YOLO/build/tools/caffe 00619000-0061a000 r--p 00019000 08:04 53741120 /home/sx/worksoft/MobileNet-YOLO/build/tools/caffe 0061a000-0061b000 rw-p 0001a000 08:04 53741120 /home/sx/worksoft/MobileNet-YOLO/build/tools/caffe 02471000-0f53b000 rw-p 00000000 00:00 0 [heap] 200000000-200200000 ---p 00000000 00:00 0 200200000-200400000 rw-s 00000000 00:06 510 /dev/nvidiactl 200400000-200600000 rw-s 00000000 00:06 511 /dev/nvidia0 200600000-203e00000 rw-s 00000000 00:06 510 /dev/nvidiactl 203e00000-204000000 rw-s 00000000 00:06 511 /dev/nvidia0 204000000-204a00000 ---p 00000000 00:00 0 204a00000-204c00000 rw-s 00000000 00:06 510 /dev/nvidiactl 204c00000-204e00000 rw-s 00000000 00:06 510 /dev/nvidiactl 204e00000-205000000 rw-s 204e00000 00:06 527 /dev/nvidia-uvm 205000000-205200000 rw-s 00000000 00:06 510 /dev/nvidiactl 205200000-205400000 ---p 00000000 00:00 0 205400000-205600000 rw-s 00000000 00:06 510 /dev/nvidiactl 205600000-300200000 ---p 00000000 00:00 0 400000000-400200000 ---p 00000000 00:00 0 400200000-400400000 rw-s 00000000 00:06 510 /dev/nvidiactl 400400000-400600000 rw-s 00000000 00:06 511 /dev/nvidia0 400600000-403e00000 rw-s 00000000 00:06 510 /dev/nvidiactl 403e00000-404000000 rw-s 00000000 00:06 511 /dev/nvidia0 404000000-404a00000 ---p 00000000 00:00 0 404a00000-404c00000 rw-s 00000000 00:06 510 /dev/nvidiactl 404c00000-404e00000 rw-s 00000000 00:06 510 /dev/nvidiactl 404e00000-405000000 rw-s 404e00000 00:06 527 /dev/nvidia-uvm 405000000-405200000 rw-s 00000000 00:06 510 /dev/nvidiactl 405200000-405400000 ---p 00000000 00:00 0 405400000-405600000 rw-s 00000000 00:06 510 /dev/nvidiactl 405600000-500200000 ---p 00000000 00:00 0 10000000000-10004200000 ---p 00000000 00:00 0 7f5cd4000000-7f5cd4021000 rw-p 00000000 00:00 0 7f5cd4021000-7f5cd8000000 ---p 00000000 00:00 0 7f5cda000000-7f5d3e000000 ---p 00000000 00:00 0 7f5d3e000000-7f5d3e200000 rw-s 00000000 00:05 134061 /dev/zero (deleted) 7f5d3e200000-7f5d3e400000 rw-s 00000000 00:05 134062 /dev/zero (deleted) 7f5d3e400000-7f5d3e600000 rw-s 00000000 00:05 134063 /dev/zero (deleted) 7f5d3e600000-7f5d3e800000 rw-s 00000000 00:05 134064 /dev/zero (deleted) 7f5d3e800000-7f5d40000000 ---p 00000000 00:00 0 7f5d40000000-7f5d403eb000 rw-p 00000000 00:00 0 7f5d403eb000-7f5d44000000 ---p 00000000 00:00 0 7f5d44000000-7f5dc4000000 r--s 00000000 08:04 55181316 /home/sx/worksoft/data/VOCdevkit/VOCUADETRAC/lmdb/VOCUADETRAC_test_lmdb/data.mdb 7f5dc4000000-7f5dc40f6000 rw-p 00000000 00:00 0 7f5dc40f6000-7f5dc8000000 ---p 00000000 00:00 0 7f5dc97ff000-7f5dc9800000 ---p 00000000 00:00 0 7f5dc9800000-7f5dca000000 rwxp 00000000 00:00 0 7f5dca000000-7f5dca200000 rw-s 00000000 00:05 134051 /dev/zero (deleted) 7f5dca200000-7f5dca400000 rw-s 00000000 00:05 134052 /dev/zero (deleted) 7f5dca400000-7f5dcac00000 ---p 00000000 00:00 0 7f5dcac00000-7f5dcae00000 rw-s 00000000 00:05 134053 /dev/zero (deleted) 7f5dcae00000-7f5dcb000000 rw-s 00000000 00:05 134054 /dev/zero (deleted) 7f5dcb000000-7f5dcb200000 rw-s 00000000 00:05 134055 /dev/zero (deleted) 7f5dcb200000-7f5dcb400000 rw-s 00000000 00:05 134056 /dev/zero (deleted) 7f5dcb400000-7f5dcb600000 rw-s 00000000 00:05 134057 /dev/zero (deleted) 7f5dcb600000-7f5dcba00000 rw-s 00000000 00:05 134058 /dev/zero (deleted) 7f5dcba00000-7f5dcbe00000 rw-s 00000000 00:05 134059 /dev/zero (deleted) 7f5dcbe00000-7f5dcc000000 rw-s 00000000 00:05 134060 /dev/zero (deleted) 7f5dcc000000-7f5dcc021000 rw-p 00000000 00:00 0 7f5dcc021000-7f5dd0000000 ---p 00000000 00:00 0 7f5dd0000000-7f5dd0021000 rw-p 00000000 00:00 0 7f5dd0021000-7f5dd4000000 ---p 00000000 00:00 0 7f5dd4000000-7f5dd4021000 rw-p 00000000 00:00 0 7f5dd4021000-7f5dd8000000 ---p 00000000 00:00 0 7f5dd8000000-7f5dd8021000 rw-p 00000000 00:00 0 7f5dd8021000-7f5ddc000000 ---p 00000000 00:00 0 7f5ddc000000-7f5ddc021000 rw-p 00000000 00:00 0 7f5ddc021000-7f5de0000000 ---p 00000000 00:00 0 7f5de0000000-7f5de0021000 rw-p 00000000 00:00 0 7f5de0021000-7f5de4000000 ---p 00000000 00:00 0 7f5de4000000-7f5de4021000 rw-p 00000000 00:00 0 7f5de4021000-7f5de8000000 ---p 00000000 00:00 0 7f5de8000000-7f5de8021000 rw-p 00000000 00:00 0 7f5de8021000-7f5dec000000 ---p 00000000 00:00 0 7f5dec000000-7f5e00000000 ---p 00000000 00:00 0 7f5e007f9000-7f5e007fa000 ---p 00000000 00:00 0 7f5e007fa000-7f5e00ffa000 rwxp 00000000 00:00 0 7f5e00ffa000-7f5e00ffb000 ---p 00000000 00:00 0 7f5e00ffb000-7f5e017fb000 rwxp 00000000 00:00 0 7f5e017fb000-7f5e017fc000 ---p 00000000 00:00 0 7f5e017fc000-7f5e01ffc000 rwxp 00000000 00:00 0 7f5e01ffc000-7f5e01ffd000 ---p 00000000 00:00 0 7f5e01ffd000-7f5e027fd000 rwxp 00000000 00:00 0 7f5e027fd000-7f5e027fe000 ---p 00000000 00:00 0 7f5e027fe000-7f5e02ffe000 rwxp 00000000 00:00 0 7f5e02ffe000-7f5e02fff000 ---p 00000000 00:00 0 7f5e02fff000-7f5e037ff000 rwxp 00000000 00:00 0 7f5e037ff000-7f5e03800000 ---p 00000000 00:00 0 7f5e03800000-7f5e04000000 rwxp 00000000 00:00 0 7f5e04000000-7f5e06600000 ---p 00000000 00:00 0 7f5e06600000-7f5e06800000 rw-s 00000000 00:05 136413 /dev/zero (deleted) 7f5e06800000-7f5e06a00000 rw-s 00000000 00:06 510 /dev/nvidiactl 7f5e06a00000-7f5e06c00000 rw-s 00000000 00:05 136416 /dev/zero (deleted) 7f5e06c00000-7f5e07200000 ---p 00000000 00:00 0 7f5e07200000-7f5e07400000 rw-s 00000000 00:06 510 /dev/nvidiactl 7f5e07400000-7f5e07800000 ---p 00000000 00:00 0 7f5e07800000-7f5e07a00000 rw-s 00000000 00:05 136418 /dev/zero (deleted) 7f5e07a00000-7f5e07ae7000 rw-s 00000000 00:06 510 /dev/nvidiactl 7f5e07ae7000-7f5e08000000 ---p 00000000 00:00 0 7f5e08000000-7f5e08021000 rw-p 00000000 00:00 0 7f5e08021000-7f5e0c000000 ---p 00000000 00:00 0 7f5e0c7fd000-7f5e0c7fe000 ---p 00000000 00:00 0 7f5e0c7fe000-7f5e0cffe000 rwxp 00000000 00:00 0 7f5e0cffe000-7f5e0cfff000 ---p 00000000 00:00 0 7f5e0cfff000-7f5e0d7ff000 rwxp 00000000 00:00 0 7f5e0d7ff000-7f5e0d800000 ---p 00000000 00:00 0 7f5e0d800000-7f5e0e000000 rwxp 00000000 00:00 0 7f5e0e000000-7f5e14000000 ---p 00000000 00:00 0 7f5e14000000-7f5e15f1c000 rw-p 00000000 00:00 0 7f5e15f1c000-7f5e18000000 ---p 00000000 00:00 0 7f5e1847d000-7f5e1847e000 ---p 00000000 00:00 0 7f5e1847e000-7f5e18c7e000 rwxp 00000000 00:00 0 7f5e18c7e000-7f5e18c7f000 ---p 00000000 00:00 0 7f5e18c7f000-7f5e1947f000 rwxp 00000000 00:00 0 7f5e1947f000-7f5e19480000 ---p 00000000 00:00 0 7f5e19480000-7f5e19c80000 rwxp 00000000 00:00 0 7f5e19c80000-7f5e1bc57000 r-xp 00000000 08:04 42469093 /usr/lib/x86_64-linux-gnu/libnvidia-opencl.so.450.57 7f5e1bc57000-7f5e1be57000 ---p 01fd7000 08:04 42469093 /usr/lib/x86_64-linux-gnu/libnvidia-opencl.so.450.57 7f5e1be57000-7f5e1bf74000 rw-p 01fd7000 08:04 42469093 /usr/lib/x86_64-linux-gnu/libnvidia-opencl.so.450.57 7f5e1bf74000-7f5e1c000000 rw-p 00000000 00:00 0 7f5e1c000000-7f5e1c600000 ---p 00000000 00:00 0 7f5e1c600000-7f5e1c800000 rw-s 00000000 00:05 136403 /dev/zero (deleted) 7f5e1c800000-7f5e1ca00000 rw-s 00000000 00:05 136404 /dev/zero (deleted) 7f5e1ca00000-7f5e1cc00000 rw-s 00000000 00:05 136407 /dev/zero (deleted) 7f5e1cc00000-7f5e1ce00000 rw-s 00000000 00:05 136408 /dev/zero (deleted) 7f5e1ce00000-7f5e1d000000 rw-s 00000000 00:05 136409 /dev/zero (deleted) 7f5e1d000000-7f5e1d200000 rw-s 00000000 00:05 136412 /dev/zero (deleted) 7f5e1d200000-7f5e1d600000 rw-s 00000000 00:05 135654 /dev/zero (deleted) 7f5e1d600000-7f5e1da00000 rw-s 00000000 00:05 135655 /dev/zero (deleted) 7f5e1da00000-7f5e1dc00000 rw-s 00000000 00:05 135656 /dev/zero (deleted) 7f5e1dc00000-7f5e1de00000 rw-s 00000000 00:05 135657 /dev/zero (deleted) 7f5e1de00000-7f5e1e000000 rw-s 00000000 00:05 134594 /dev/zero (deleted) 7f5e1e000000-7f601e000000 r--s 00000000 08:04 55181319 /home/sx/worksoft/data/VOCdevkit/VOCUADETRAC/lmdb/VOCUADETRAC_trainval_lmdb/data.mdb 7f601e000000-7f6022600000 ---p 00000000 00:00 0 7f6022600000-7f6022800000 rw-s 00000000 00:05 136402 /dev/zero (deleted) 7f6022800000-7f6025800000 ---p 00000000 00:00 0 7f6025800000-7f6025a00000 rw-s 00000000 00:05 134589 /dev/zero (deleted) 7f6025a00000-7f6025e3b000 rw-s 00000000 00:05 134590 /dev/zero (deleted) 7f6025e3b000-7f6026000000 ---p 00000000 00:00 0 7f6026000000-7f602643b000 rw-s 00000000 00:05 134591 /dev/zero (deleted) 7f602643b000-7f6026600000 ---p 00000000 00:00 0 7f6026600000-7f6026a3b000 rw-s 00000000 00:05 134592 /dev/zero (deleted) 7f6026a3b000-7f6026c00000 ---p 00000000 00:00 0 7f6026c00000-7f602703b000 rw-s 00000000 00:05 134593 /dev/zero (deleted) 7f602703b000-7f6027800000 ---p 00000000 00:00 0 7f6027800000-7f6027a00000 rw-s 00000000 00:05 134595 /dev/zero (deleted) 7f6027a00000-7f6028000000 ---p 00000000 00:00 0 7f6028000000-7f6028021000 rw-p 00000000 00:00 0 7f6028021000-7f602c000000 ---p 00000000 00:00 0 7f602c000000-7f602c120000 rw-p 00000000 00:00 0 7f602c120000-7f6030000000 ---p 00000000 00:00 0 7f6030000000-7f6040000000 ---p 00000000 00:00 0 7f60407fd000-7f60407fe000 ---p 00000000 00:00 0 7f60407fe000-7f6040ffe000 rwxp 00000000 00:00 0 7f6040ffe000-7f6040fff000 ---p 00000000 00:00 0 7f6040fff000-7f60417ff000 rwxp 00000000 00:00 0 7f60417ff000-7f6041800000 ---p 00000000 00:00 0 7f6041800000-7f6042000000 rwxp 00000000 00:00 0 7f6042000000-7f6043e00000 ---p 00000000 00:00 0 7f6043e00000-7f6044000000 rw-s 00000000 00:05 134596 /dev/zero (deleted) 7f60445fd000-7f60447fd000 rw-s 00000000 00:05 136417 /dev/zero (deleted) 7f60447fd000-7f60447fe000 ---p 00000000 00:00 0 7f60447fe000-7f6044ffe000 rwxp 00000000 00:00 0 7f6044ffe000-7f6044fff000 ---p 00000000 00:00 0 7f6044fff000-7f60457ff000 rwxp 00000000 00:00 0 7f60457ff000-7f6045800000 ---p 00000000 00:00 0 7f6045800000-7f6046000000 rwxp 00000000 00:00 0 7f6046000000-7f6046400000 ---p 00000000 00:00 0 7f6046400000-7f6046600000 rw-s 00000000 00:05 134043 /dev/zero (deleted) 7f6046600000-7f6046800000 rw-s 00000000 00:06 510 /dev/nvidiactl 7f6046800000-7f6046a00000 rw-s 00000000 00:05 134044 /dev/zero (deleted) 7f6046a00000-7f6047000000 ---p 00000000 00:00 0 7f6047000000-7f6047200000 rw-s 00000000 00:06 510 /dev/nvidiactl 7f6047200000-7f6047400000 rw-s 00000000 00:05 134046 /dev/zero (deleted) 7f6047400000-7f60474e7000 rw-s 00000000 00:06 510 /dev/nvidiactl 7f60474e7000-7f6047c00000 ---p 00000000 00:00 0 7f6047c00000-7f6047e00000 rw-s 00000000 00:05 134049 /dev/zero (deleted) 7f6047e00000-7f6048000000 rw-s 00000000 00:05 134050 /dev/zero (deleted) 7f604819b000-7f604819c000 rw-p 00000000 00:00 0 7f604819c000-7f604819e000 rw-s 00000000 08:04 55181315 /home/sx/worksoft/data/VOCdevkit/VOCUADETRAC/lmdb/VOCUADETRAC_test_lmdb/lock.mdb 7f604819e000-7f604819f000 rw-s 00000000 00:06 510 /dev/nvidiactl 7f604819f000-7f60481a0000 rw-s 00000000 00:06 510 /dev/nvidiactl 7f60481a0000-7f60481a1000 rw-s 00000000 00:06 510 /dev/nvidiactl 7f60481a1000-7f60481a2000 rw-s 00000000 00:06 510 /dev/nvidiactl 7f60481a2000-7f60481a3000 rw-s 00000000 00:06 510 /dev/nvidiactl 7f60481a3000-7f60481a4000 rw-s 00000000 00:06 510 /dev/nvidiactl 7f60481a4000-7f60481a5000 rw-s 00000000 00:06 510 /dev/nvidiactl 7f60481a5000-7f60481a6000 rw-s 00000000 00:06 510 /dev/nvidiactl 7f60481a6000-7f60481a7000 rw-s 00000000 00:06 510 /dev/nvidiactl 7f60481a7000-7f60481a8000 rw-s 00000000 00:06 510 /dev/nvidiactl 7f60481a8000-7f60481a9000 rw-s 00000000 00:06 510 /dev/nvidiactl 7f60481a9000-7f60481aa000 rw-s 00000000 00:06 510 /dev/nvidiactl 7f60481aa000-7f60481ab000 rw-s 00000000 00:06 510 /dev/nvidiactl 7f60481ab000-7f60481ac000 rw-s 00000000 00:06 510 /dev/nvidiactl 7f60481ac000-7f60481ad000 rw-s 00000000 00:06 510 /dev/nvidiactl 7f60481ad000-7f60481ae000 rw-s 00000000 00:06 510 /dev/nvidiactl 7f60481ae000-7f60481af000 rw-s 00000000 00:06 510 /dev/nvidiactl 7f60481af000-7f60481b0000 rw-s 00000000 00:06 510 /dev/nvidiactl 7f60481b0000-7f60481b1000 rw-s 00000000 00:06 510 /dev/nvidiactl 7f60481b1000-7f60481c1000 -w-s 00000000 00:06 511 /dev/nvidia0 7f60481c1000-7f60481c7000 r-xp 00000000 08:04 44185163 /usr/local/cuda-10.0/lib64/libOpenCL.so.1.1 7f60481c7000-7f60483c6000 ---p 00006000 08:04 44185163 /usr/local/cuda-10.0/lib64/libOpenCL.so.1.1 7f60483c6000-7f60483c8000 rw-p 00005000 08:04 44185163 /usr/local/cuda-10.0/lib64/libOpenCL.so.1.1 7f60483c8000-7f60484c1000 rw-p 00000000 00:00 0 7f60484c1000-7f60486c1000 rw-s 00000000 00:05 134045 /dev/zero (deleted) 7f60486c1000-7f6048742000 rw-p 00000000 00:00 0 7f6048742000-7f6048743000 ---p 00000000 00:00 0 7f6048743000-7f6048f43000 rwxp 00000000 00:00 0 7f6048f43000-7f6048f44000 ---p 00000000 00:00 0 7f6048f44000-7f6049744000 rwxp 00000000 00:00 0 7f6049744000-7f604f744000 ---p 00000000 00:00 0 7f604f744000-7f605084f000 r-xp 00000000 08:04 42469056 /usr/lib/x86_64-linux-gnu/libcuda.so.450.57 7f605084f000-7f6050a4f000 ---p 0110b000 08:04 42469056 /usr/lib/x86_64-linux-gnu/libcuda.so.450.57 7f6050a4f000-7f6050b6b000 rw-p 0110b000 08:04 42469056 /usr/lib/x86_64-linux-gnu/libcuda.so.450.57 7f6050b6b000-7f6050bf7000 rw-p 00000000 00:00 0 7f6050bf7000-7f6050bf8000 ---p 00000000 00:00 0 7f6050bf8000-7f60513f8000 rwxp 00000000 00:00 0 7f60513f8000-7f60573f8000 rw-p 00000000 00:00 0 7f60573f8000-7f60573f9000 ---p 00000000 00:00 0 7f60573f9000-7f6057bf9000 rwxp 00000000 00:00 0 7f6057bf9000-7f6057bfa000 ---p 00000000 00:00 0 7f6057bfa000-7f60583fa000 rwxp 00000000 00:00 0 7f60583fa000-7f60583fb000 ---p 00000000 00:00 0 7f60583fb000-7f6058bfb000 rwxp 00000000 00:00 0 7f6058bfb000-7f605a4b1000 r-xp 00000000 08:04 42469958 /usr/lib/x86_64-linux-gnu/libicudata.so.55.1 7f605a4b1000-7f605a6b0000 ---p 018b6000 08:04 42469958 /usr/lib/x86_64-linux-gnu/libicudata.so.55.1 7f605a6b0000-7f605a6b1000 r--p 018b5000 08:04 42469958 /usr/lib/x86_64-linux-gnu/libicudata.so.55.1 7f605a6b1000-7f605a6b2000 rw-p 018b6000 08:04 42469958 /usr/lib/x86_64-linux-gnu/libicudata.so.55.1 7f605a6b2000-7f605a6b5000 r-xp 00000000 08:04 7869044 /lib/x86_64-linux-gnu/libkeyutils.so.1.5 7f605a6b5000-7f605a8b4000 ---p 00003000 08:04 7869044 /lib/x86_64-linux-gnu/libkeyutils.so.1.5 7f605a8b4000-7f605a8b5000 r--p 00002000 08:04 7869044 /lib/x86_64-linux-gnu/libkeyutils.so.1.5 7f605a8b5000-7f605a8b6000 rw-p 00003000 08:04 7869044 /lib/x86_64-linux-gnu/libkeyutils.so.1.5 7f605a8b6000-7f605aa35000 r-xp 00000000 08:04 42469955 /usr/lib/x86_64-linux-gnu/libicuuc.so.55.1 7f605aa35000-7f605ac35000 ---p 0017f000 08:04 42469955 /usr/lib/x86_64-linux-gnu/libicuuc.so.55.1 7f605ac35000-7f605ac45000 r--p 0017f000 08:04 42469955 /usr/lib/x86_64-linux-gnu/libicuuc.so.55.1 7f605ac45000-7f605ac46000 rw-p 0018f000 08:04 42469955 /usr/lib/x86_64-linux-gnu/libicuuc.so.55.1 7f605ac46000-7f605ac4a000 rw-p 00000000 00:00 0 7f605ac4a000-7f605ac54000 r-xp 00000000 08:04 42475908 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 7f605ac54000-7f605ae53000 ---p 0000a000 08:04 42475908 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 7f605ae53000-7f605ae54000 r--p 00009000 08:04 42475908 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 7f605ae54000-7f605ae55000 rw-p 0000a000 08:04 42475908 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 7f605ae55000-7f605ae58000 r-xp 00000000 08:04 7868992 /lib/x86_64-linux-gnu/libcom_err.so.2.1 7f605ae58000-7f605b057000 ---p 00003000 08:04 7868992 /lib/x86_64-linux-gnu/libcom_err.so.2.1 7f605b057000-7f605b058000 r--p 00002000 08:04 7868992 /lib/x86_64-linux-gnu/libcom_err.so.2.1 7f605b058000-7f605b059000 rw-p 00003000 08:04 7868992 /lib/x86_64-linux-gnu/libcom_err.so.2.1 7f605b059000-7f605b085000 r-xp 00000000 08:04 42475898 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 7f605b085000-7f605b284000 ---p 0002c000 08:04 42475898 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 7f605b284000-7f605b286000 r--p 0002b000 08:04 42475898 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 7f605b286000-7f605b287000 rw-p 0002d000 08:04 42475898 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 7f605b287000-7f605b288000 rw-p 00000000 00:00 0 7f605b288000-7f605b34b000 r-xp 00000000 08:04 42475906 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 7f605b34b000-7f605b54b000 ---p 000c3000 08:04 42475906 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 7f605b54b000-7f605b558000 r--p 000c3000 08:04 42475906 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 7f605b558000-7f605b55a000 rw-p 000d0000 08:04 42475906 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 7f605b55a000-7f605b56c000 r-xp 00000000 08:04 7869027 /lib/x86_64-linux-gnu/libgpg-error.so.0.17.0 7f605b56c000-7f605b76c000 ---p 00012000 08:04 7869027 /lib/x86_64-linux-gnu/libgpg-error.so.0.17.0 7f605b76c000-7f605b76d000 r--p 00012000 08:04 7869027 /lib/x86_64-linux-gnu/libgpg-error.so.0.17.0 7f605b76d000-7f605b76e000 rw-p 00013000 08:04 7869027 /lib/x86_64-linux-gnu/libgpg-error.so.0.17.0 7f605b76e000-7f605b78f000 r-xp 00000000 08:04 42474301 /usr/lib/x86_64-linux-gnu/libgomp.so.1.0.0 7f605b78f000-7f605b98e000 ---p 00021000 08:04 42474301 /usr/lib/x86_64-linux-gnu/libgomp.so.1.0.0 7f605b98e000-7f605b98f000 r--p 00020000 08:04 42474301 /usr/lib/x86_64-linux-gnu/libgomp.so.1.0.0 7f605b98f000-7f605b990000 rw-p 00021000 08:04 42474301 /usr/lib/x86_64-linux-gnu/libgomp.so.1.0.0 7f605b990000-7f605b996000 r-xp 00000000 08:04 42475403 /usr/lib/x86_64-linux-gnu/libdatrie.so.1.3.3 7f605b996000-7f605bb96000 ---p 00006000 08:04 42475403 /usr/lib/x86_64-linux-gnu/libdatrie.so.1.3.3 7f605bb96000-7f605bb97000 r--p 00006000 08:04 42475403 /usr/lib/x86_64-linux-gnu/libdatrie.so.1.3.3 7f605bb97000-7f605bb98000 rw-p 00007000 08:04 42475403 /usr/lib/x86_64-linux-gnu/libdatrie.so.1.3.3 7f605bb98000-7f605bbbc000 r-xp 00000000 08:04 42475707 /usr/lib/x86_64-linux-gnu/libgraphite2.so.3.0.1 7f605bbbc000-7f605bdbb000 ---p 00024000 08:04 42475707 /usr/lib/x86_64-linux-gnu/libgraphite2.so.3.0.1 7f605bdbb000-7f605bdbd000 r--p 00023000 08:04 42475707 /usr/lib/x86_64-linux-gnu/libgraphite2.so.3.0.1 7f605bdbd000-7f605bdbe000 rw-p 00025000 08:04 42475707 /usr/lib/x86_64-linux-gnu/libgraphite2.so.3.0.1 7f605bdbe000-7f605bdcf000 r-xp 00000000 08:04 42476324 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.5.1 7f605bdcf000-7f605bfcf000 ---p 00011000 08:04 42476324 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.5.1 7f605bfcf000-7f605bfd0000 r--p 00011000 08:04 42476324 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.5.1 7f605bfd0000-7f605bfd1000 rw-p 00012000 08:04 42476324 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.5.1 7f605bfd1000-7f605c002000 r-xp 00000000 08:04 42475852 /usr/lib/x86_64-linux-gnu/libidn.so.11.6.15 7f605c002000-7f605c202000 ---p 00031000 08:04 42475852 /usr/lib/x86_64-linux-gnu/libidn.so.11.6.15 7f605c202000-7f605c203000 r--p 00031000 08:04 42475852 /usr/lib/x86_64-linux-gnu/libidn.so.11.6.15 7f605c203000-7f605c204000 rw-p 00032000 08:04 42475852 /usr/lib/x86_64-linux-gnu/libidn.so.11.6.15 7f605c204000-7f605c25d000 r-xp 00000000 08:04 42476091 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.1.0 7f605c25d000-7f605c45c000 ---p 00059000 08:04 42476091 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.1.0 7f605c45c000-7f605c466000 r--p 00058000 08:04 42476091 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.1.0 7f605c466000-7f605c468000 rw-p 00062000 08:04 42476091 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.1.0 7f605c468000-7f605c619000 r-xp 00000000 08:04 42476576 /usr/lib/x86_64-linux-gnu/libxml2.so.2.9.3 7f605c619000-7f605c818000 ---p 001b1000 08:04 42476576 /usr/lib/x86_64-linux-gnu/libxml2.so.2.9.3 7f605c818000-7f605c820000 r--p 001b0000 08:04 42476576 /usr/lib/x86_64-linux-gnu/libxml2.so.2.9.3 7f605c820000-7f605c822000 rw-p 001b8000 08:04 42476576 /usr/lib/x86_64-linux-gnu/libxml2.so.2.9.3 7f605c822000-7f605c823000 rw-p 00000000 00:00 0 7f605c823000-7f605c8a2000 r-xp 00000000 08:04 42475665 /usr/lib/x86_64-linux-gnu/libgmp.so.10.3.0 7f605c8a2000-7f605caa1000 ---p 0007f000 08:04 42475665 /usr/lib/x86_64-linux-gnu/libgmp.so.10.3.0 7f605caa1000-7f605caa2000 r--p 0007e000 08:04 42475665 /usr/lib/x86_64-linux-gnu/libgmp.so.10.3.0 7f605caa2000-7f605caa3000 rw-p 0007f000 08:04 42475665 /usr/lib/x86_64-linux-gnu/libgmp.so.10.3.0 7f605caa3000-7f605cad7000 r-xp 00000000 08:04 42476028 /usr/lib/x86_64-linux-gnu/libnettle.so.6.2 7f605cad7000-7f605ccd6000 ---p 00034000 08:04 42476028 /usr/lib/x86_64-linux-gnu/libnettle.so.6.2 7f605ccd6000-7f605ccd8000 r--p 00033000 08:04 42476028 /usr/lib/x86_64-linux-gnu/libnettle.so.6.2 7f605ccd8000-7f605ccd9000 rw-p 00035000 08:04 42476028 /usr/lib/x86_64-linux-gnu/libnettle.so.6.2 7f605ccd9000-7f605cd0b000 r-xp 00000000 08:04 42475803 /usr/lib/x86_64-linux-gnu/libhogweed.so.4.2 7f605cd0b000-7f605cf0a000 ---p 00032000 08:04 42475803 /usr/lib/x86_64-linux-gnu/libhogweed.so.4.2 7f605cf0a000-7f605cf0b000 r--p 00031000 08:04 42475803 /usr/lib/x86_64-linux-gnu/libhogweed.so.4.2 7f605cf0b000-7f605cf0c000 rw-p 00032000 08:04 42475803 /usr/lib/x86_64-linux-gnu/libhogweed.so.4.2 7f605cf0c000-7f605cf53000 r-xp 00000000 08:04 42475722 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 7f605cf53000-7f605d152000 ---p 00047000 08:04 42475722 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 7f605d152000-7f605d154000 r--p 00046000 08:04 42475722 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 7f605d154000-7f605d156000 rw-p 00048000 08:04 42475722 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 7f605d156000-7f605d22d000 r-xp 00000000 08:04 7869023 /lib/x86_64-linux-gnu/libgcrypt.so.20.0.5 7f605d22d000-7f605d42d000 ---p 000d7000 08:04 7869023 /lib/x86_64-linux-gnu/libgcrypt.so.20.0.5 7f605d42d000-7f605d42e000 r--p 000d7000 08:04 7869023 /lib/x86_64-linux-gnu/libgcrypt.so.20.0.5 7f605d42e000-7f605d436000 rw-p 000d8000 08:04 7869023 /lib/x86_64-linux-gnu/libgcrypt.so.20.0.5 7f605d436000-7f605d437000 rw-p 00000000 00:00 0 7f605d437000-7f605d4b2000 r-xp 00000000 08:04 42476081 /usr/lib/x86_64-linux-gnu/liborc-0.4.so.0.25.0 7f605d4b2000-7f605d6b1000 ---p 0007b000 08:04 42476081 /usr/lib/x86_64-linux-gnu/liborc-0.4.so.0.25.0 7f605d6b1000-7f605d6b3000 r--p 0007a000 08:04 42476081 /usr/lib/x86_64-linux-gnu/liborc-0.4.so.0.25.0 7f605d6b3000-7f605d6b7000 rw-p 0007c000 08:04 42476081 /usr/lib/x86_64-linux-gnu/liborc-0.4.so.0.25.0 7f605d6b7000-7f605d6be000 r-xp 00000000 08:04 42476075 /usr/lib/x86_64-linux-gnu/libogg.so.0.8.2 7f605d6be000-7f605d8be000 ---p 00007000 08:04 42476075 /usr/lib/x86_64-linux-gnu/libogg.so.0.8.2 7f605d8be000-7f605d8bf000 r--p 00007000 08:04 42476075 /usr/lib/x86_64-linux-gnu/libogg.so.0.8.2 7f605d8bf000-7f605d8c0000 rw-p 00008000 08:04 42476075 /usr/lib/x86_64-linux-gnu/libogg.so.0.8.2 7f605d8c0000-7f605d8ca000 r-xp 00000000 08:04 42476063 /usr/lib/x86_64-linux-gnu/libnuma.so.1.0.0 7f605d8ca000-7f605dac9000 ---p 0000a000 08:04 42476063 /usr/lib/x86_64-linux-gnu/libnuma.so.1.0.0 7f605dac9000-7f605daca000 r--p 00009000 08:04 42476063 /usr/lib/x86_64-linux-gnu/libnuma.so.1.0.0 7f605daca000-7f605dacb000 rw-p 0000a000 08:04 42476063 /usr/lib/x86_64-linux-gnu/libnuma.so.1.0.0 7f605dacb000-7f605daf9000 r-xp 00000000 08:04 42476953 /usr/lib/x86_64-linux-gnu/libsoxr.so.0.1.1 7f605daf9000-7f605dcf8000 ---p 0002e000 08:04 42476953 /usr/lib/x86_64-linux-gnu/libsoxr.so.0.1.1 7f605dcf8000-7f605dcfa000 r--p 0002d000 08:04 42476953 /usr/lib/x86_64-linux-gnu/libsoxr.so.0.1.1 7f605dcfa000-7f605dcfb000 rw-p 0002f000 08:04 42476953 /usr/lib/x86_64-linux-gnu/libsoxr.so.0.1.1 7f605dcfb000-7f605dd30000 rw-p 00000000 00:00 0 7f605dd30000-7f605dd35000 r-xp 00000000 08:04 42475153 /usr/lib/x86_64-linux-gnu/libXdmcp.so.6.0.0 7f605dd35000-7f605df34000 ---p 00005000 08:04 42475153 /usr/lib/x86_64-linux-gnu/libXdmcp.so.6.0.0 7f605df34000-7f605df35000 r--p 00004000 08:04 42475153 /usr/lib/x86_64-linux-gnu/libXdmcp.so.6.0.0 7f605df35000-7f605df36000 rw-p 00005000 08:04 42475153 /usr/lib/x86_64-linux-gnu/libXdmcp.so.6.0.0 7f605df36000-7f605df38000 r-xp 00000000 08:04 42475142 /usr/lib/x86_64-linux-gnu/libXau.so.6.0.0 7f605df38000-7f605e138000 ---p 00002000 08:04 42475142 /usr/lib/x86_64-linux-gnu/libXau.so.6.0.0 7f605e138000-7f605e139000 r--p 00002000 08:04 42475142 /usr/lib/x86_64-linux-gnu/libXau.so.6.0.0 7f605e139000-7f605e13a000 rw-p 00003000 08:04 42475142 /usr/lib/x86_64-linux-gnu/libXau.so.6.0.0 7f605e13a000-7f605e160000 r-xp 00000000 08:04 7864415 /lib/x86_64-linux-gnu/libexpat.so.1.6.0 7f605e160000-7f605e360000 ---p 00026000 08:04 7864415 /lib/x86_64-linux-gnu/libexpat.so.1.6.0 7f605e360000-7f605e362000 r--p 00026000 08:04 7864415 /lib/x86_64-linux-gnu/libexpat.so.1.6.0 7f605e362000-7f605e363000 rw-p 00028000 08:04 7864415 /lib/x86_64-linux-gnu/libexpat.so.1.6.0 7f605e363000-7f605e36b000 r-xp 00000000 08:04 42476336 /usr/lib/x86_64-linux-gnu/libthai.so.0.2.4 7f605e36b000-7f605e56a000 ---p 00008000 08:04 42476336 /usr/lib/x86_64-linux-gnu/libthai.so.0.2.4 7f605e56a000-7f605e56b000 r--p 00007000 08:04 42476336 /usr/lib/x86_64-linux-gnu/libthai.so.0.2.4 7f605e56b000-7f605e56c000 rw-p 00008000 08:04 42476336 /usr/lib/x86_64-linux-gnu/libthai.so.0.2.4 7f605e56c000-7f605e5c8000 r-xp 00000000 08:04 42475795 /usr/lib/x86_64-linux-gnu/libharfbuzz.so.0.10000.1 7f605e5c8000-7f605e7c8000 ---p 0005c000 08:04 42475795 /usr/lib/x86_64-linux-gnu/libharfbuzz.so.0.10000.1 Aborted at 1597808577 (unix time) try "date -d @1597808577" if you are using GNU date PC: @ 0x7f607d22e428 gsignal SIGABRT (@0x3e800000feb) received by PID 4075 (TID 0x7f6044ffd700) from PID 4075; stack trace: @ 0x7f607d22e4b0 (unknown) @ 0x7f607d22e428 gsignal @ 0x7f607d23002a abort @ 0x7f607d2707ea (unknown) @ 0x7f607d27937a (unknown) @ 0x7f607d27d53c cfree @ 0x7f60701537d9 cv::Mat::deallocate() @ 0x7f607e92b019 cv::Mat::~Mat() @ 0x7f607eb5ae55 caffe::DataTransformer<>::CropImage() @ 0x7f607eb5cbdd caffe::DataTransformer<>::CropImage() @ 0x7f607ea6308c caffe::AnnotatedDataLayer<>::load_batch() @ 0x7f607ea21436 caffe::BasePrefetchingDataLayer<>::InternalThreadEntry() @ 0x7f607e8db985 caffe::InternalThread::entry() @ 0x7f607e8dca7e boost::detail::thread_data<>::run() @ 0x7f607cfe45d5 (unknown) @ 0x7f607cba56ba start_thread @ 0x7f607d30041d clone @ 0x0 (unknown)

GeneralSong commented 3 years ago

请问各位大牛有遇到过吗?谢谢

GeneralSong commented 3 years ago

who can help me ? thanks