yun-liu / RCF

Richer Convolutional Features for Edge Detection
Other
753 stars 259 forks source link

loss question #55

Open WWWJL opened 5 years ago

WWWJL commented 5 years ago

hi,I downloaded your model ,and ran the train.sh file but, The loss value of the training is very unstable and the value varies widely all the time. there is no convergence,Why is that?

yun-liu commented 5 years ago

Would you mind attaching your loss outputs?

WWWJL commented 5 years ago

the part of loss output is : Iteration 15200- 17860: I1014 05:34:10.086400 20911 sgd_solver.cpp:106] Iteration 15200, lr = 1e-07 I1014 05:34:33.629215 20911 solver.cpp:228] Iteration 15220, loss = 9940.07 I1014 05:34:33.629246 20911 solver.cpp:244] Train net output #0: dsn1_loss = 8734.66 ( 1 = 8734.66 loss) I1014 05:34:33.629253 20911 solver.cpp:244] Train net output #1: dsn2_loss = 6412.64 ( 1 = 6412.64 loss) I1014 05:34:33.629259 20911 solver.cpp:244] Train net output #2: dsn3_loss = 4789.82 ( 1 = 4789.82 loss) I1014 05:34:33.629263 20911 solver.cpp:244] Train net output #3: dsn4_loss = 4491.68 ( 1 = 4491.68 loss) I1014 05:34:33.629267 20911 solver.cpp:244] Train net output #4: dsn5_loss = 4334.2 ( 1 = 4334.2 loss) I1014 05:34:33.629271 20911 solver.cpp:244] Train net output #5: fuse_loss = 4108.05 ( 1 = 4108.05 loss) I1014 05:34:33.629276 20911 sgd_solver.cpp:106] Iteration 15220, lr = 1e-07 I1014 05:34:59.515687 20911 solver.cpp:228] Iteration 15240, loss = 10129.9 I1014 05:34:59.515718 20911 solver.cpp:244] Train net output #0: dsn1_loss = 598.434 ( 1 = 598.434 loss) I1014 05:34:59.515725 20911 solver.cpp:244] Train net output #1: dsn2_loss = 480.888 ( 1 = 480.888 loss) I1014 05:34:59.515731 20911 solver.cpp:244] Train net output #2: dsn3_loss = 293.844 ( 1 = 293.844 loss) I1014 05:34:59.515735 20911 solver.cpp:244] Train net output #3: dsn4_loss = 255.153 ( 1 = 255.153 loss) I1014 05:34:59.515739 20911 solver.cpp:244] Train net output #4: dsn5_loss = 224.06 ( 1 = 224.06 loss) I1014 05:34:59.515744 20911 solver.cpp:244] Train net output #5: fuse_loss = 242.571 ( 1 = 242.571 loss) I1014 05:34:59.515748 20911 sgd_solver.cpp:106] Iteration 15240, lr = 1e-07 I1014 05:35:23.255265 20911 solver.cpp:228] Iteration 15260, loss = 10179.4 I1014 05:35:23.255298 20911 solver.cpp:244] Train net output #0: dsn1_loss = 421.179 ( 1 = 421.179 loss) I1014 05:35:23.255306 20911 solver.cpp:244] Train net output #1: dsn2_loss = 348.122 ( 1 = 348.122 loss) I1014 05:35:23.255311 20911 solver.cpp:244] Train net output #2: dsn3_loss = 301.959 ( 1 = 301.959 loss) I1014 05:35:23.255316 20911 solver.cpp:244] Train net output #3: dsn4_loss = 271.149 ( 1 = 271.149 loss) I1014 05:35:23.255321 20911 solver.cpp:244] Train net output #4: dsn5_loss = 260.382 ( 1 = 260.382 loss) I1014 05:35:23.255323 20911 solver.cpp:244] Train net output #5: fuse_loss = 239.731 ( 1 = 239.731 loss) I1014 05:35:23.255326 20911 sgd_solver.cpp:106] Iteration 15260, lr = 1e-07 I1014 05:35:48.329201 20911 solver.cpp:228] Iteration 15280, loss = 9945.28 I1014 05:35:48.329231 20911 solver.cpp:244] Train net output #0: dsn1_loss = 298.163 ( 1 = 298.163 loss) I1014 05:35:48.329239 20911 solver.cpp:244] Train net output #1: dsn2_loss = 207.912 ( 1 = 207.912 loss) I1014 05:35:48.329246 20911 solver.cpp:244] Train net output #2: dsn3_loss = 189.546 ( 1 = 189.546 loss) I1014 05:35:48.329248 20911 solver.cpp:244] Train net output #3: dsn4_loss = 214.296 ( 1 = 214.296 loss) I1014 05:35:48.329253 20911 solver.cpp:244] Train net output #4: dsn5_loss = 196.24 ( 1 = 196.24 loss) I1014 05:35:48.329257 20911 solver.cpp:244] Train net output #5: fuse_loss = 132.356 ( 1 = 132.356 loss) I1014 05:35:48.329260 20911 sgd_solver.cpp:106] Iteration 15280, lr = 1e-07 I1014 05:36:14.140686 20911 solver.cpp:228] Iteration 15300, loss = 10197.7 I1014 05:36:14.140727 20911 solver.cpp:244] Train net output #0: dsn1_loss = 1683.62 ( 1 = 1683.62 loss) I1014 05:36:14.140734 20911 solver.cpp:244] Train net output #1: dsn2_loss = 1087.07 ( 1 = 1087.07 loss) I1014 05:36:14.140739 20911 solver.cpp:244] Train net output #2: dsn3_loss = 927.866 ( 1 = 927.866 loss) I1014 05:36:14.140745 20911 solver.cpp:244] Train net output #3: dsn4_loss = 1113.2 ( 1 = 1113.2 loss) I1014 05:36:14.140750 20911 solver.cpp:244] Train net output #4: dsn5_loss = 1094.26 ( 1 = 1094.26 loss) I1014 05:36:14.140755 20911 solver.cpp:244] Train net output #5: fuse_loss = 843.991 ( 1 = 843.991 loss) I1014 05:36:14.140759 20911 sgd_solver.cpp:106] Iteration 15300, lr = 1e-07 I1014 05:36:39.684062 20911 solver.cpp:228] Iteration 15320, loss = 10084.7 I1014 05:36:39.684094 20911 solver.cpp:244] Train net output #0: dsn1_loss = 5232.36 ( 1 = 5232.36 loss) I1014 05:36:39.684103 20911 solver.cpp:244] Train net output #1: dsn2_loss = 4505.66 ( 1 = 4505.66 loss) I1014 05:36:39.684108 20911 solver.cpp:244] Train net output #2: dsn3_loss = 4045.51 ( 1 = 4045.51 loss) I1014 05:36:39.684113 20911 solver.cpp:244] Train net output #3: dsn4_loss = 4489.57 ( 1 = 4489.57 loss) I1014 05:36:39.684116 20911 solver.cpp:244] Train net output #4: dsn5_loss = 4125.91 ( 1 = 4125.91 loss) I1014 05:36:39.684120 20911 solver.cpp:244] Train net output #5: fuse_loss = 3585.47 ( 1 = 3585.47 loss) I1014 05:36:39.684124 20911 sgd_solver.cpp:106] Iteration 15320, lr = 1e-07 I1014 05:37:04.691246 20911 solver.cpp:228] Iteration 15340, loss = 10748.3 I1014 05:37:04.691280 20911 solver.cpp:244] Train net output #0: dsn1_loss = 10705.2 ( 1 = 10705.2 loss) I1014 05:37:04.691287 20911 solver.cpp:244] Train net output #1: dsn2_loss = 9192.07 ( 1 = 9192.07 loss) I1014 05:37:04.691293 20911 solver.cpp:244] Train net output #2: dsn3_loss = 8995.01 ( 1 = 8995.01 loss) I1014 05:37:04.691298 20911 solver.cpp:244] Train net output #3: dsn4_loss = 9126.67 ( 1 = 9126.67 loss) I1014 05:37:04.691303 20911 solver.cpp:244] Train net output #4: dsn5_loss = 8623.87 ( 1 = 8623.87 loss) I1014 05:37:04.691306 20911 solver.cpp:244] Train net output #5: fuse_loss = 8258.05 ( 1 = 8258.05 loss) I1014 05:37:04.691310 20911 sgd_solver.cpp:106] Iteration 15340, lr = 1e-07 I1014 05:37:29.303936 20911 solver.cpp:228] Iteration 15360, loss = 11121.9 I1014 05:37:29.303962 20911 solver.cpp:244] Train net output #0: dsn1_loss = 7.16362 ( 1 = 7.16362 loss) I1014 05:37:29.303967 20911 solver.cpp:244] Train net output #1: dsn2_loss = 5.91694 ( 1 = 5.91694 loss) I1014 05:37:29.303972 20911 solver.cpp:244] Train net output #2: dsn3_loss = 6.03596 ( 1 = 6.03596 loss) I1014 05:37:29.303977 20911 solver.cpp:244] Train net output #3: dsn4_loss = 4.13075 ( 1 = 4.13075 loss) I1014 05:37:29.303982 20911 solver.cpp:244] Train net output #4: dsn5_loss = 8.72642 ( 1 = 8.72642 loss) I1014 05:37:29.303984 20911 solver.cpp:244] Train net output #5: fuse_loss = 5.23251 ( 1 = 5.23251 loss) I1014 05:37:29.303987 20911 sgd_solver.cpp:106] Iteration 15360, lr = 1e-07 I1014 05:37:53.880198 20911 solver.cpp:228] Iteration 15380, loss = 10975.4 I1014 05:37:53.880225 20911 solver.cpp:244] Train net output #0: dsn1_loss = 5.1412 ( 1 = 5.1412 loss) I1014 05:37:53.880233 20911 solver.cpp:244] Train net output #1: dsn2_loss = 3.21328 ( 1 = 3.21328 loss) I1014 05:37:53.880237 20911 solver.cpp:244] Train net output #2: dsn3_loss = 2.76107 ( 1 = 2.76107 loss) I1014 05:37:53.880242 20911 solver.cpp:244] Train net output #3: dsn4_loss = 2.731 ( 1 = 2.731 loss) I1014 05:37:53.880246 20911 solver.cpp:244] Train net output #4: dsn5_loss = 3.80422 ( 1 = 3.80422 loss) I1014 05:37:53.880249 20911 solver.cpp:244] Train net output #5: fuse_loss = 2.70468 ( 1 = 2.70468 loss) I1014 05:37:53.880254 20911 sgd_solver.cpp:106] Iteration 15380, lr = 1e-07 I1014 05:38:20.636854 20911 solver.cpp:228] Iteration 15400, loss = 10879.3 I1014 05:38:20.636885 20911 solver.cpp:244] Train net output #0: dsn1_loss = 8120.87 ( 1 = 8120.87 loss) I1014 05:38:20.636893 20911 solver.cpp:244] Train net output #1: dsn2_loss = 7109.86 ( 1 = 7109.86 loss) I1014 05:38:20.636898 20911 solver.cpp:244] Train net output #2: dsn3_loss = 5977.25 ( 1 = 5977.25 loss) I1014 05:38:20.636904 20911 solver.cpp:244] Train net output #3: dsn4_loss = 5534.32 ( 1 = 5534.32 loss) I1014 05:38:20.636906 20911 solver.cpp:244] Train net output #4: dsn5_loss = 4765.9 ( 1 = 4765.9 loss) I1014 05:38:20.636910 20911 solver.cpp:244] Train net output #5: fuse_loss = 5015.57 ( 1 = 5015.57 loss) I1014 05:38:20.636914 20911 sgd_solver.cpp:106] Iteration 15400, lr = 1e-07 I1014 05:38:46.348800 20911 solver.cpp:228] Iteration 15420, loss = 10406.2 I1014 05:38:46.348826 20911 solver.cpp:244] Train net output #0: dsn1_loss = 21.3204 ( 1 = 21.3204 loss) I1014 05:38:46.348834 20911 solver.cpp:244] Train net output #1: dsn2_loss = 16.1099 ( 1 = 16.1099 loss) I1014 05:38:46.348839 20911 solver.cpp:244] Train net output #2: dsn3_loss = 15.5387 ( 1 = 15.5387 loss) I1014 05:38:46.348845 20911 solver.cpp:244] Train net output #3: dsn4_loss = 21.5612 ( 1 = 21.5612 loss) I1014 05:38:46.348848 20911 solver.cpp:244] Train net output #4: dsn5_loss = 16.423 ( 1 = 16.423 loss) I1014 05:38:46.348851 20911 solver.cpp:244] Train net output #5: fuse_loss = 12.9524 ( 1 = 12.9524 loss) I1014 05:38:46.348855 20911 sgd_solver.cpp:106] Iteration 15420, lr = 1e-07 I1014 05:39:10.124040 20911 solver.cpp:228] Iteration 15440, loss = 9603.42 I1014 05:39:10.124069 20911 solver.cpp:244] Train net output #0: dsn1_loss = 291.798 ( 1 = 291.798 loss) I1014 05:39:10.124076 20911 solver.cpp:244] Train net output #1: dsn2_loss = 269.89 ( 1 = 269.89 loss) I1014 05:39:10.124081 20911 solver.cpp:244] Train net output #2: dsn3_loss = 227.379 ( 1 = 227.379 loss) I1014 05:39:10.124088 20911 solver.cpp:244] Train net output #3: dsn4_loss = 155.784 ( 1 = 155.784 loss) I1014 05:39:10.124091 20911 solver.cpp:244] Train net output #4: dsn5_loss = 109.402 ( 1 = 109.402 loss) I1014 05:39:10.124095 20911 solver.cpp:244] Train net output #5: fuse_loss = 136.139 ( 1 = 136.139 loss) I1014 05:39:10.124099 20911 sgd_solver.cpp:106] Iteration 15440, lr = 1e-07 I1014 05:39:35.299187 20911 solver.cpp:228] Iteration 15460, loss = 9682.55 I1014 05:39:35.299214 20911 solver.cpp:244] Train net output #0: dsn1_loss = 3118.98 ( 1 = 3118.98 loss) I1014 05:39:35.299221 20911 solver.cpp:244] Train net output #1: dsn2_loss = 1700.46 ( 1 = 1700.46 loss) I1014 05:39:35.299227 20911 solver.cpp:244] Train net output #2: dsn3_loss = 1381.55 ( 1 = 1381.55 loss) I1014 05:39:35.299232 20911 solver.cpp:244] Train net output #3: dsn4_loss = 1271.95 ( 1 = 1271.95 loss) I1014 05:39:35.299235 20911 solver.cpp:244] Train net output #4: dsn5_loss = 1358.14 ( 1 = 1358.14 loss) I1014 05:39:35.299238 20911 solver.cpp:244] Train net output #5: fuse_loss = 1247.58 ( 1 = 1247.58 loss) I1014 05:39:35.299242 20911 sgd_solver.cpp:106] Iteration 15460, lr = 1e-07 I1014 05:40:01.743142 20911 solver.cpp:228] Iteration 15480, loss = 9863.69 I1014 05:40:01.743170 20911 solver.cpp:244] Train net output #0: dsn1_loss = 944.945 ( 1 = 944.945 loss) I1014 05:40:01.743176 20911 solver.cpp:244] Train net output #1: dsn2_loss = 810.689 ( 1 = 810.689 loss) I1014 05:40:01.743182 20911 solver.cpp:244] Train net output #2: dsn3_loss = 599.995 ( 1 = 599.995 loss) I1014 05:40:01.743186 20911 solver.cpp:244] Train net output #3: dsn4_loss = 482.045 ( 1 = 482.045 loss) I1014 05:40:01.743191 20911 solver.cpp:244] Train net output #4: dsn5_loss = 394.902 ( 1 = 394.902 loss) I1014 05:40:01.743194 20911 solver.cpp:244] Train net output #5: fuse_loss = 477.084 ( 1 = 477.084 loss) I1014 05:40:01.743198 20911 sgd_solver.cpp:106] Iteration 15480, lr = 1e-07 I1014 05:40:27.964581 20911 solver.cpp:228] Iteration 15500, loss = 10849.1 I1014 05:40:27.964607 20911 solver.cpp:244] Train net output #0: dsn1_loss = 7043.99 ( 1 = 7043.99 loss) I1014 05:40:27.964614 20911 solver.cpp:244] Train net output #1: dsn2_loss = 5246.11 ( 1 = 5246.11 loss) I1014 05:40:27.964620 20911 solver.cpp:244] Train net output #2: dsn3_loss = 4048.04 ( 1 = 4048.04 loss) I1014 05:40:27.964624 20911 solver.cpp:244] Train net output #3: dsn4_loss = 3597.6 ( 1 = 3597.6 loss) I1014 05:40:27.964629 20911 solver.cpp:244] Train net output #4: dsn5_loss = 3018.8 ( 1 = 3018.8 loss) I1014 05:40:27.964632 20911 solver.cpp:244] Train net output #5: fuse_loss = 3211.63 ( 1 = 3211.63 loss) I1014 05:40:27.964637 20911 sgd_solver.cpp:106] Iteration 15500, lr = 1e-07 I1014 05:40:52.034391 20911 solver.cpp:228] Iteration 15520, loss = 11580.7 I1014 05:40:52.034420 20911 solver.cpp:244] Train net output #0: dsn1_loss = 2.51215 ( 1 = 2.51215 loss) I1014 05:40:52.034425 20911 solver.cpp:244] Train net output #1: dsn2_loss = 1.80652 ( 1 = 1.80652 loss) I1014 05:40:52.034430 20911 solver.cpp:244] Train net output #2: dsn3_loss = 1.38291 ( 1 = 1.38291 loss) I1014 05:40:52.034435 20911 solver.cpp:244] Train net output #3: dsn4_loss = 1.65932 ( 1 = 1.65932 loss) I1014 05:40:52.034438 20911 solver.cpp:244] Train net output #4: dsn5_loss = 1.39213 ( 1 = 1.39213 loss) I1014 05:40:52.034441 20911 solver.cpp:244] Train net output #5: fuse_loss = 1.23653 ( 1 = 1.23653 loss) I1014 05:40:52.034446 20911 sgd_solver.cpp:106] Iteration 15520, lr = 1e-07 I1014 05:41:16.475379 20911 solver.cpp:228] Iteration 15540, loss = 11177.7 I1014 05:41:16.475409 20911 solver.cpp:244] Train net output #0: dsn1_loss = 1496.25 ( 1 = 1496.25 loss) I1014 05:41:16.475416 20911 solver.cpp:244] Train net output #1: dsn2_loss = 1033.11 ( 1 = 1033.11 loss) I1014 05:41:16.475421 20911 solver.cpp:244] Train net output #2: dsn3_loss = 783.78 ( 1 = 783.78 loss) I1014 05:41:16.475427 20911 solver.cpp:244] Train net output #3: dsn4_loss = 866.193 ( 1 = 866.193 loss) I1014 05:41:16.475431 20911 solver.cpp:244] Train net output #4: dsn5_loss = 819.051 ( 1 = 819.051 loss) I1014 05:41:16.475435 20911 solver.cpp:244] Train net output #5: fuse_loss = 717.836 ( 1 = 717.836 loss) I1014 05:41:16.475440 20911 sgd_solver.cpp:106] Iteration 15540, lr = 1e-07 I1014 05:41:40.754981 20911 solver.cpp:228] Iteration 15560, loss = 10859.4 I1014 05:41:40.755007 20911 solver.cpp:244] Train net output #0: dsn1_loss = 0 ( 1 = 0 loss) I1014 05:41:40.755012 20911 solver.cpp:244] Train net output #1: dsn2_loss = 0 ( 1 = 0 loss) I1014 05:41:40.755017 20911 solver.cpp:244] Train net output #2: dsn3_loss = 0 ( 1 = 0 loss) I1014 05:41:40.755023 20911 solver.cpp:244] Train net output #3: dsn4_loss = 0 ( 1 = 0 loss) I1014 05:41:40.755025 20911 solver.cpp:244] Train net output #4: dsn5_loss = 0 ( 1 = 0 loss) I1014 05:41:40.755028 20911 solver.cpp:244] Train net output #5: fuse_loss = 0 ( 1 = 0 loss) I1014 05:41:40.755033 20911 sgd_solver.cpp:106] Iteration 15560, lr = 1e-07 I1014 05:42:06.032444 20911 solver.cpp:228] Iteration 15580, loss = 10675 I1014 05:42:06.032474 20911 solver.cpp:244] Train net output #0: dsn1_loss = 476.518 ( 1 = 476.518 loss) I1014 05:42:06.032481 20911 solver.cpp:244] Train net output #1: dsn2_loss = 455.213 ( 1 = 455.213 loss) I1014 05:42:06.032486 20911 solver.cpp:244] Train net output #2: dsn3_loss = 399.478 ( 1 = 399.478 loss) I1014 05:42:06.032492 20911 solver.cpp:244] Train net output #3: dsn4_loss = 429.075 ( 1 = 429.075 loss) I1014 05:42:06.032495 20911 solver.cpp:244] Train net output #4: dsn5_loss = 427.305 ( 1 = 427.305 loss) I1014 05:42:06.032500 20911 solver.cpp:244] Train net output #5: fuse_loss = 335.449 ( 1 = 335.449 loss) I1014 05:42:06.032502 20911 sgd_solver.cpp:106] Iteration 15580, lr = 1e-07 I1014 05:42:32.208433 20911 solver.cpp:228] Iteration 15600, loss = 10526 I1014 05:42:32.208467 20911 solver.cpp:244] Train net output #0: dsn1_loss = 8271.85 ( 1 = 8271.85 loss) I1014 05:42:32.208472 20911 solver.cpp:244] Train net output #1: dsn2_loss = 7366.77 ( 1 = 7366.77 loss) I1014 05:42:32.208479 20911 solver.cpp:244] Train net output #2: dsn3_loss = 6081.08 ( 1 = 6081.08 loss) I1014 05:42:32.208484 20911 solver.cpp:244] Train net output #3: dsn4_loss = 5733.12 ( 1 = 5733.12 loss) I1014 05:42:32.208488 20911 solver.cpp:244] Train net output #4: dsn5_loss = 5219.05 ( 1 = 5219.05 loss) I1014 05:42:32.208492 20911 solver.cpp:244] Train net output #5: fuse_loss = 4907.45 ( 1 = 4907.45 loss) I1014 05:42:32.208495 20911 sgd_solver.cpp:106] Iteration 15600, lr = 1e-07 I1014 05:42:57.924850 20911 solver.cpp:228] Iteration 15620, loss = 10517.3 I1014 05:42:57.924880 20911 solver.cpp:244] Train net output #0: dsn1_loss = 11957.3 ( 1 = 11957.3 loss) I1014 05:42:57.924887 20911 solver.cpp:244] Train net output #1: dsn2_loss = 10757.7 ( 1 = 10757.7 loss) I1014 05:42:57.924893 20911 solver.cpp:244] Train net output #2: dsn3_loss = 9617.6 ( 1 = 9617.6 loss) I1014 05:42:57.924898 20911 solver.cpp:244] Train net output #3: dsn4_loss = 9110.02 ( 1 = 9110.02 loss) I1014 05:42:57.924902 20911 solver.cpp:244] Train net output #4: dsn5_loss = 8772.76 ( 1 = 8772.76 loss) I1014 05:42:57.924906 20911 solver.cpp:244] Train net output #5: fuse_loss = 8337.74 ( 1 = 8337.74 loss) I1014 05:42:57.924911 20911 sgd_solver.cpp:106] Iteration 15620, lr = 1e-07 I1014 05:43:23.372690 20911 solver.cpp:228] Iteration 15640, loss = 10904.7 I1014 05:43:23.372722 20911 solver.cpp:244] Train net output #0: dsn1_loss = 0.821647 ( 1 = 0.821647 loss) I1014 05:43:23.372730 20911 solver.cpp:244] Train net output #1: dsn2_loss = 0.464498 ( 1 = 0.464498 loss) I1014 05:43:23.372735 20911 solver.cpp:244] Train net output #2: dsn3_loss = 0.433278 ( 1 = 0.433278 loss) I1014 05:43:23.372740 20911 solver.cpp:244] Train net output #3: dsn4_loss = 0.470036 ( 1 = 0.470036 loss) I1014 05:43:23.372745 20911 solver.cpp:244] Train net output #4: dsn5_loss = 0.656476 ( 1 = 0.656476 loss) I1014 05:43:23.372750 20911 solver.cpp:244] Train net output #5: fuse_loss = 0.429376 ( 1 = 0.429376 loss) I1014 05:43:23.372756 20911 sgd_solver.cpp:106] Iteration 15640, lr = 1e-07 I1014 05:43:49.004968 20911 solver.cpp:228] Iteration 15660, loss = 11860 I1014 05:43:49.005000 20911 solver.cpp:244] Train net output #0: dsn1_loss = 5445.25 ( 1 = 5445.25 loss) I1014 05:43:49.005007 20911 solver.cpp:244] Train net output #1: dsn2_loss = 3755.11 ( 1 = 3755.11 loss) I1014 05:43:49.005012 20911 solver.cpp:244] Train net output #2: dsn3_loss = 3843.39 ( 1 = 3843.39 loss) I1014 05:43:49.005017 20911 solver.cpp:244] Train net output #3: dsn4_loss = 3978.5 ( 1 = 3978.5 loss) I1014 05:43:49.005022 20911 solver.cpp:244] Train net output #4: dsn5_loss = 3553.97 ( 1 = 3553.97 loss) I1014 05:43:49.005025 20911 solver.cpp:244] Train net output #5: fuse_loss = 3100.27 ( 1 = 3100.27 loss) I1014 05:43:49.005028 20911 sgd_solver.cpp:106] Iteration 15660, lr = 1e-07 I1014 05:44:13.195367 20911 solver.cpp:228] Iteration 15680, loss = 11782 I1014 05:44:13.195395 20911 solver.cpp:244] Train net output #0: dsn1_loss = 5.0414 ( 1 = 5.0414 loss) I1014 05:44:13.195401 20911 solver.cpp:244] Train net output #1: dsn2_loss = 4.9662 ( 1 = 4.9662 loss) I1014 05:44:13.195406 20911 solver.cpp:244] Train net output #2: dsn3_loss = 5.73716 ( 1 = 5.73716 loss) I1014 05:44:13.195412 20911 solver.cpp:244] Train net output #3: dsn4_loss = 7.66155 ( 1 = 7.66155 loss) I1014 05:44:13.195416 20911 solver.cpp:244] Train net output #4: dsn5_loss = 7.28621 ( 1 = 7.28621 loss) I1014 05:44:13.195420 20911 solver.cpp:244] Train net output #5: fuse_loss = 5.45148 ( 1 = 5.45148 loss) I1014 05:44:13.195423 20911 sgd_solver.cpp:106] Iteration 15680, lr = 1e-07 I1014 05:44:38.709636 20911 solver.cpp:228] Iteration 15700, loss = 11002.4 I1014 05:44:38.709664 20911 solver.cpp:244] Train net output #0: dsn1_loss = 2215.7 ( 1 = 2215.7 loss) I1014 05:44:38.709671 20911 solver.cpp:244] Train net output #1: dsn2_loss = 2021.88 ( 1 = 2021.88 loss) I1014 05:44:38.709676 20911 solver.cpp:244] Train net output #2: dsn3_loss = 1829.35 ( 1 = 1829.35 loss) I1014 05:44:38.709681 20911 solver.cpp:244] Train net output #3: dsn4_loss = 1552.12 ( 1 = 1552.12 loss) I1014 05:44:38.709687 20911 solver.cpp:244] Train net output #4: dsn5_loss = 1289.72 ( 1 = 1289.72 loss) I1014 05:44:38.709693 20911 solver.cpp:244] Train net output #5: fuse_loss = 1444.63 ( 1 = 1444.63 loss) I1014 05:44:38.709697 20911 sgd_solver.cpp:106] Iteration 15700, lr = 1e-07 I1014 05:45:05.359202 20911 solver.cpp:228] Iteration 15720, loss = 10526.8 I1014 05:45:05.359237 20911 solver.cpp:244] Train net output #0: dsn1_loss = 1445.56 ( 1 = 1445.56 loss) I1014 05:45:05.359244 20911 solver.cpp:244] Train net output #1: dsn2_loss = 694.754 ( 1 = 694.754 loss) I1014 05:45:05.359251 20911 solver.cpp:244] Train net output #2: dsn3_loss = 564.937 ( 1 = 564.937 loss) I1014 05:45:05.359256 20911 solver.cpp:244] Train net output #3: dsn4_loss = 705.243 ( 1 = 705.243 loss) I1014 05:45:05.359259 20911 solver.cpp:244] Train net output #4: dsn5_loss = 929.147 ( 1 = 929.147 loss) I1014 05:45:05.359263 20911 solver.cpp:244] Train net output #5: fuse_loss = 564.22 ( 1 = 564.22 loss) I1014 05:45:05.359267 20911 sgd_solver.cpp:106] Iteration 15720, lr = 1e-07 I1014 05:45:32.078522 20911 solver.cpp:228] Iteration 15740, loss = 10543.9 I1014 05:45:32.078552 20911 solver.cpp:244] Train net output #0: dsn1_loss = 7398.08 ( 1 = 7398.08 loss) I1014 05:45:32.078559 20911 solver.cpp:244] Train net output #1: dsn2_loss = 6300.53 ( 1 = 6300.53 loss) I1014 05:45:32.078565 20911 solver.cpp:244] Train net output #2: dsn3_loss = 4835.53 ( 1 = 4835.53 loss) I1014 05:45:32.078570 20911 solver.cpp:244] Train net output #3: dsn4_loss = 4084.03 ( 1 = 4084.03 loss) I1014 05:45:32.078574 20911 solver.cpp:244] Train net output #4: dsn5_loss = 3221.01 ( 1 = 3221.01 loss) I1014 05:45:32.078577 20911 solver.cpp:244] Train net output #5: fuse_loss = 3378.85 ( 1 = 3378.85 loss) I1014 05:45:32.078582 20911 sgd_solver.cpp:106] Iteration 15740, lr = 1e-07 I1014 05:45:57.523128 20911 solver.cpp:228] Iteration 15760, loss = 11471.7 I1014 05:45:57.523155 20911 solver.cpp:244] Train net output #0: dsn1_loss = 46.9955 ( 1 = 46.9955 loss) I1014 05:45:57.523161 20911 solver.cpp:244] Train net output #1: dsn2_loss = 29.7005 ( 1 = 29.7005 loss) I1014 05:45:57.523167 20911 solver.cpp:244] Train net output #2: dsn3_loss = 21.7094 ( 1 = 21.7094 loss) I1014 05:45:57.523172 20911 solver.cpp:244] Train net output #3: dsn4_loss = 22.7717 ( 1 = 22.7717 loss) I1014 05:45:57.523175 20911 solver.cpp:244] Train net output #4: dsn5_loss = 22.3725 ( 1 = 22.3725 loss) I1014 05:45:57.523180 20911 solver.cpp:244] Train net output #5: fuse_loss = 18.8462 ( 1 = 18.8462 loss) I1014 05:45:57.523183 20911 sgd_solver.cpp:106] Iteration 15760, lr = 1e-07 I1014 05:46:24.263411 20911 solver.cpp:228] Iteration 15780, loss = 10880.8 I1014 05:46:24.263439 20911 solver.cpp:244] Train net output #0: dsn1_loss = 227.864 ( 1 = 227.864 loss) I1014 05:46:24.263447 20911 solver.cpp:244] Train net output #1: dsn2_loss = 196.429 ( 1 = 196.429 loss) I1014 05:46:24.263451 20911 solver.cpp:244] Train net output #2: dsn3_loss = 149.718 ( 1 = 149.718 loss) I1014 05:46:24.263455 20911 solver.cpp:244] Train net output #3: dsn4_loss = 120.896 ( 1 = 120.896 loss) I1014 05:46:24.263459 20911 solver.cpp:244] Train net output #4: dsn5_loss = 84.8554 ( 1 = 84.8554 loss) I1014 05:46:24.263463 20911 solver.cpp:244] Train net output #5: fuse_loss = 97.1727 ( 1 = 97.1727 loss) I1014 05:46:24.263466 20911 sgd_solver.cpp:106] Iteration 15780, lr = 1e-07 I1014 05:46:48.771651 20911 solver.cpp:228] Iteration 15800, loss = 10258.6 I1014 05:46:48.771677 20911 solver.cpp:244] Train net output #0: dsn1_loss = 1.74919 ( 1 = 1.74919 loss) I1014 05:46:48.771683 20911 solver.cpp:244] Train net output #1: dsn2_loss = 2.02634 ( 1 = 2.02634 loss) I1014 05:46:48.771688 20911 solver.cpp:244] Train net output #2: dsn3_loss = 1.42163 ( 1 = 1.42163 loss) I1014 05:46:48.771693 20911 solver.cpp:244] Train net output #3: dsn4_loss = 1.18217 ( 1 = 1.18217 loss) I1014 05:46:48.771698 20911 solver.cpp:244] Train net output #4: dsn5_loss = 0.940239 ( 1 = 0.940239 loss) I1014 05:46:48.771701 20911 solver.cpp:244] Train net output #5: fuse_loss = 1.09887 ( 1 = 1.09887 loss) I1014 05:46:48.771705 20911 sgd_solver.cpp:106] Iteration 15800, lr = 1e-07 I1014 05:47:11.809387 20911 solver.cpp:228] Iteration 15820, loss = 9733.33 I1014 05:47:11.809414 20911 solver.cpp:244] Train net output #0: dsn1_loss = 395.214 ( 1 = 395.214 loss) I1014 05:47:11.809420 20911 solver.cpp:244] Train net output #1: dsn2_loss = 286.748 ( 1 = 286.748 loss) I1014 05:47:11.809427 20911 solver.cpp:244] Train net output #2: dsn3_loss = 217.512 ( 1 = 217.512 loss) I1014 05:47:11.809430 20911 solver.cpp:244] Train net output #3: dsn4_loss = 267.828 ( 1 = 267.828 loss) I1014 05:47:11.809434 20911 solver.cpp:244] Train net output #4: dsn5_loss = 259.289 ( 1 = 259.289 loss) I1014 05:47:11.809438 20911 solver.cpp:244] Train net output #5: fuse_loss = 210.667 ( 1 = 210.667 loss) I1014 05:47:11.809442 20911 sgd_solver.cpp:106] Iteration 15820, lr = 1e-07 I1014 05:47:37.127895 20911 solver.cpp:228] Iteration 15840, loss = 10509.7 I1014 05:47:37.127923 20911 solver.cpp:244] Train net output #0: dsn1_loss = 5880.04 ( 1 = 5880.04 loss) I1014 05:47:37.127930 20911 solver.cpp:244] Train net output #1: dsn2_loss = 5183.94 ( 1 = 5183.94 loss) I1014 05:47:37.127936 20911 solver.cpp:244] Train net output #2: dsn3_loss = 4486.18 ( 1 = 4486.18 loss) I1014 05:47:37.127941 20911 solver.cpp:244] Train net output #3: dsn4_loss = 4413.16 ( 1 = 4413.16 loss) I1014 05:47:37.127945 20911 solver.cpp:244] Train net output #4: dsn5_loss = 4032.77 ( 1 = 4032.77 loss) I1014 05:47:37.127949 20911 solver.cpp:244] Train net output #5: fuse_loss = 3878.12 ( 1 = 3878.12 loss) I1014 05:47:37.127954 20911 sgd_solver.cpp:106] Iteration 15840, lr = 1e-07 I1014 05:48:01.917176 20911 solver.cpp:228] Iteration 15860, loss = 9988.72 I1014 05:48:01.917203 20911 solver.cpp:244] Train net output #0: dsn1_loss = 0 ( 1 = 0 loss) I1014 05:48:01.917208 20911 solver.cpp:244] Train net output #1: dsn2_loss = 0 ( 1 = 0 loss) I1014 05:48:01.917213 20911 solver.cpp:244] Train net output #2: dsn3_loss = 0 ( 1 = 0 loss) I1014 05:48:01.917217 20911 solver.cpp:244] Train net output #3: dsn4_loss = 0 ( 1 = 0 loss) I1014 05:48:01.917222 20911 solver.cpp:244] Train net output #4: dsn5_loss = 0 ( 1 = 0 loss) I1014 05:48:01.917224 20911 solver.cpp:244] Train net output #5: fuse_loss = 0 ( 1 = 0 loss) I1014 05:48:01.917227 20911 sgd_solver.cpp:106] Iteration 15860, lr = 1e-07 I1014 05:48:28.190279 20911 solver.cpp:228] Iteration 15880, loss = 10827.5 I1014 05:48:28.190306 20911 solver.cpp:244] Train net output #0: dsn1_loss = 3.06366 ( 1 = 3.06366 loss) I1014 05:48:28.190311 20911 solver.cpp:244] Train net output #1: dsn2_loss = 1.76942 ( 1 = 1.76942 loss) I1014 05:48:28.190316 20911 solver.cpp:244] Train net output #2: dsn3_loss = 2.25607 ( 1 = 2.25607 loss) I1014 05:48:28.190321 20911 solver.cpp:244] Train net output #3: dsn4_loss = 3.26533 ( 1 = 3.26533 loss) I1014 05:48:28.190325 20911 solver.cpp:244] Train net output #4: dsn5_loss = 2.72206 ( 1 = 2.72206 loss) I1014 05:48:28.190330 20911 solver.cpp:244] Train net output #5: fuse_loss = 1.92809 ( 1 = 1.92809 loss) I1014 05:48:28.190332 20911 sgd_solver.cpp:106] Iteration 15880, lr = 1e-07 I1014 05:48:51.662297 20911 solver.cpp:228] Iteration 15900, loss = 11561.3 I1014 05:48:51.662323 20911 solver.cpp:244] Train net output #0: dsn1_loss = 813.038 ( 1 = 813.038 loss) I1014 05:48:51.662329 20911 solver.cpp:244] Train net output #1: dsn2_loss = 642.072 ( 1 = 642.072 loss) I1014 05:48:51.662335 20911 solver.cpp:244] Train net output #2: dsn3_loss = 477.312 ( 1 = 477.312 loss) I1014 05:48:51.662340 20911 solver.cpp:244] Train net output #3: dsn4_loss = 422.256 ( 1 = 422.256 loss) I1014 05:48:51.662344 20911 solver.cpp:244] Train net output #4: dsn5_loss = 317.064 ( 1 = 317.064 loss) I1014 05:48:51.662348 20911 solver.cpp:244] Train net output #5: fuse_loss = 369.301 ( 1 = 369.301 loss) I1014 05:48:51.662351 20911 sgd_solver.cpp:106] Iteration 15900, lr = 1e-07 I1014 05:49:17.982653 20911 solver.cpp:228] Iteration 15920, loss = 11168 I1014 05:49:17.982682 20911 solver.cpp:244] Train net output #0: dsn1_loss = 35.5561 ( 1 = 35.5561 loss) I1014 05:49:17.982689 20911 solver.cpp:244] Train net output #1: dsn2_loss = 28.0262 ( 1 = 28.0262 loss) I1014 05:49:17.982695 20911 solver.cpp:244] Train net output #2: dsn3_loss = 24.7187 ( 1 = 24.7187 loss) I1014 05:49:17.982699 20911 solver.cpp:244] Train net output #3: dsn4_loss = 30.6996 ( 1 = 30.6996 loss) I1014 05:49:17.982704 20911 solver.cpp:244] Train net output #4: dsn5_loss = 32.4744 ( 1 = 32.4744 loss) I1014 05:49:17.982708 20911 solver.cpp:244] Train net output #5: fuse_loss = 23.1734 ( 1 = 23.1734 loss) I1014 05:49:17.982712 20911 sgd_solver.cpp:106] Iteration 15920, lr = 1e-07 I1014 05:49:44.697584 20911 solver.cpp:228] Iteration 15940, loss = 10341.2 I1014 05:49:44.697615 20911 solver.cpp:244] Train net output #0: dsn1_loss = 554.558 ( 1 = 554.558 loss) I1014 05:49:44.697621 20911 solver.cpp:244] Train net output #1: dsn2_loss = 426.584 ( 1 = 426.584 loss) I1014 05:49:44.697628 20911 solver.cpp:244] Train net output #2: dsn3_loss = 317.418 ( 1 = 317.418 loss) I1014 05:49:44.697631 20911 solver.cpp:244] Train net output #3: dsn4_loss = 365.196 ( 1 = 365.196 loss) I1014 05:49:44.697635 20911 solver.cpp:244] Train net output #4: dsn5_loss = 344.502 ( 1 = 344.502 loss) I1014 05:49:44.697639 20911 solver.cpp:244] Train net output #5: fuse_loss = 306.424 ( 1 = 306.424 loss) I1014 05:49:44.697643 20911 sgd_solver.cpp:106] Iteration 15940, lr = 1e-07 I1014 05:50:09.639932 20911 solver.cpp:228] Iteration 15960, loss = 9907.17 I1014 05:50:09.639961 20911 solver.cpp:244] Train net output #0: dsn1_loss = 5.83051 ( 1 = 5.83051 loss) I1014 05:50:09.639966 20911 solver.cpp:244] Train net output #1: dsn2_loss = 3.46239 ( 1 = 3.46239 loss) I1014 05:50:09.639971 20911 solver.cpp:244] Train net output #2: dsn3_loss = 2.49443 ( 1 = 2.49443 loss) I1014 05:50:09.639976 20911 solver.cpp:244] Train net output #3: dsn4_loss = 3.66154 ( 1 = 3.66154 loss) I1014 05:50:09.639981 20911 solver.cpp:244] Train net output #4: dsn5_loss = 3.29817 ( 1 = 3.29817 loss) I1014 05:50:09.639983 20911 solver.cpp:244] Train net output #5: fuse_loss = 2.8282 ( 1 = 2.8282 loss) I1014 05:50:09.639986 20911 sgd_solver.cpp:106] Iteration 15960, lr = 1e-07 I1014 05:50:36.231796 20911 solver.cpp:228] Iteration 15980, loss = 9992.55 I1014 05:50:36.231824 20911 solver.cpp:244] Train net output #0: dsn1_loss = 1222.35 ( 1 = 1222.35 loss) I1014 05:50:36.231830 20911 solver.cpp:244] Train net output #1: dsn2_loss = 870.714 ( 1 = 870.714 loss) I1014 05:50:36.231837 20911 solver.cpp:244] Train net output #2: dsn3_loss = 626.268 ( 1 = 626.268 loss) I1014 05:50:36.231842 20911 solver.cpp:244] Train net output #3: dsn4_loss = 547.872 ( 1 = 547.872 loss) I1014 05:50:36.231847 20911 solver.cpp:244] Train net output #4: dsn5_loss = 456.31 ( 1 = 456.31 loss) I1014 05:50:36.231850 20911 solver.cpp:244] Train net output #5: fuse_loss = 537.605 ( 1 = 537.605 loss) I1014 05:50:36.231854 20911 sgd_solver.cpp:106] Iteration 15980, lr = 1e-07 I1014 05:51:01.147485 20911 solver.cpp:228] Iteration 16000, loss = 9651 I1014 05:51:01.147524 20911 solver.cpp:244] Train net output #0: dsn1_loss = 633.208 ( 1 = 633.208 loss) I1014 05:51:01.147531 20911 solver.cpp:244] Train net output #1: dsn2_loss = 508.023 ( 1 = 508.023 loss) I1014 05:51:01.147536 20911 solver.cpp:244] Train net output #2: dsn3_loss = 418.373 ( 1 = 418.373 loss) I1014 05:51:01.147542 20911 solver.cpp:244] Train net output #3: dsn4_loss = 342.313 ( 1 = 342.313 loss) I1014 05:51:01.147547 20911 solver.cpp:244] Train net output #4: dsn5_loss = 286.748 ( 1 = 286.748 loss) I1014 05:51:01.147550 20911 solver.cpp:244] Train net output #5: fuse_loss = 297.344 ( 1 = 297.344 loss) I1014 05:51:01.147555 20911 sgd_solver.cpp:106] Iteration 16000, lr = 1e-07 I1014 05:51:26.148540 20911 solver.cpp:228] Iteration 16020, loss = 10478.6 I1014 05:51:26.148566 20911 solver.cpp:244] Train net output #0: dsn1_loss = 314.498 ( 1 = 314.498 loss) I1014 05:51:26.148572 20911 solver.cpp:244] Train net output #1: dsn2_loss = 172.7 ( 1 = 172.7 loss) I1014 05:51:26.148578 20911 solver.cpp:244] Train net output #2: dsn3_loss = 113.997 ( 1 = 113.997 loss) I1014 05:51:26.148583 20911 solver.cpp:244] Train net output #3: dsn4_loss = 70.1352 ( 1 = 70.1352 loss) I1014 05:51:26.148586 20911 solver.cpp:244] Train net output #4: dsn5_loss = 62.8258 ( 1 = 62.8258 loss) I1014 05:51:26.148591 20911 solver.cpp:244] Train net output #5: fuse_loss = 77.8756 ( 1 = 77.8756 loss) I1014 05:51:26.148593 20911 sgd_solver.cpp:106] Iteration 16020, lr = 1e-07 I1014 05:51:50.148157 20911 solver.cpp:228] Iteration 16040, loss = 10611.8 I1014 05:51:50.148186 20911 solver.cpp:244] Train net output #0: dsn1_loss = 352.797 ( 1 = 352.797 loss) I1014 05:51:50.148195 20911 solver.cpp:244] Train net output #1: dsn2_loss = 301.491 ( 1 = 301.491 loss) I1014 05:51:50.148200 20911 solver.cpp:244] Train net output #2: dsn3_loss = 173.59 ( 1 = 173.59 loss) I1014 05:51:50.148203 20911 solver.cpp:244] Train net output #3: dsn4_loss = 146.365 ( 1 = 146.365 loss) I1014 05:51:50.148207 20911 solver.cpp:244] Train net output #4: dsn5_loss = 149.126 ( 1 = 149.126 loss) I1014 05:51:50.148211 20911 solver.cpp:244] Train net output #5: fuse_loss = 141.085 ( 1 = 141.085 loss) I1014 05:51:50.148216 20911 sgd_solver.cpp:106] Iteration 16040, lr = 1e-07 I1014 05:52:15.011241 20911 solver.cpp:228] Iteration 16060, loss = 10602.3 I1014 05:52:15.011271 20911 solver.cpp:244] Train net output #0: dsn1_loss = 4129.56 ( 1 = 4129.56 loss) I1014 05:52:15.011278 20911 solver.cpp:244] Train net output #1: dsn2_loss = 3378.79 ( 1 = 3378.79 loss) I1014 05:52:15.011284 20911 solver.cpp:244] Train net output #2: dsn3_loss = 2635.86 ( 1 = 2635.86 loss) I1014 05:52:15.011289 20911 solver.cpp:244] Train net output #3: dsn4_loss = 2112.05 ( 1 = 2112.05 loss) I1014 05:52:15.011293 20911 solver.cpp:244] Train net output #4: dsn5_loss = 1715.98 ( 1 = 1715.98 loss) I1014 05:52:15.011297 20911 solver.cpp:244] Train net output #5: fuse_loss = 1739.36 ( 1 = 1739.36 loss) I1014 05:52:15.011301 20911 sgd_solver.cpp:106] Iteration 16060, lr = 1e-07 I1014 05:52:39.515117 20911 solver.cpp:228] Iteration 16080, loss = 10737.3 I1014 05:52:39.515148 20911 solver.cpp:244] Train net output #0: dsn1_loss = 19486.9 ( 1 = 19486.9 loss) I1014 05:52:39.515154 20911 solver.cpp:244] Train net output #1: dsn2_loss = 17980 ( 1 = 17980 loss) I1014 05:52:39.515161 20911 solver.cpp:244] Train net output #2: dsn3_loss = 17246.8 ( 1 = 17246.8 loss) I1014 05:52:39.515164 20911 solver.cpp:244] Train net output #3: dsn4_loss = 18130.5 ( 1 = 18130.5 loss) I1014 05:52:39.515169 20911 solver.cpp:244] Train net output #4: dsn5_loss = 16936.3 ( 1 = 16936.3 loss) I1014 05:52:39.515172 20911 solver.cpp:244] Train net output #5: fuse_loss = 15310.2 ( 1 = 15310.2 loss) I1014 05:52:39.515177 20911 sgd_solver.cpp:106] Iteration 16080, lr = 1e-07 I1014 05:53:04.493036 20911 solver.cpp:228] Iteration 16100, loss = 9990.64 I1014 05:53:04.493068 20911 solver.cpp:244] Train net output #0: dsn1_loss = 4321.22 ( 1 = 4321.22 loss) I1014 05:53:04.493075 20911 solver.cpp:244] Train net output #1: dsn2_loss = 3456.07 ( 1 = 3456.07 loss) I1014 05:53:04.493082 20911 solver.cpp:244] Train net output #2: dsn3_loss = 2794.12 ( 1 = 2794.12 loss) I1014 05:53:04.493085 20911 solver.cpp:244] Train net output #3: dsn4_loss = 2556.45 ( 1 = 2556.45 loss) I1014 05:53:04.493090 20911 solver.cpp:244] Train net output #4: dsn5_loss = 2009.21 ( 1 = 2009.21 loss) I1014 05:53:04.493093 20911 solver.cpp:244] Train net output #5: fuse_loss = 2084.57 ( 1 = 2084.57 loss) I1014 05:53:04.493098 20911 sgd_solver.cpp:106] Iteration 16100, lr = 1e-07 I1014 05:53:30.118983 20911 solver.cpp:228] Iteration 16120, loss = 10049.5 I1014 05:53:30.119010 20911 solver.cpp:244] Train net output #0: dsn1_loss = 177.161 ( 1 = 177.161 loss) I1014 05:53:30.119017 20911 solver.cpp:244] Train net output #1: dsn2_loss = 117.259 ( 1 = 117.259 loss) I1014 05:53:30.119022 20911 solver.cpp:244] Train net output #2: dsn3_loss = 100.756 ( 1 = 100.756 loss) I1014 05:53:30.119026 20911 solver.cpp:244] Train net output #3: dsn4_loss = 114.712 ( 1 = 114.712 loss) I1014 05:53:30.119031 20911 solver.cpp:244] Train net output #4: dsn5_loss = 122.877 ( 1 = 122.877 loss) I1014 05:53:30.119035 20911 solver.cpp:244] Train net output #5: fuse_loss = 83.8302 ( 1 = 83.8302 loss) I1014 05:53:30.119038 20911 sgd_solver.cpp:106] Iteration 16120, lr = 1e-07 I1014 05:53:56.116410 20911 solver.cpp:228] Iteration 16140, loss = 10693.4 I1014 05:53:56.116441 20911 solver.cpp:244] Train net output #0: dsn1_loss = 5353.58 ( 1 = 5353.58 loss) I1014 05:53:56.116447 20911 solver.cpp:244] Train net output #1: dsn2_loss = 4519.56 ( 1 = 4519.56 loss) I1014 05:53:56.116453 20911 solver.cpp:244] Train net output #2: dsn3_loss = 3552.88 ( 1 = 3552.88 loss) I1014 05:53:56.116458 20911 solver.cpp:244] Train net output #3: dsn4_loss = 2799.65 ( 1 = 2799.65 loss) I1014 05:53:56.116462 20911 solver.cpp:244] Train net output #4: dsn5_loss = 2200.58 ( 1 = 2200.58 loss) I1014 05:53:56.116466 20911 solver.cpp:244] Train net output #5: fuse_loss = 2266.22 ( 1 = 2266.22 loss) I1014 05:53:56.116469 20911 sgd_solver.cpp:106] Iteration 16140, lr = 1e-07 I1014 05:54:21.995559 20911 solver.cpp:228] Iteration 16160, loss = 10376 I1014 05:54:21.995590 20911 solver.cpp:244] Train net output #0: dsn1_loss = 4362.66 ( 1 = 4362.66 loss) I1014 05:54:21.995596 20911 solver.cpp:244] Train net output #1: dsn2_loss = 3439.99 ( 1 = 3439.99 loss) I1014 05:54:21.995602 20911 solver.cpp:244] Train net output #2: dsn3_loss = 3041.2 ( 1 = 3041.2 loss) I1014 05:54:21.995609 20911 solver.cpp:244] Train net output #3: dsn4_loss = 2790.9 ( 1 = 2790.9 loss) I1014 05:54:21.995613 20911 solver.cpp:244] Train net output #4: dsn5_loss = 2803.71 ( 1 = 2803.71 loss) I1014 05:54:21.995616 20911 solver.cpp:244] Train net output #5: fuse_loss = 2560.08 ( 1 = 2560.08 loss) I1014 05:54:21.995620 20911 sgd_solver.cpp:106] Iteration 16160, lr = 1e-07 I1014 05:54:47.238253 20911 solver.cpp:228] Iteration 16180, loss = 9686.8 I1014 05:54:47.238281 20911 solver.cpp:244] Train net output #0: dsn1_loss = 762.385 ( 1 = 762.385 loss) I1014 05:54:47.238288 20911 solver.cpp:244] Train net output #1: dsn2_loss = 337.555 ( 1 = 337.555 loss) I1014 05:54:47.238293 20911 solver.cpp:244] Train net output #2: dsn3_loss = 269.875 ( 1 = 269.875 loss) I1014 05:54:47.238298 20911 solver.cpp:244] Train net output #3: dsn4_loss = 374.055 ( 1 = 374.055 loss) I1014 05:54:47.238302 20911 solver.cpp:244] Train net output #4: dsn5_loss = 317.367 ( 1 = 317.367 loss) I1014 05:54:47.238306 20911 solver.cpp:244] Train net output #5: fuse_loss = 257.547 ( 1 = 257.547 loss) I1014 05:54:47.238310 20911 sgd_solver.cpp:106] Iteration 16180, lr = 1e-07 I1014 05:55:12.526772 20911 solver.cpp:228] Iteration 16200, loss = 9587.72 I1014 05:55:12.526798 20911 solver.cpp:244] Train net output #0: dsn1_loss = 175.133 ( 1 = 175.133 loss) I1014 05:55:12.526805 20911 solver.cpp:244] Train net output #1: dsn2_loss = 82.2639 ( 1 = 82.2639 loss) I1014 05:55:12.526811 20911 solver.cpp:244] Train net output #2: dsn3_loss = 49.0098 ( 1 = 49.0098 loss) I1014 05:55:12.526815 20911 solver.cpp:244] Train net output #3: dsn4_loss = 51.701 ( 1 = 51.701 loss) I1014 05:55:12.526819 20911 solver.cpp:244] Train net output #4: dsn5_loss = 31.6585 ( 1 = 31.6585 loss) I1014 05:55:12.526823 20911 solver.cpp:244] Train net output #5: fuse_loss = 32.317 ( 1 = 32.317 loss) I1014 05:55:12.526827 20911 sgd_solver.cpp:106] Iteration 16200, lr = 1e-07 I1014 05:55:38.083823 20911 solver.cpp:228] Iteration 16220, loss = 9756.06 I1014 05:55:38.083850 20911 solver.cpp:244] Train net output #0: dsn1_loss = 1.79719 ( 1 = 1.79719 loss) I1014 05:55:38.083856 20911 solver.cpp:244] Train net output #1: dsn2_loss = 1.22828 ( 1 = 1.22828 loss) I1014 05:55:38.083861 20911 solver.cpp:244] Train net output #2: dsn3_loss = 1.24752 ( 1 = 1.24752 loss) I1014 05:55:38.083864 20911 solver.cpp:244] Train net output #3: dsn4_loss = 1.38832 ( 1 = 1.38832 loss) I1014 05:55:38.083868 20911 solver.cpp:244] Train net output #4: dsn5_loss = 1.53064 ( 1 = 1.53064 loss) I1014 05:55:38.083871 20911 solver.cpp:244] Train net output #5: fuse_loss = 0.96859 ( 1 = 0.96859 loss) I1014 05:55:38.083875 20911 sgd_solver.cpp:106] Iteration 16220, lr = 1e-07 I1014 05:56:02.971521 20911 solver.cpp:228] Iteration 16240, loss = 10134.7 I1014 05:56:02.971557 20911 solver.cpp:244] Train net output #0: dsn1_loss = 1703.5 ( 1 = 1703.5 loss) I1014 05:56:02.971565 20911 solver.cpp:244] Train net output #1: dsn2_loss = 1376.44 ( 1 = 1376.44 loss) I1014 05:56:02.971570 20911 solver.cpp:244] Train net output #2: dsn3_loss = 1111.56 ( 1 = 1111.56 loss) I1014 05:56:02.971575 20911 solver.cpp:244] Train net output #3: dsn4_loss = 1259.75 ( 1 = 1259.75 loss) I1014 05:56:02.971578 20911 solver.cpp:244] Train net output #4: dsn5_loss = 1550.7 ( 1 = 1550.7 loss) I1014 05:56:02.971582 20911 solver.cpp:244] Train net output #5: fuse_loss = 1082.21 ( 1 = 1082.21 loss) I1014 05:56:02.971585 20911 sgd_solver.cpp:106] Iteration 16240, lr = 1e-07 I1014 05:56:27.523618 20911 solver.cpp:228] Iteration 16260, loss = 9986.08 I1014 05:56:27.523648 20911 solver.cpp:244] Train net output #0: dsn1_loss = 7.13524 ( 1 = 7.13524 loss) I1014 05:56:27.523653 20911 solver.cpp:244] Train net output #1: dsn2_loss = 7.65038 ( 1 = 7.65038 loss) I1014 05:56:27.523658 20911 solver.cpp:244] Train net output #2: dsn3_loss = 8.31419 ( 1 = 8.31419 loss) I1014 05:56:27.523663 20911 solver.cpp:244] Train net output #3: dsn4_loss = 14.6759 ( 1 = 14.6759 loss) I1014 05:56:27.523669 20911 solver.cpp:244] Train net output #4: dsn5_loss = 22.1535 ( 1 = 22.1535 loss) I1014 05:56:27.523671 20911 solver.cpp:244] Train net output #5: fuse_loss = 11.1874 ( 1 = 11.1874 loss) I1014 05:56:27.523674 20911 sgd_solver.cpp:106] Iteration 16260, lr = 1e-07 I1014 05:56:52.536909 20911 solver.cpp:228] Iteration 16280, loss = 9853.57 I1014 05:56:52.536937 20911 solver.cpp:244] Train net output #0: dsn1_loss = 716.981 ( 1 = 716.981 loss) I1014 05:56:52.536944 20911 solver.cpp:244] Train net output #1: dsn2_loss = 517.056 ( 1 = 517.056 loss) I1014 05:56:52.536950 20911 solver.cpp:244] Train net output #2: dsn3_loss = 459.267 ( 1 = 459.267 loss) I1014 05:56:52.536955 20911 solver.cpp:244] Train net output #3: dsn4_loss = 364.506 ( 1 = 364.506 loss) I1014 05:56:52.536958 20911 solver.cpp:244] Train net output #4: dsn5_loss = 197.497 ( 1 = 197.497 loss) I1014 05:56:52.536962 20911 solver.cpp:244] Train net output #5: fuse_loss = 240.235 ( 1 = 240.235 loss) I1014 05:56:52.536967 20911 sgd_solver.cpp:106] Iteration 16280, lr = 1e-07 I1014 05:57:17.503501 20911 solver.cpp:228] Iteration 16300, loss = 10006.2 I1014 05:57:17.503535 20911 solver.cpp:244] Train net output #0: dsn1_loss = 2228.88 ( 1 = 2228.88 loss) I1014 05:57:17.503542 20911 solver.cpp:244] Train net output #1: dsn2_loss = 2451.56 ( 1 = 2451.56 loss) I1014 05:57:17.503547 20911 solver.cpp:244] Train net output #2: dsn3_loss = 1567.4 ( 1 = 1567.4 loss) I1014 05:57:17.503553 20911 solver.cpp:244] Train net output #3: dsn4_loss = 1208.82 ( 1 = 1208.82 loss) I1014 05:57:17.503557 20911 solver.cpp:244] Train net output #4: dsn5_loss = 726.554 ( 1 = 726.554 loss) I1014 05:57:17.503561 20911 solver.cpp:244] Train net output #5: fuse_loss = 1378.35 ( 1 = 1378.35 loss) I1014 05:57:17.503566 20911 sgd_solver.cpp:106] Iteration 16300, lr = 1e-07 I1014 05:57:41.668489 20911 solver.cpp:228] Iteration 16320, loss = 10834 I1014 05:57:41.668517 20911 solver.cpp:244] Train net output #0: dsn1_loss = 35.5056 ( 1 = 35.5056 loss) I1014 05:57:41.668524 20911 solver.cpp:244] Train net output #1: dsn2_loss = 31.1508 ( 1 = 31.1508 loss) I1014 05:57:41.668529 20911 solver.cpp:244] Train net output #2: dsn3_loss = 29.2239 ( 1 = 29.2239 loss) I1014 05:57:41.668534 20911 solver.cpp:244] Train net output #3: dsn4_loss = 43.7482 ( 1 = 43.7482 loss) I1014 05:57:41.668540 20911 solver.cpp:244] Train net output #4: dsn5_loss = 42.7887 ( 1 = 42.7887 loss) I1014 05:57:41.668542 20911 solver.cpp:244] Train net output #5: fuse_loss = 29.085 ( 1 = 29.085 loss) I1014 05:57:41.668546 20911 sgd_solver.cpp:106] Iteration 16320, lr = 1e-07 I1014 05:58:06.013041 20911 solver.cpp:228] Iteration 16340, loss = 11645.8 I1014 05:58:06.013072 20911 solver.cpp:244] Train net output #0: dsn1_loss = 3204.75 ( 1 = 3204.75 loss) I1014 05:58:06.013080 20911 solver.cpp:244] Train net output #1: dsn2_loss = 2436.9 ( 1 = 2436.9 loss) I1014 05:58:06.013087 20911 solver.cpp:244] Train net output #2: dsn3_loss = 2074.96 ( 1 = 2074.96 loss) I1014 05:58:06.013092 20911 solver.cpp:244] Train net output #3: dsn4_loss = 1935.38 ( 1 = 1935.38 loss) I1014 05:58:06.013095 20911 solver.cpp:244] Train net output #4: dsn5_loss = 1656.68 ( 1 = 1656.68 loss) I1014 05:58:06.013098 20911 solver.cpp:244] Train net output #5: fuse_loss = 1710.21 ( 1 = 1710.21 loss) I1014 05:58:06.013103 20911 sgd_solver.cpp:106] Iteration 16340, lr = 1e-07 I1014 05:58:30.647099 20911 solver.cpp:228] Iteration 16360, loss = 10772.4 I1014 05:58:30.647125 20911 solver.cpp:244] Train net output #0: dsn1_loss = 2.13976 ( 1 = 2.13976 loss) I1014 05:58:30.647132 20911 solver.cpp:244] Train net output #1: dsn2_loss = 1.28984 ( 1 = 1.28984 loss) I1014 05:58:30.647137 20911 solver.cpp:244] Train net output #2: dsn3_loss = 1.65222 ( 1 = 1.65222 loss) I1014 05:58:30.647142 20911 solver.cpp:244] Train net output #3: dsn4_loss = 1.99033 ( 1 = 1.99033 loss) I1014 05:58:30.647146 20911 solver.cpp:244] Train net output #4: dsn5_loss = 1.60033 ( 1 = 1.60033 loss) I1014 05:58:30.647150 20911 solver.cpp:244] Train net output #5: fuse_loss = 1.48198 ( 1 = 1.48198 loss) I1014 05:58:30.647152 20911 sgd_solver.cpp:106] Iteration 16360, lr = 1e-07 I1014 05:58:55.182524 20911 solver.cpp:228] Iteration 16380, loss = 10577.2 I1014 05:58:55.182552 20911 solver.cpp:244] Train net output #0: dsn1_loss = 317.736 ( 1 = 317.736 loss) I1014 05:58:55.182559 20911 solver.cpp:244] Train net output #1: dsn2_loss = 198.983 ( 1 = 198.983 loss) I1014 05:58:55.182565 20911 solver.cpp:244] Train net output #2: dsn3_loss = 128.888 ( 1 = 128.888 loss) I1014 05:58:55.182570 20911 solver.cpp:244] Train net output #3: dsn4_loss = 111.455 ( 1 = 111.455 loss) I1014 05:58:55.182574 20911 solver.cpp:244] Train net output #4: dsn5_loss = 106.67 ( 1 = 106.67 loss) I1014 05:58:55.182579 20911 solver.cpp:244] Train net output #5: fuse_loss = 95.5867 ( 1 = 95.5867 loss) I1014 05:58:55.182582 20911 sgd_solver.cpp:106] Iteration 16380, lr = 1e-07 I1014 05:59:19.169096 20911 solver.cpp:228] Iteration 16400, loss = 10652.7 I1014 05:59:19.169126 20911 solver.cpp:244] Train net output #0: dsn1_loss = 5216.55 ( 1 = 5216.55 loss) I1014 05:59:19.169131 20911 solver.cpp:244] Train net output #1: dsn2_loss = 4685.56 ( 1 = 4685.56 loss) I1014 05:59:19.169137 20911 solver.cpp:244] Train net output #2: dsn3_loss = 3455.37 ( 1 = 3455.37 loss) I1014 05:59:19.169142 20911 solver.cpp:244] Train net output #3: dsn4_loss = 2688.21 ( 1 = 2688.21 loss) I1014 05:59:19.169147 20911 solver.cpp:244] Train net output #4: dsn5_loss = 2245.43 ( 1 = 2245.43 loss) I1014 05:59:19.169152 20911 solver.cpp:244] Train net output #5: fuse_loss = 2516.61 ( 1 = 2516.61 loss) I1014 05:59:19.169155 20911 sgd_solver.cpp:106] Iteration 16400, lr = 1e-07 I1014 05:59:44.069021 20911 solver.cpp:228] Iteration 16420, loss = 10618.9 I1014 05:59:44.069056 20911 solver.cpp:244] Train net output #0: dsn1_loss = 5725.64 ( 1 = 5725.64 loss) I1014 05:59:44.069062 20911 solver.cpp:244] Train net output #1: dsn2_loss = 4988.75 ( 1 = 4988.75 loss) I1014 05:59:44.069067 20911 solver.cpp:244] Train net output #2: dsn3_loss = 4192.96 ( 1 = 4192.96 loss) I1014 05:59:44.069073 20911 solver.cpp:244] Train net output #3: dsn4_loss = 4053.23 ( 1 = 4053.23 loss) I1014 05:59:44.069077 20911 solver.cpp:244] Train net output #4: dsn5_loss = 3501.44 ( 1 = 3501.44 loss) I1014 05:59:44.069080 20911 solver.cpp:244] Train net output #5: fuse_loss = 3573.09 ( 1 = 3573.09 loss) I1014 05:59:44.069084 20911 sgd_solver.cpp:106] Iteration 16420, lr = 1e-07 I1014 06:00:09.370942 20911 solver.cpp:228] Iteration 16440, loss = 10502.7 I1014 06:00:09.370976 20911 solver.cpp:244] Train net output #0: dsn1_loss = 6377.67 ( 1 = 6377.67 loss) I1014 06:00:09.370983 20911 solver.cpp:244] Train net output #1: dsn2_loss = 5692.62 ( 1 = 5692.62 loss) I1014 06:00:09.370990 20911 solver.cpp:244] Train net output #2: dsn3_loss = 6057.85 ( 1 = 6057.85 loss) I1014 06:00:09.370995 20911 solver.cpp:244] Train net output #3: dsn4_loss = 5242.75 ( 1 = 5242.75 loss) I1014 06:00:09.370998 20911 solver.cpp:244] Train net output #4: dsn5_loss = 4816.72 ( 1 = 4816.72 loss) I1014 06:00:09.371002 20911 solver.cpp:244] Train net output #5: fuse_loss = 4720.86 ( 1 = 4720.86 loss) I1014 06:00:09.371006 20911 sgd_solver.cpp:106] Iteration 16440, lr = 1e-07 I1014 06:00:35.305788 20911 solver.cpp:228] Iteration 16460, loss = 10140.6 I1014 06:00:35.305820 20911 solver.cpp:244] Train net output #0: dsn1_loss = 185.226 ( 1 = 185.226 loss) I1014 06:00:35.305827 20911 solver.cpp:244] Train net output #1: dsn2_loss = 147.872 ( 1 = 147.872 loss) I1014 06:00:35.305835 20911 solver.cpp:244] Train net output #2: dsn3_loss = 120.204 ( 1 = 120.204 loss) I1014 06:00:35.305837 20911 solver.cpp:244] Train net output #3: dsn4_loss = 76.9019 ( 1 = 76.9019 loss) I1014 06:00:35.305841 20911 solver.cpp:244] Train net output #4: dsn5_loss = 51.619 ( 1 = 51.619 loss) I1014 06:00:35.305845 20911 solver.cpp:244] Train net output #5: fuse_loss = 79.8798 ( 1 = 79.8798 loss) I1014 06:00:35.305850 20911 sgd_solver.cpp:106] Iteration 16460, lr = 1e-07 I1014 06:01:00.978884 20911 solver.cpp:228] Iteration 16480, loss = 9875.51 I1014 06:01:00.978911 20911 solver.cpp:244] Train net output #0: dsn1_loss = 471.941 ( 1 = 471.941 loss) I1014 06:01:00.978919 20911 solver.cpp:244] Train net output #1: dsn2_loss = 221.971 ( 1 = 221.971 loss) I1014 06:01:00.978924 20911 solver.cpp:244] Train net output #2: dsn3_loss = 151.338 ( 1 = 151.338 loss) I1014 06:01:00.978929 20911 solver.cpp:244] Train net output #3: dsn4_loss = 94.5365 ( 1 = 94.5365 loss) I1014 06:01:00.978933 20911 solver.cpp:244] Train net output #4: dsn5_loss = 44.9636 ( 1 = 44.9636 loss) I1014 06:01:00.978937 20911 solver.cpp:244] Train net output #5: fuse_loss = 69.8908 ( 1 = 69.8908 loss) I1014 06:01:00.978941 20911 sgd_solver.cpp:106] Iteration 16480, lr = 1e-07 I1014 06:01:27.849673 20911 solver.cpp:228] Iteration 16500, loss = 10291.5 I1014 06:01:27.849701 20911 solver.cpp:244] Train net output #0: dsn1_loss = 0.704547 ( 1 = 0.704547 loss) I1014 06:01:27.849709 20911 solver.cpp:244] Train net output #1: dsn2_loss = 0.583943 ( 1 = 0.583943 loss) I1014 06:01:27.849714 20911 solver.cpp:244] Train net output #2: dsn3_loss = 0.471432 ( 1 = 0.471432 loss) I1014 06:01:27.849719 20911 solver.cpp:244] Train net output #3: dsn4_loss = 0.873354 ( 1 = 0.873354 loss) I1014 06:01:27.849721 20911 solver.cpp:244] Train net output #4: dsn5_loss = 0.716333 ( 1 = 0.716333 loss) I1014 06:01:27.849725 20911 solver.cpp:244] Train net output #5: fuse_loss = 0.532933 ( 1 = 0.532933 loss) I1014 06:01:27.849728 20911 sgd_solver.cpp:106] Iteration 16500, lr = 1e-07 I1014 06:01:52.623425 20911 solver.cpp:228] Iteration 16520, loss = 10854.3 I1014 06:01:52.623451 20911 solver.cpp:244] Train net output #0: dsn1_loss = 3.65184 ( 1 = 3.65184 loss) I1014 06:01:52.623458 20911 solver.cpp:244] Train net output #1: dsn2_loss = 2.68432 ( 1 = 2.68432 loss) I1014 06:01:52.623462 20911 solver.cpp:244] Train net output #2: dsn3_loss = 2.37431 ( 1 = 2.37431 loss) I1014 06:01:52.623468 20911 solver.cpp:244] Train net output #3: dsn4_loss = 2.04626 ( 1 = 2.04626 loss) I1014 06:01:52.623472 20911 solver.cpp:244] Train net output #4: dsn5_loss = 2.6021 ( 1 = 2.6021 loss) I1014 06:01:52.623476 20911 solver.cpp:244] Train net output #5: fuse_loss = 1.89754 ( 1 = 1.89754 loss) I1014 06:01:52.623479 20911 sgd_solver.cpp:106] Iteration 16520, lr = 1e-07 I1014 06:02:18.968715 20911 solver.cpp:228] Iteration 16540, loss = 10576.9 I1014 06:02:18.968746 20911 solver.cpp:244] Train net output #0: dsn1_loss = 4467.03 ( 1 = 4467.03 loss) I1014 06:02:18.968752 20911 solver.cpp:244] Train net output #1: dsn2_loss = 3921.37 ( 1 = 3921.37 loss) I1014 06:02:18.968758 20911 solver.cpp:244] Train net output #2: dsn3_loss = 3599.78 ( 1 = 3599.78 loss) I1014 06:02:18.968765 20911 solver.cpp:244] Train net output #3: dsn4_loss = 3398.6 ( 1 = 3398.6 loss) I1014 06:02:18.968770 20911 solver.cpp:244] Train net output #4: dsn5_loss = 3075.47 ( 1 = 3075.47 loss) I1014 06:02:18.968773 20911 solver.cpp:244] Train net output #5: fuse_loss = 3074.04 ( 1 = 3074.04 loss) I1014 06:02:18.968777 20911 sgd_solver.cpp:106] Iteration 16540, lr = 1e-07 I1014 06:02:43.028919 20911 solver.cpp:228] Iteration 16560, loss = 9770.52 I1014 06:02:43.028952 20911 solver.cpp:244] Train net output #0: dsn1_loss = 10482.9 ( 1 = 10482.9 loss) I1014 06:02:43.028959 20911 solver.cpp:244] Train net output #1: dsn2_loss = 9039.25 ( 1 = 9039.25 loss) I1014 06:02:43.028965 20911 solver.cpp:244] Train net output #2: dsn3_loss = 6044.56 ( 1 = 6044.56 loss) I1014 06:02:43.028970 20911 solver.cpp:244] Train net output #3: dsn4_loss = 5637.78 ( 1 = 5637.78 loss) I1014 06:02:43.028975 20911 solver.cpp:244] Train net output #4: dsn5_loss = 5502.65 ( 1 = 5502.65 loss) I1014 06:02:43.028978 20911 solver.cpp:244] Train net output #5: fuse_loss = 5413.52 ( 1 = 5413.52 loss) I1014 06:02:43.028982 20911 sgd_solver.cpp:106] Iteration 16560, lr = 1e-07 I1014 06:03:08.073918 20911 solver.cpp:228] Iteration 16580, loss = 10575 I1014 06:03:08.073952 20911 solver.cpp:244] Train net output #0: dsn1_loss = 344.559 ( 1 = 344.559 loss) I1014 06:03:08.073957 20911 solver.cpp:244] Train net output #1: dsn2_loss = 209.284 ( 1 = 209.284 loss) I1014 06:03:08.073961 20911 solver.cpp:244] Train net output #2: dsn3_loss = 88.3105 ( 1 = 88.3105 loss) I1014 06:03:08.073964 20911 solver.cpp:244] Train net output #3: dsn4_loss = 60.5969 ( 1 = 60.5969 loss) I1014 06:03:08.073968 20911 solver.cpp:244] Train net output #4: dsn5_loss = 47.0126 ( 1 = 47.0126 loss) I1014 06:03:08.073972 20911 solver.cpp:244] Train net output #5: fuse_loss = 57.4098 ( 1 = 57.4098 loss) I1014 06:03:08.073976 20911 sgd_solver.cpp:106] Iteration 16580, lr = 1e-07 I1014 06:03:32.652819 20911 solver.cpp:228] Iteration 16600, loss = 10646 I1014 06:03:32.652849 20911 solver.cpp:244] Train net output #0: dsn1_loss = 4298.23 ( 1 = 4298.23 loss) I1014 06:03:32.652854 20911 solver.cpp:244] Train net output #1: dsn2_loss = 3775.03 ( 1 = 3775.03 loss) I1014 06:03:32.652858 20911 solver.cpp:244] Train net output #2: dsn3_loss = 4272.06 ( 1 = 4272.06 loss) I1014 06:03:32.652861 20911 solver.cpp:244] Train net output #3: dsn4_loss = 4429.7 ( 1 = 4429.7 loss) I1014 06:03:32.652865 20911 solver.cpp:244] Train net output #4: dsn5_loss = 4283.1 ( 1 = 4283.1 loss) I1014 06:03:32.652869 20911 solver.cpp:244] Train net output #5: fuse_loss = 3756.7 ( 1 = 3756.7 loss) I1014 06:03:32.652873 20911 sgd_solver.cpp:106] Iteration 16600, lr = 1e-07 I1014 06:03:57.742897 20911 solver.cpp:228] Iteration 16620, loss = 10440.4 I1014 06:03:57.742928 20911 solver.cpp:244] Train net output #0: dsn1_loss = 6712.89 ( 1 = 6712.89 loss) I1014 06:03:57.742934 20911 solver.cpp:244] Train net output #1: dsn2_loss = 5655.7 ( 1 = 5655.7 loss) I1014 06:03:57.742938 20911 solver.cpp:244] Train net output #2: dsn3_loss = 4487.52 ( 1 = 4487.52 loss) I1014 06:03:57.742941 20911 solver.cpp:244] Train net output #3: dsn4_loss = 4051.05 ( 1 = 4051.05 loss) I1014 06:03:57.742945 20911 solver.cpp:244] Train net output #4: dsn5_loss = 4434.61 ( 1 = 4434.61 loss) I1014 06:03:57.742949 20911 solver.cpp:244] Train net output #5: fuse_loss = 3433.48 ( 1 = 3433.48 loss) I1014 06:03:57.742952 20911 sgd_solver.cpp:106] Iteration 16620, lr = 1e-07 I1014 06:04:24.400725 20911 solver.cpp:228] Iteration 16640, loss = 10245.4 I1014 06:04:24.400758 20911 solver.cpp:244] Train net output #0: dsn1_loss = 3501.49 ( 1 = 3501.49 loss) I1014 06:04:24.400763 20911 solver.cpp:244] Train net output #1: dsn2_loss = 2811.84 ( 1 = 2811.84 loss) I1014 06:04:24.400766 20911 solver.cpp:244] Train net output #2: dsn3_loss = 2083.51 ( 1 = 2083.51 loss) I1014 06:04:24.400770 20911 solver.cpp:244] Train net output #3: dsn4_loss = 1611.37 ( 1 = 1611.37 loss) I1014 06:04:24.400774 20911 solver.cpp:244] Train net output #4: dsn5_loss = 1625.15 ( 1 = 1625.15 loss) I1014 06:04:24.400779 20911 solver.cpp:244] Train net output #5: fuse_loss = 1567.22 ( 1 = 1567.22 loss) I1014 06:04:24.400781 20911 sgd_solver.cpp:106] Iteration 16640, lr = 1e-07 I1014 06:04:50.150880 20911 solver.cpp:228] Iteration 16660, loss = 10885.9 I1014 06:04:50.150912 20911 solver.cpp:244] Train net output #0: dsn1_loss = 363.659 ( 1 = 363.659 loss) I1014 06:04:50.150918 20911 solver.cpp:244] Train net output #1: dsn2_loss = 232.315 ( 1 = 232.315 loss) I1014 06:04:50.150920 20911 solver.cpp:244] Train net output #2: dsn3_loss = 111.952 ( 1 = 111.952 loss) I1014 06:04:50.150924 20911 solver.cpp:244] Train net output #3: dsn4_loss = 98.9181 ( 1 = 98.9181 loss) I1014 06:04:50.150928 20911 solver.cpp:244] Train net output #4: dsn5_loss = 85.2289 ( 1 = 85.2289 loss) I1014 06:04:50.150931 20911 solver.cpp:244] Train net output #5: fuse_loss = 88.5579 ( 1 = 88.5579 loss) I1014 06:04:50.150936 20911 sgd_solver.cpp:106] Iteration 16660, lr = 1e-07 I1014 06:05:14.700987 20911 solver.cpp:228] Iteration 16680, loss = 11047.6 I1014 06:05:14.701014 20911 solver.cpp:244] Train net output #0: dsn1_loss = 0 ( 1 = 0 loss) I1014 06:05:14.701017 20911 solver.cpp:244] Train net output #1: dsn2_loss = 0 ( 1 = 0 loss) I1014 06:05:14.701021 20911 solver.cpp:244] Train net output #2: dsn3_loss = 0 ( 1 = 0 loss) I1014 06:05:14.701023 20911 solver.cpp:244] Train net output #3: dsn4_loss = 0 ( 1 = 0 loss) I1014 06:05:14.701026 20911 solver.cpp:244] Train net output #4: dsn5_loss = 0 ( 1 = 0 loss) I1014 06:05:14.701030 20911 solver.cpp:244] Train net output #5: fuse_loss = 0 ( 1 = 0 loss) I1014 06:05:14.701032 20911 sgd_solver.cpp:106] Iteration 16680, lr = 1e-07 I1014 06:05:39.575654 20911 solver.cpp:228] Iteration 16700, loss = 10774.9 I1014 06:05:39.575680 20911 solver.cpp:244] Train net output #0: dsn1_loss = 13.5047 ( 1 = 13.5047 loss) I1014 06:05:39.575685 20911 solver.cpp:244] Train net output #1: dsn2_loss = 11.7523 ( 1 = 11.7523 loss) I1014 06:05:39.575688 20911 solver.cpp:244] Train net output #2: dsn3_loss = 10.7108 ( 1 = 10.7108 loss) I1014 06:05:39.575691 20911 solver.cpp:244] Train net output #3: dsn4_loss = 11.5296 ( 1 = 11.5296 loss) I1014 06:05:39.575695 20911 solver.cpp:244] Train net output #4: dsn5_loss = 11.958 ( 1 = 11.958 loss) I1014 06:05:39.575698 20911 solver.cpp:244] Train net output #5: fuse_loss = 10.4322 ( 1 = 10.4322 loss) I1014 06:05:39.575702 20911 sgd_solver.cpp:106] Iteration 16700, lr = 1e-07 I1014 06:06:04.852396 20911 solver.cpp:228] Iteration 16720, loss = 10711.8 I1014 06:06:04.852423 20911 solver.cpp:244] Train net output #0: dsn1_loss = 7621.26 ( 1 = 7621.26 loss) I1014 06:06:04.852428 20911 solver.cpp:244] Train net output #1: dsn2_loss = 6808.27 ( 1 = 6808.27 loss) I1014 06:06:04.852432 20911 solver.cpp:244] Train net output #2: dsn3_loss = 6555.27 ( 1 = 6555.27 loss) I1014 06:06:04.852435 20911 solver.cpp:244] Train net output #3: dsn4_loss = 7347.2 ( 1 = 7347.2 loss) I1014 06:06:04.852439 20911 solver.cpp:244] Train net output #4: dsn5_loss = 7011.59 ( 1 = 7011.59 loss) I1014 06:06:04.852443 20911 solver.cpp:244] Train net output #5: fuse_loss = 6145.36 ( 1 = 6145.36 loss) I1014 06:06:04.852447 20911 sgd_solver.cpp:106] Iteration 16720, lr = 1e-07 I1014 06:06:29.752275 20911 solver.cpp:228] Iteration 16740, loss = 10564.3 I1014 06:06:29.752305 20911 solver.cpp:244] Train net output #0: dsn1_loss = 2433.66 ( 1 = 2433.66 loss) I1014 06:06:29.752310 20911 solver.cpp:244] Train net output #1: dsn2_loss = 1752.48 ( 1 = 1752.48 loss) I1014 06:06:29.752315 20911 solver.cpp:244] Train net output #2: dsn3_loss = 1488.18 ( 1 = 1488.18 loss) I1014 06:06:29.752318 20911 solver.cpp:244] Train net output #3: dsn4_loss = 1467.87 ( 1 = 1467.87 loss) I1014 06:06:29.752322 20911 solver.cpp:244] Train net output #4: dsn5_loss = 1214.56 ( 1 = 1214.56 loss) I1014 06:06:29.752326 20911 solver.cpp:244] Train net output #5: fuse_loss = 1303.24 ( 1 = 1303.24 loss) I1014 06:06:29.752329 20911 sgd_solver.cpp:106] Iteration 16740, lr = 1e-07 I1014 06:06:56.161664 20911 solver.cpp:228] Iteration 16760, loss = 10883.7 I1014 06:06:56.161695 20911 solver.cpp:244] Train net output #0: dsn1_loss = 4411.2 ( 1 = 4411.2 loss) I1014 06:06:56.161700 20911 solver.cpp:244] Train net output #1: dsn2_loss = 3260.3 ( 1 = 3260.3 loss) I1014 06:06:56.161703 20911 solver.cpp:244] Train net output #2: dsn3_loss = 2961.88 ( 1 = 2961.88 loss) I1014 06:06:56.161707 20911 solver.cpp:244] Train net output #3: dsn4_loss = 3461.15 ( 1 = 3461.15 loss) I1014 06:06:56.161711 20911 solver.cpp:244] Train net output #4: dsn5_loss = 3303.33 ( 1 = 3303.33 loss) I1014 06:06:56.161715 20911 solver.cpp:244] Train net output #5: fuse_loss = 2768.45 ( 1 = 2768.45 loss) I1014 06:06:56.161720 20911 sgd_solver.cpp:106] Iteration 16760, lr = 1e-07 I1014 06:07:19.926509 20911 solver.cpp:228] Iteration 16780, loss = 9930.63 I1014 06:07:19.926540 20911 solver.cpp:244] Train net output #0: dsn1_loss = 5666.68 ( 1 = 5666.68 loss) I1014 06:07:19.926545 20911 solver.cpp:244] Train net output #1: dsn2_loss = 4639.6 ( 1 = 4639.6 loss) I1014 06:07:19.926549 20911 solver.cpp:244] Train net output #2: dsn3_loss = 4163.93 ( 1 = 4163.93 loss) I1014 06:07:19.926553 20911 solver.cpp:244] Train net output #3: dsn4_loss = 4075.06 ( 1 = 4075.06 loss) I1014 06:07:19.926558 20911 solver.cpp:244] Train net output #4: dsn5_loss = 4024.22 ( 1 = 4024.22 loss) I1014 06:07:19.926561 20911 solver.cpp:244] Train net output #5: fuse_loss = 3487.13 ( 1 = 3487.13 loss) I1014 06:07:19.926564 20911 sgd_solver.cpp:106] Iteration 16780, lr = 1e-07 I1014 06:07:44.883677 20911 solver.cpp:228] Iteration 16800, loss = 9758.13 I1014 06:07:44.883708 20911 solver.cpp:244] Train net output #0: dsn1_loss = 1022.93 ( 1 = 1022.93 loss) I1014 06:07:44.883713 20911 solver.cpp:244] Train net output #1: dsn2_loss = 917.26 ( 1 = 917.26 loss) I1014 06:07:44.883716 20911 solver.cpp:244] Train net output #2: dsn3_loss = 638.128 ( 1 = 638.128 loss) I1014 06:07:44.883719 20911 solver.cpp:244] Train net output #3: dsn4_loss = 510.632 ( 1 = 510.632 loss) I1014 06:07:44.883723 20911 solver.cpp:244] Train net output #4: dsn5_loss = 328.996 ( 1 = 328.996 loss) I1014 06:07:44.883726 20911 solver.cpp:244] Train net output #5: fuse_loss = 431.554 ( 1 = 431.554 loss) I1014 06:07:44.883730 20911 sgd_solver.cpp:106] Iteration 16800, lr = 1e-07 I1014 06:08:11.177716 20911 solver.cpp:228] Iteration 16820, loss = 10044.1 I1014 06:08:11.177747 20911 solver.cpp:244] Train net output #0: dsn1_loss = 4511.34 ( 1 = 4511.34 loss) I1014 06:08:11.177750 20911 solver.cpp:244] Train net output #1: dsn2_loss = 4004.11 ( 1 = 4004.11 loss) I1014 06:08:11.177754 20911 solver.cpp:244] Train net output #2: dsn3_loss = 3833.13 ( 1 = 3833.13 loss) I1014 06:08:11.177758 20911 solver.cpp:244] Train net output #3: dsn4_loss = 3712.67 ( 1 = 3712.67 loss) I1014 06:08:11.177762 20911 solver.cpp:244] Train net output #4: dsn5_loss = 3424.74 ( 1 = 3424.74 loss) I1014 06:08:11.177767 20911 solver.cpp:244] Train net output #5: fuse_loss = 3292.68 ( 1 = 3292.68 loss) I1014 06:08:11.177770 20911 sgd_solver.cpp:106] Iteration 16820, lr = 1e-07 I1014 06:08:37.262506 20911 solver.cpp:228] Iteration 16840, loss = 11141.8 I1014 06:08:37.262537 20911 solver.cpp:244] Train net output #0: dsn1_loss = 163.727 ( 1 = 163.727 loss) I1014 06:08:37.262542 20911 solver.cpp:244] Train net output #1: dsn2_loss = 123.233 ( 1 = 123.233 loss) I1014 06:08:37.262547 20911 solver.cpp:244] Train net output #2: dsn3_loss = 72.6476 ( 1 = 72.6476 loss) I1014 06:08:37.262549 20911 solver.cpp:244] Train net output #3: dsn4_loss = 53.5755 ( 1 = 53.5755 loss) I1014 06:08:37.262553 20911 solver.cpp:244] Train net output #4: dsn5_loss = 47.8599 ( 1 = 47.8599 loss) I1014 06:08:37.262557 20911 solver.cpp:244] Train net output #5: fuse_loss = 44.6252 ( 1 = 44.6252 loss) I1014 06:08:37.262560 20911 sgd_solver.cpp:106] Iteration 16840, lr = 1e-07 I1014 06:09:00.949702 20911 solver.cpp:228] Iteration 16860, loss = 10530.6 I1014 06:09:00.949731 20911 solver.cpp:244] Train net output #0: dsn1_loss = 1844.71 ( 1 = 1844.71 loss) I1014 06:09:00.949736 20911 solver.cpp:244] Train net output #1: dsn2_loss = 1370.99 ( 1 = 1370.99 loss) I1014 06:09:00.949740 20911 solver.cpp:244] Train net output #2: dsn3_loss = 960.622 ( 1 = 960.622 loss) I1014 06:09:00.949743 20911 solver.cpp:244] Train net output #3: dsn4_loss = 717.646 ( 1 = 717.646 loss) I1014 06:09:00.949748 20911 solver.cpp:244] Train net output #4: dsn5_loss = 659.55 ( 1 = 659.55 loss) I1014 06:09:00.949751 20911 solver.cpp:244] Train net output #5: fuse_loss = 668.594 ( 1 = 668.594 loss) I1014 06:09:00.949756 20911 sgd_solver.cpp:106] Iteration 16860, lr = 1e-07 I1014 06:09:25.450436 20911 solver.cpp:228] Iteration 16880, loss = 9730.32 I1014 06:09:25.450464 20911 solver.cpp:244] Train net output #0: dsn1_loss = 18.4312 ( 1 = 18.4312 loss) I1014 06:09:25.450469 20911 solver.cpp:244] Train net output #1: dsn2_loss = 13.4137 ( 1 = 13.4137 loss) I1014 06:09:25.450472 20911 solver.cpp:244] Train net output #2: dsn3_loss = 11.4722 ( 1 = 11.4722 loss) I1014 06:09:25.450476 20911 solver.cpp:244] Train net output #3: dsn4_loss = 13.6246 ( 1 = 13.6246 loss) I1014 06:09:25.450479 20911 solver.cpp:244] Train net output #4: dsn5_loss = 15.0869 ( 1 = 15.0869 loss) I1014 06:09:25.450484 20911 solver.cpp:244] Train net output #5: fuse_loss = 11.0057 ( 1 = 11.0057 loss) I1014 06:09:25.450486 20911 sgd_solver.cpp:106] Iteration 16880, lr = 1e-07 I1014 06:09:50.830574 20911 solver.cpp:228] Iteration 16900, loss = 10431 I1014 06:09:50.830601 20911 solver.cpp:244] Train net output #0: dsn1_loss = 23.9899 ( 1 = 23.9899 loss) I1014 06:09:50.830606 20911 solver.cpp:244] Train net output #1: dsn2_loss = 17.5701 ( 1 = 17.5701 loss) I1014 06:09:50.830610 20911 solver.cpp:244] Train net output #2: dsn3_loss = 13.5739 ( 1 = 13.5739 loss) I1014 06:09:50.830615 20911 solver.cpp:244] Train net output #3: dsn4_loss = 14.2346 ( 1 = 14.2346 loss) I1014 06:09:50.830618 20911 solver.cpp:244] Train net output #4: dsn5_loss = 14.3502 ( 1 = 14.3502 loss) I1014 06:09:50.830621 20911 solver.cpp:244] Train net output #5: fuse_loss = 13.2513 ( 1 = 13.2513 loss) I1014 06:09:50.830626 20911 sgd_solver.cpp:106] Iteration 16900, lr = 1e-07 I1014 06:10:16.193476 20911 solver.cpp:228] Iteration 16920, loss = 10451.4 I1014 06:10:16.193506 20911 solver.cpp:244] Train net output #0: dsn1_loss = 383.978 ( 1 = 383.978 loss) I1014 06:10:16.193509 20911 solver.cpp:244] Train net output #1: dsn2_loss = 275.098 ( 1 = 275.098 loss) I1014 06:10:16.193513 20911 solver.cpp:244] Train net output #2: dsn3_loss = 159.046 ( 1 = 159.046 loss) I1014 06:10:16.193517 20911 solver.cpp:244] Train net output #3: dsn4_loss = 133.988 ( 1 = 133.988 loss) I1014 06:10:16.193521 20911 solver.cpp:244] Train net output #4: dsn5_loss = 129.31 ( 1 = 129.31 loss) I1014 06:10:16.193526 20911 solver.cpp:244] Train net output #5: fuse_loss = 129.028 ( 1 = 129.028 loss) I1014 06:10:16.193528 20911 sgd_solver.cpp:106] Iteration 16920, lr = 1e-07 I1014 06:10:42.145558 20911 solver.cpp:228] Iteration 16940, loss = 9881.25 I1014 06:10:42.145583 20911 solver.cpp:244] Train net output #0: dsn1_loss = 5841.87 ( 1 = 5841.87 loss) I1014 06:10:42.145588 20911 solver.cpp:244] Train net output #1: dsn2_loss = 4915.6 ( 1 = 4915.6 loss) I1014 06:10:42.145592 20911 solver.cpp:244] Train net output #2: dsn3_loss = 4510.82 ( 1 = 4510.82 loss) I1014 06:10:42.145596 20911 solver.cpp:244] Train net output #3: dsn4_loss = 4421.82 ( 1 = 4421.82 loss) I1014 06:10:42.145601 20911 solver.cpp:244] Train net output #4: dsn5_loss = 3873 ( 1 = 3873 loss) I1014 06:10:42.145603 20911 solver.cpp:244] Train net output #5: fuse_loss = 3707.69 ( 1 = 3707.69 loss) I1014 06:10:42.145607 20911 sgd_solver.cpp:106] Iteration 16940, lr = 1e-07 I1014 06:11:07.121354 20911 solver.cpp:228] Iteration 16960, loss = 10679.8 I1014 06:11:07.121382 20911 solver.cpp:244] Train net output #0: dsn1_loss = 7132.6 ( 1 = 7132.6 loss) I1014 06:11:07.121387 20911 solver.cpp:244] Train net output #1: dsn2_loss = 6211.23 ( 1 = 6211.23 loss) I1014 06:11:07.121392 20911 solver.cpp:244] Train net output #2: dsn3_loss = 5472.64 ( 1 = 5472.64 loss) I1014 06:11:07.121395 20911 solver.cpp:244] Train net output #3: dsn4_loss = 5049.93 ( 1 = 5049.93 loss) I1014 06:11:07.121399 20911 solver.cpp:244] Train net output #4: dsn5_loss = 4560.74 ( 1 = 4560.74 loss) I1014 06:11:07.121403 20911 solver.cpp:244] Train net output #5: fuse_loss = 4274.9 ( 1 = 4274.9 loss) I1014 06:11:07.121407 20911 sgd_solver.cpp:106] Iteration 16960, lr = 1e-07 I1014 06:11:31.819911 20911 solver.cpp:228] Iteration 16980, loss = 10833.1 I1014 06:11:31.819936 20911 solver.cpp:244] Train net output #0: dsn1_loss = 13.7353 ( 1 = 13.7353 loss) I1014 06:11:31.819941 20911 solver.cpp:244] Train net output #1: dsn2_loss = 9.93422 ( 1 = 9.93422 loss) I1014 06:11:31.819943 20911 solver.cpp:244] Train net output #2: dsn3_loss = 9.55829 ( 1 = 9.55829 loss) I1014 06:11:31.819947 20911 solver.cpp:244] Train net output #3: dsn4_loss = 11.1488 ( 1 = 11.1488 loss) I1014 06:11:31.819950 20911 solver.cpp:244] Train net output #4: dsn5_loss = 11.1214 ( 1 = 11.1214 loss) I1014 06:11:31.819954 20911 solver.cpp:244] Train net output #5: fuse_loss = 8.30104 ( 1 = 8.30104 loss) I1014 06:11:31.819957 20911 sgd_solver.cpp:106] Iteration 16980, lr = 1e-07 I1014 06:11:56.125711 20911 solver.cpp:228] Iteration 17000, loss = 11541.5 I1014 06:11:56.125746 20911 solver.cpp:244] Train net output #0: dsn1_loss = 2936.06 ( 1 = 2936.06 loss) I1014 06:11:56.125751 20911 solver.cpp:244] Train net output #1: dsn2_loss = 2364.41 ( 1 = 2364.41 loss) I1014 06:11:56.125754 20911 solver.cpp:244] Train net output #2: dsn3_loss = 1740.28 ( 1 = 1740.28 loss) I1014 06:11:56.125758 20911 solver.cpp:244] Train net output #3: dsn4_loss = 1515.34 ( 1 = 1515.34 loss) I1014 06:11:56.125761 20911 solver.cpp:244] Train net output #4: dsn5_loss = 1284.69 ( 1 = 1284.69 loss) I1014 06:11:56.125766 20911 solver.cpp:244] Train net output #5: fuse_loss = 1289.39 ( 1 = 1289.39 loss) I1014 06:11:56.125769 20911 sgd_solver.cpp:106] Iteration 17000, lr = 1e-07 I1014 06:12:21.165143 20911 solver.cpp:228] Iteration 17020, loss = 10817.6 I1014 06:12:21.165174 20911 solver.cpp:244] Train net output #0: dsn1_loss = 4406.86 ( 1 = 4406.86 loss) I1014 06:12:21.165179 20911 solver.cpp:244] Train net output #1: dsn2_loss = 3375.73 ( 1 = 3375.73 loss) I1014 06:12:21.165184 20911 solver.cpp:244] Train net output #2: dsn3_loss = 2658.02 ( 1 = 2658.02 loss) I1014 06:12:21.165187 20911 solver.cpp:244] Train net output #3: dsn4_loss = 2180.84 ( 1 = 2180.84 loss) I1014 06:12:21.165191 20911 solver.cpp:244] Train net output #4: dsn5_loss = 1861.38 ( 1 = 1861.38 loss) I1014 06:12:21.165194 20911 solver.cpp:244] Train net output #5: fuse_loss = 1964.38 ( 1 = 1964.38 loss) I1014 06:12:21.165199 20911 sgd_solver.cpp:106] Iteration 17020, lr = 1e-07 I1014 06:12:45.480537 20911 solver.cpp:228] Iteration 17040, loss = 10611.2 I1014 06:12:45.480562 20911 solver.cpp:244] Train net output #0: dsn1_loss = 21.1074 ( 1 = 21.1074 loss) I1014 06:12:45.480566 20911 solver.cpp:244] Train net output #1: dsn2_loss = 9.6985 ( 1 = 9.6985 loss) I1014 06:12:45.480571 20911 solver.cpp:244] Train net output #2: dsn3_loss = 8.91897 ( 1 = 8.91897 loss) I1014 06:12:45.480573 20911 solver.cpp:244] Train net output #3: dsn4_loss = 11.3183 ( 1 = 11.3183 loss) I1014 06:12:45.480577 20911 solver.cpp:244] Train net output #4: dsn5_loss = 11.6586 ( 1 = 11.6586 loss) I1014 06:12:45.480581 20911 solver.cpp:244] Train net output #5: fuse_loss = 9.04959 ( 1 = 9.04959 loss) I1014 06:12:45.480584 20911 sgd_solver.cpp:106] Iteration 17040, lr = 1e-07 I1014 06:13:11.198786 20911 solver.cpp:228] Iteration 17060, loss = 10086.1 I1014 06:13:11.198822 20911 solver.cpp:244] Train net output #0: dsn1_loss = 4348.74 ( 1 = 4348.74 loss) I1014 06:13:11.198827 20911 solver.cpp:244] Train net output #1: dsn2_loss = 3606.1 ( 1 = 3606.1 loss) I1014 06:13:11.198832 20911 solver.cpp:244] Train net output #2: dsn3_loss = 2771.27 ( 1 = 2771.27 loss) I1014 06:13:11.198834 20911 solver.cpp:244] Train net output #3: dsn4_loss = 2385.91 ( 1 = 2385.91 loss) I1014 06:13:11.198838 20911 solver.cpp:244] Train net output #4: dsn5_loss = 1882.03 ( 1 = 1882.03 loss) I1014 06:13:11.198843 20911 solver.cpp:244] Train net output #5: fuse_loss = 1989.48 ( 1 = 1989.48 loss) I1014 06:13:11.198846 20911 sgd_solver.cpp:106] Iteration 17060, lr = 1e-07 I1014 06:13:35.854650 20911 solver.cpp:228] Iteration 17080, loss = 10594.2 I1014 06:13:35.854681 20911 solver.cpp:244] Train net output #0: dsn1_loss = 9252.72 ( 1 = 9252.72 loss) I1014 06:13:35.854686 20911 solver.cpp:244] Train net output #1: dsn2_loss = 8538.11 ( 1 = 8538.11 loss) I1014 06:13:35.854691 20911 solver.cpp:244] Train net output #2: dsn3_loss = 6877.83 ( 1 = 6877.83 loss) I1014 06:13:35.854693 20911 solver.cpp:244] Train net output #3: dsn4_loss = 6162.19 ( 1 = 6162.19 loss) I1014 06:13:35.854697 20911 solver.cpp:244] Train net output #4: dsn5_loss = 5415.1 ( 1 = 5415.1 loss) I1014 06:13:35.854701 20911 solver.cpp:244] Train net output #5: fuse_loss = 5467.84 ( 1 = 5467.84 loss) I1014 06:13:35.854706 20911 sgd_solver.cpp:106] Iteration 17080, lr = 1e-07 I1014 06:14:01.481426 20911 solver.cpp:228] Iteration 17100, loss = 10633.6 I1014 06:14:01.481458 20911 solver.cpp:244] Train net output #0: dsn1_loss = 1661.67 ( 1 = 1661.67 loss) I1014 06:14:01.481463 20911 solver.cpp:244] Train net output #1: dsn2_loss = 951.367 ( 1 = 951.367 loss) I1014 06:14:01.481467 20911 solver.cpp:244] Train net output #2: dsn3_loss = 543.144 ( 1 = 543.144 loss) I1014 06:14:01.481472 20911 solver.cpp:244] Train net output #3: dsn4_loss = 562.564 ( 1 = 562.564 loss) I1014 06:14:01.481474 20911 solver.cpp:244] Train net output #4: dsn5_loss = 494.344 ( 1 = 494.344 loss) I1014 06:14:01.481478 20911 solver.cpp:244] Train net output #5: fuse_loss = 513.29 ( 1 = 513.29 loss) I1014 06:14:01.481482 20911 sgd_solver.cpp:106] Iteration 17100, lr = 1e-07 I1014 06:14:26.491955 20911 solver.cpp:228] Iteration 17120, loss = 10367.3 I1014 06:14:26.491986 20911 solver.cpp:244] Train net output #0: dsn1_loss = 3151.43 ( 1 = 3151.43 loss) I1014 06:14:26.491991 20911 solver.cpp:244] Train net output #1: dsn2_loss = 2422.13 ( 1 = 2422.13 loss) I1014 06:14:26.491994 20911 solver.cpp:244] Train net output #2: dsn3_loss = 2068.93 ( 1 = 2068.93 loss) I1014 06:14:26.491998 20911 solver.cpp:244] Train net output #3: dsn4_loss = 1258.07 ( 1 = 1258.07 loss) I1014 06:14:26.492002 20911 solver.cpp:244] Train net output #4: dsn5_loss = 1068.45 ( 1 = 1068.45 loss) I1014 06:14:26.492007 20911 solver.cpp:244] Train net output #5: fuse_loss = 1310.25 ( 1 = 1310.25 loss) I1014 06:14:26.492010 20911 sgd_solver.cpp:106] Iteration 17120, lr = 1e-07 I1014 06:14:51.140960 20911 solver.cpp:228] Iteration 17140, loss = 9513.02 I1014 06:14:51.140990 20911 solver.cpp:244] Train net output #0: dsn1_loss = 1531.96 ( 1 = 1531.96 loss) I1014 06:14:51.140995 20911 solver.cpp:244] Train net output #1: dsn2_loss = 1068.95 ( 1 = 1068.95 loss) I1014 06:14:51.140998 20911 solver.cpp:244] Train net output #2: dsn3_loss = 913.459 ( 1 = 913.459 loss) I1014 06:14:51.141002 20911 solver.cpp:244] Train net output #3: dsn4_loss = 303.454 ( 1 = 303.454 loss) I1014 06:14:51.141006 20911 solver.cpp:244] Train net output #4: dsn5_loss = 225.743 ( 1 = 225.743 loss) I1014 06:14:51.141010 20911 solver.cpp:244] Train net output #5: fuse_loss = 323.937 ( 1 = 323.937 loss) I1014 06:14:51.141013 20911 sgd_solver.cpp:106] Iteration 17140, lr = 1e-07 I1014 06:15:17.465776 20911 solver.cpp:228] Iteration 17160, loss = 10552.1 I1014 06:15:17.465811 20911 solver.cpp:244] Train net output #0: dsn1_loss = 587.009 ( 1 = 587.009 loss) I1014 06:15:17.465814 20911 solver.cpp:244] Train net output #1: dsn2_loss = 435.175 ( 1 = 435.175 loss) I1014 06:15:17.465818 20911 solver.cpp:244] Train net output #2: dsn3_loss = 278.661 ( 1 = 278.661 loss) I1014 06:15:17.465822 20911 solver.cpp:244] Train net output #3: dsn4_loss = 219.945 ( 1 = 219.945 loss) I1014 06:15:17.465826 20911 solver.cpp:244] Train net output #4: dsn5_loss = 205.938 ( 1 = 205.938 loss) I1014 06:15:17.465829 20911 solver.cpp:244] Train net output #5: fuse_loss = 208.561 ( 1 = 208.561 loss) I1014 06:15:17.465832 20911 sgd_solver.cpp:106] Iteration 17160, lr = 1e-07 I1014 06:15:40.797871 20911 solver.cpp:228] Iteration 17180, loss = 11055.7 I1014 06:15:40.797899 20911 solver.cpp:244] Train net output #0: dsn1_loss = 1.04056 ( 1 = 1.04056 loss) I1014 06:15:40.797904 20911 solver.cpp:244] Train net output #1: dsn2_loss = 0.855647 ( 1 = 0.855647 loss) I1014 06:15:40.797907 20911 solver.cpp:244] Train net output #2: dsn3_loss = 0.777881 ( 1 = 0.777881 loss) I1014 06:15:40.797911 20911 solver.cpp:244] Train net output #3: dsn4_loss = 0.707237 ( 1 = 0.707237 loss) I1014 06:15:40.797915 20911 solver.cpp:244] Train net output #4: dsn5_loss = 0.612388 ( 1 = 0.612388 loss) I1014 06:15:40.797920 20911 solver.cpp:244] Train net output #5: fuse_loss = 0.649212 ( 1 = 0.649212 loss) I1014 06:15:40.797925 20911 sgd_solver.cpp:106] Iteration 17180, lr = 1e-07 I1014 06:16:05.058589 20911 solver.cpp:228] Iteration 17200, loss = 11196.7 I1014 06:16:05.058615 20911 solver.cpp:244] Train net output #0: dsn1_loss = 4.47239 ( 1 = 4.47239 loss) I1014 06:16:05.058619 20911 solver.cpp:244] Train net output #1: dsn2_loss = 3.53133 ( 1 = 3.53133 loss) I1014 06:16:05.058622 20911 solver.cpp:244] Train net output #2: dsn3_loss = 2.96814 ( 1 = 2.96814 loss) I1014 06:16:05.058625 20911 solver.cpp:244] Train net output #3: dsn4_loss = 3.13658 ( 1 = 3.13658 loss) I1014 06:16:05.058629 20911 solver.cpp:244] Train net output #4: dsn5_loss = 2.76121 ( 1 = 2.76121 loss) I1014 06:16:05.058632 20911 solver.cpp:244] Train net output #5: fuse_loss = 2.9702 ( 1 = 2.9702 loss) I1014 06:16:05.058635 20911 sgd_solver.cpp:106] Iteration 17200, lr = 1e-07 I1014 06:16:31.084764 20911 solver.cpp:228] Iteration 17220, loss = 9925.7 I1014 06:16:31.084792 20911 solver.cpp:244] Train net output #0: dsn1_loss = 6846.92 ( 1 = 6846.92 loss) I1014 06:16:31.084797 20911 solver.cpp:244] Train net output #1: dsn2_loss = 5425.76 ( 1 = 5425.76 loss) I1014 06:16:31.084800 20911 solver.cpp:244] Train net output #2: dsn3_loss = 4687.33 ( 1 = 4687.33 loss) I1014 06:16:31.084805 20911 solver.cpp:244] Train net output #3: dsn4_loss = 4466.74 ( 1 = 4466.74 loss) I1014 06:16:31.084808 20911 solver.cpp:244] Train net output #4: dsn5_loss = 4316.88 ( 1 = 4316.88 loss) I1014 06:16:31.084812 20911 solver.cpp:244] Train net output #5: fuse_loss = 4137.84 ( 1 = 4137.84 loss) I1014 06:16:31.084817 20911 sgd_solver.cpp:106] Iteration 17220, lr = 1e-07 I1014 06:16:55.514008 20911 solver.cpp:228] Iteration 17240, loss = 9438.15 I1014 06:16:55.514039 20911 solver.cpp:244] Train net output #0: dsn1_loss = 12231.9 ( 1 = 12231.9 loss) I1014 06:16:55.514044 20911 solver.cpp:244] Train net output #1: dsn2_loss = 9231.36 ( 1 = 9231.36 loss) I1014 06:16:55.514046 20911 solver.cpp:244] Train net output #2: dsn3_loss = 7697.39 ( 1 = 7697.39 loss) I1014 06:16:55.514050 20911 solver.cpp:244] Train net output #3: dsn4_loss = 7427.4 ( 1 = 7427.4 loss) I1014 06:16:55.514055 20911 solver.cpp:244] Train net output #4: dsn5_loss = 6965.29 ( 1 = 6965.29 loss) I1014 06:16:55.514058 20911 solver.cpp:244] Train net output #5: fuse_loss = 6733.24 ( 1 = 6733.24 loss) I1014 06:16:55.514062 20911 sgd_solver.cpp:106] Iteration 17240, lr = 1e-07 I1014 06:17:20.071039 20911 solver.cpp:228] Iteration 17260, loss = 9479.04 I1014 06:17:20.071064 20911 solver.cpp:244] Train net output #0: dsn1_loss = 312.341 ( 1 = 312.341 loss) I1014 06:17:20.071069 20911 solver.cpp:244] Train net output #1: dsn2_loss = 282.348 ( 1 = 282.348 loss) I1014 06:17:20.071074 20911 solver.cpp:244] Train net output #2: dsn3_loss = 202.67 ( 1 = 202.67 loss) I1014 06:17:20.071076 20911 solver.cpp:244] Train net output #3: dsn4_loss = 227.15 ( 1 = 227.15 loss) I1014 06:17:20.071080 20911 solver.cpp:244] Train net output #4: dsn5_loss = 255.047 ( 1 = 255.047 loss) I1014 06:17:20.071084 20911 solver.cpp:244] Train net output #5: fuse_loss = 195.787 ( 1 = 195.787 loss) I1014 06:17:20.071089 20911 sgd_solver.cpp:106] Iteration 17260, lr = 1e-07 I1014 06:17:46.277192 20911 solver.cpp:228] Iteration 17280, loss = 10360.2 I1014 06:17:46.277225 20911 solver.cpp:244] Train net output #0: dsn1_loss = 2354.03 ( 1 = 2354.03 loss) I1014 06:17:46.277230 20911 solver.cpp:244] Train net output #1: dsn2_loss = 1686.51 ( 1 = 1686.51 loss) I1014 06:17:46.277233 20911 solver.cpp:244] Train net output #2: dsn3_loss = 1538 ( 1 = 1538 loss) I1014 06:17:46.277237 20911 solver.cpp:244] Train net output #3: dsn4_loss = 1750.84 ( 1 = 1750.84 loss) I1014 06:17:46.277241 20911 solver.cpp:244] Train net output #4: dsn5_loss = 2063.38 ( 1 = 2063.38 loss) I1014 06:17:46.277246 20911 solver.cpp:244] Train net output #5: fuse_loss = 1365.32 ( 1 = 1365.32 loss) I1014 06:17:46.277251 20911 sgd_solver.cpp:106] Iteration 17280, lr = 1e-07 I1014 06:18:11.518421 20911 solver.cpp:228] Iteration 17300, loss = 10527.4 I1014 06:18:11.518456 20911 solver.cpp:244] Train net output #0: dsn1_loss = 1657.69 ( 1 = 1657.69 loss) I1014 06:18:11.518461 20911 solver.cpp:244] Train net output #1: dsn2_loss = 1265.97 ( 1 = 1265.97 loss) I1014 06:18:11.518465 20911 solver.cpp:244] Train net output #2: dsn3_loss = 1091.3 ( 1 = 1091.3 loss) I1014 06:18:11.518470 20911 solver.cpp:244] Train net output #3: dsn4_loss = 1107 ( 1 = 1107 loss) I1014 06:18:11.518472 20911 solver.cpp:244] Train net output #4: dsn5_loss = 932.074 ( 1 = 932.074 loss) I1014 06:18:11.518476 20911 solver.cpp:244] Train net output #5: fuse_loss = 837.649 ( 1 = 837.649 loss) I1014 06:18:11.518481 20911 sgd_solver.cpp:106] Iteration 17300, lr = 1e-07 I1014 06:18:37.806651 20911 solver.cpp:228] Iteration 17320, loss = 10868.9 I1014 06:18:37.806687 20911 solver.cpp:244] Train net output #0: dsn1_loss = 1256.3 ( 1 = 1256.3 loss) I1014 06:18:37.806691 20911 solver.cpp:244] Train net output #1: dsn2_loss = 812.338 ( 1 = 812.338 loss) I1014 06:18:37.806696 20911 solver.cpp:244] Train net output #2: dsn3_loss = 582.675 ( 1 = 582.675 loss) I1014 06:18:37.806700 20911 solver.cpp:244] Train net output #3: dsn4_loss = 548.849 ( 1 = 548.849 loss) I1014 06:18:37.806704 20911 solver.cpp:244] Train net output #4: dsn5_loss = 439.413 ( 1 = 439.413 loss) I1014 06:18:37.806707 20911 solver.cpp:244] Train net output #5: fuse_loss = 449.013 ( 1 = 449.013 loss) I1014 06:18:37.806712 20911 sgd_solver.cpp:106] Iteration 17320, lr = 1e-07 I1014 06:19:01.623241 20911 solver.cpp:228] Iteration 17340, loss = 11197 I1014 06:19:01.623267 20911 solver.cpp:244] Train net output #0: dsn1_loss = 5686.43 ( 1 = 5686.43 loss) I1014 06:19:01.623272 20911 solver.cpp:244] Train net output #1: dsn2_loss = 4880.61 ( 1 = 4880.61 loss) I1014 06:19:01.623275 20911 solver.cpp:244] Train net output #2: dsn3_loss = 3855.27 ( 1 = 3855.27 loss) I1014 06:19:01.623280 20911 solver.cpp:244] Train net output #3: dsn4_loss = 3363.5 ( 1 = 3363.5 loss) I1014 06:19:01.623283 20911 solver.cpp:244] Train net output #4: dsn5_loss = 3121.05 ( 1 = 3121.05 loss) I1014 06:19:01.623286 20911 solver.cpp:244] Train net output #5: fuse_loss = 2987.21 ( 1 = 2987.21 loss) I1014 06:19:01.623291 20911 sgd_solver.cpp:106] Iteration 17340, lr = 1e-07 I1014 06:19:27.538549 20911 solver.cpp:228] Iteration 17360, loss = 10466.2 I1014 06:19:27.538579 20911 solver.cpp:244] Train net output #0: dsn1_loss = 3367.35 ( 1 = 3367.35 loss) I1014 06:19:27.538584 20911 solver.cpp:244] Train net output #1: dsn2_loss = 2638.69 ( 1 = 2638.69 loss) I1014 06:19:27.538589 20911 solver.cpp:244] Train net output #2: dsn3_loss = 1920.2 ( 1 = 1920.2 loss) I1014 06:19:27.538592 20911 solver.cpp:244] Train net output #3: dsn4_loss = 1687.42 ( 1 = 1687.42 loss) I1014 06:19:27.538596 20911 solver.cpp:244] Train net output #4: dsn5_loss = 1669.23 ( 1 = 1669.23 loss) I1014 06:19:27.538599 20911 solver.cpp:244] Train net output #5: fuse_loss = 1596.75 ( 1 = 1596.75 loss) I1014 06:19:27.538604 20911 sgd_solver.cpp:106] Iteration 17360, lr = 1e-07 I1014 06:19:52.811216 20911 solver.cpp:228] Iteration 17380, loss = 10399.3 I1014 06:19:52.811244 20911 solver.cpp:244] Train net output #0: dsn1_loss = 6930.42 ( 1 = 6930.42 loss) I1014 06:19:52.811249 20911 solver.cpp:244] Train net output #1: dsn2_loss = 5418.41 ( 1 = 5418.41 loss) I1014 06:19:52.811251 20911 solver.cpp:244] Train net output #2: dsn3_loss = 4619.3 ( 1 = 4619.3 loss) I1014 06:19:52.811255 20911 solver.cpp:244] Train net output #3: dsn4_loss = 4591.94 ( 1 = 4591.94 loss) I1014 06:19:52.811259 20911 solver.cpp:244] Train net output #4: dsn5_loss = 4151.14 ( 1 = 4151.14 loss) I1014 06:19:52.811262 20911 solver.cpp:244] Train net output #5: fuse_loss = 4087.57 ( 1 = 4087.57 loss) I1014 06:19:52.811265 20911 sgd_solver.cpp:106] Iteration 17380, lr = 1e-07 I1014 06:20:17.147521 20911 solver.cpp:228] Iteration 17400, loss = 10632.1 I1014 06:20:17.147557 20911 solver.cpp:244] Train net output #0: dsn1_loss = 1470.86 ( 1 = 1470.86 loss) I1014 06:20:17.147562 20911 solver.cpp:244] Train net output #1: dsn2_loss = 1171.51 ( 1 = 1171.51 loss) I1014 06:20:17.147564 20911 solver.cpp:244] Train net output #2: dsn3_loss = 546.003 ( 1 = 546.003 loss) I1014 06:20:17.147568 20911 solver.cpp:244] Train net output #3: dsn4_loss = 434.96 ( 1 = 434.96 loss) I1014 06:20:17.147572 20911 solver.cpp:244] Train net output #4: dsn5_loss = 472.074 ( 1 = 472.074 loss) I1014 06:20:17.147577 20911 solver.cpp:244] Train net output #5: fuse_loss = 452.216 ( 1 = 452.216 loss) I1014 06:20:17.147579 20911 sgd_solver.cpp:106] Iteration 17400, lr = 1e-07 I1014 06:20:42.437880 20911 solver.cpp:228] Iteration 17420, loss = 11001.9 I1014 06:20:42.437904 20911 solver.cpp:244] Train net output #0: dsn1_loss = 1.85305 ( 1 = 1.85305 loss) I1014 06:20:42.437908 20911 solver.cpp:244] Train net output #1: dsn2_loss = 2.12688 ( 1 = 2.12688 loss) I1014 06:20:42.437911 20911 solver.cpp:244] Train net output #2: dsn3_loss = 2.0985 ( 1 = 2.0985 loss) I1014 06:20:42.437914 20911 solver.cpp:244] Train net output #3: dsn4_loss = 2.90395 ( 1 = 2.90395 loss) I1014 06:20:42.437918 20911 solver.cpp:244] Train net output #4: dsn5_loss = 2.38611 ( 1 = 2.38611 loss) I1014 06:20:42.437922 20911 solver.cpp:244] Train net output #5: fuse_loss = 1.87458 ( 1 = 1.87458 loss) I1014 06:20:42.437925 20911 sgd_solver.cpp:106] Iteration 17420, lr = 1e-07 I1014 06:21:08.147997 20911 solver.cpp:228] Iteration 17440, loss = 10869.5 I1014 06:21:08.148022 20911 solver.cpp:244] Train net output #0: dsn1_loss = 4209.24 ( 1 = 4209.24 loss) I1014 06:21:08.148027 20911 solver.cpp:244] Train net output #1: dsn2_loss = 3323.25 ( 1 = 3323.25 loss) I1014 06:21:08.148030 20911 solver.cpp:244] Train net output #2: dsn3_loss = 2678.44 ( 1 = 2678.44 loss) I1014 06:21:08.148035 20911 solver.cpp:244] Train net output #3: dsn4_loss = 2485.71 ( 1 = 2485.71 loss) I1014 06:21:08.148037 20911 solver.cpp:244] Train net output #4: dsn5_loss = 2089.31 ( 1 = 2089.31 loss) I1014 06:21:08.148041 20911 solver.cpp:244] Train net output #5: fuse_loss = 2252.59 ( 1 = 2252.59 loss) I1014 06:21:08.148046 20911 sgd_solver.cpp:106] Iteration 17440, lr = 1e-07 I1014 06:21:32.127687 20911 solver.cpp:228] Iteration 17460, loss = 10527.7 I1014 06:21:32.127722 20911 solver.cpp:244] Train net output #0: dsn1_loss = 653.727 ( 1 = 653.727 loss) I1014 06:21:32.127727 20911 solver.cpp:244] Train net output #1: dsn2_loss = 539.302 ( 1 = 539.302 loss) I1014 06:21:32.127729 20911 solver.cpp:244] Train net output #2: dsn3_loss = 385.846 ( 1 = 385.846 loss) I1014 06:21:32.127733 20911 solver.cpp:244] Train net output #3: dsn4_loss = 270.926 ( 1 = 270.926 loss) I1014 06:21:32.127737 20911 solver.cpp:244] Train net output #4: dsn5_loss = 263.342 ( 1 = 263.342 loss) I1014 06:21:32.127740 20911 solver.cpp:244] Train net output #5: fuse_loss = 215.386 ( 1 = 215.386 loss) I1014 06:21:32.127744 20911 sgd_solver.cpp:106] Iteration 17460, lr = 1e-07 I1014 06:21:57.994457 20911 solver.cpp:228] Iteration 17480, loss = 10919.4 I1014 06:21:57.994488 20911 solver.cpp:244] Train net output #0: dsn1_loss = 4660.3 ( 1 = 4660.3 loss) I1014 06:21:57.994493 20911 solver.cpp:244] Train net output #1: dsn2_loss = 4074.57 ( 1 = 4074.57 loss) I1014 06:21:57.994496 20911 solver.cpp:244] Train net output #2: dsn3_loss = 3423.29 ( 1 = 3423.29 loss) I1014 06:21:57.994500 20911 solver.cpp:244] Train net output #3: dsn4_loss = 3113 ( 1 = 3113 loss) I1014 06:21:57.994503 20911 solver.cpp:244] Train net output #4: dsn5_loss = 2651.08 ( 1 = 2651.08 loss) I1014 06:21:57.994508 20911 solver.cpp:244] Train net output #5: fuse_loss = 2554.85 ( 1 = 2554.85 loss) I1014 06:21:57.994511 20911 sgd_solver.cpp:106] Iteration 17480, lr = 1e-07 I1014 06:22:23.384407 20911 solver.cpp:228] Iteration 17500, loss = 10346.2 I1014 06:22:23.384433 20911 solver.cpp:244] Train net output #0: dsn1_loss = 0 ( 1 = 0 loss) I1014 06:22:23.384436 20911 solver.cpp:244] Train net output #1: dsn2_loss = 0 ( 1 = 0 loss) I1014 06:22:23.384440 20911 solver.cpp:244] Train net output #2: dsn3_loss = 0 ( 1 = 0 loss) I1014 06:22:23.384443 20911 solver.cpp:244] Train net output #3: dsn4_loss = 0 ( 1 = 0 loss) I1014 06:22:23.384445 20911 solver.cpp:244] Train net output #4: dsn5_loss = 0 ( 1 = 0 loss) I1014 06:22:23.384449 20911 solver.cpp:244] Train net output #5: fuse_loss = 0 ( 1 = 0 loss) I1014 06:22:23.384451 20911 sgd_solver.cpp:106] Iteration 17500, lr = 1e-07 I1014 06:22:49.241433 20911 solver.cpp:228] Iteration 17520, loss = 10524.8 I1014 06:22:49.241462 20911 solver.cpp:244] Train net output #0: dsn1_loss = 262.256 ( 1 = 262.256 loss) I1014 06:22:49.241466 20911 solver.cpp:244] Train net output #1: dsn2_loss = 239.158 ( 1 = 239.158 loss) I1014 06:22:49.241470 20911 solver.cpp:244] Train net output #2: dsn3_loss = 211.789 ( 1 = 211.789 loss) I1014 06:22:49.241474 20911 solver.cpp:244] Train net output #3: dsn4_loss = 195.305 ( 1 = 195.305 loss) I1014 06:22:49.241477 20911 solver.cpp:244] Train net output #4: dsn5_loss = 191.312 ( 1 = 191.312 loss) I1014 06:22:49.241482 20911 solver.cpp:244] Train net output #5: fuse_loss = 176.931 ( 1 = 176.931 loss) I1014 06:22:49.241485 20911 sgd_solver.cpp:106] Iteration 17520, lr = 1e-07 I1014 06:23:13.815886 20911 solver.cpp:228] Iteration 17540, loss = 11537.4 I1014 06:23:13.815913 20911 solver.cpp:244] Train net output #0: dsn1_loss = 1.04085 ( 1 = 1.04085 loss) I1014 06:23:13.815918 20911 solver.cpp:244] Train net output #1: dsn2_loss = 0.735496 ( 1 = 0.735496 loss) I1014 06:23:13.815922 20911 solver.cpp:244] Train net output #2: dsn3_loss = 0.621013 ( 1 = 0.621013 loss) I1014 06:23:13.815925 20911 solver.cpp:244] Train net output #3: dsn4_loss = 0.67541 ( 1 = 0.67541 loss) I1014 06:23:13.815928 20911 solver.cpp:244] Train net output #4: dsn5_loss = 0.598172 ( 1 = 0.598172 loss) I1014 06:23:13.815932 20911 solver.cpp:244] Train net output #5: fuse_loss = 0.567175 ( 1 = 0.567175 loss) I1014 06:23:13.815935 20911 sgd_solver.cpp:106] Iteration 17540, lr = 1e-07 I1014 06:23:40.095712 20911 solver.cpp:228] Iteration 17560, loss = 12284.2 I1014 06:23:40.095744 20911 solver.cpp:244] Train net output #0: dsn1_loss = 225.747 ( 1 = 225.747 loss) I1014 06:23:40.095748 20911 solver.cpp:244] Train net output #1: dsn2_loss = 194.783 ( 1 = 194.783 loss) I1014 06:23:40.095752 20911 solver.cpp:244] Train net output #2: dsn3_loss = 249.127 ( 1 = 249.127 loss) I1014 06:23:40.095755 20911 solver.cpp:244] Train net output #3: dsn4_loss = 186.502 ( 1 = 186.502 loss) I1014 06:23:40.095759 20911 solver.cpp:244] Train net output #4: dsn5_loss = 149.41 ( 1 = 149.41 loss) I1014 06:23:40.095763 20911 solver.cpp:244] Train net output #5: fuse_loss = 130.459 ( 1 = 130.459 loss) I1014 06:23:40.095767 20911 sgd_solver.cpp:106] Iteration 17560, lr = 1e-07 I1014 06:24:03.725445 20911 solver.cpp:228] Iteration 17580, loss = 11306.6 I1014 06:24:03.725473 20911 solver.cpp:244] Train net output #0: dsn1_loss = 3858.63 ( 1 = 3858.63 loss) I1014 06:24:03.725478 20911 solver.cpp:244] Train net output #1: dsn2_loss = 2211.17 ( 1 = 2211.17 loss) I1014 06:24:03.725482 20911 solver.cpp:244] Train net output #2: dsn3_loss = 1693.11 ( 1 = 1693.11 loss) I1014 06:24:03.725486 20911 solver.cpp:244] Train net output #3: dsn4_loss = 1715.58 ( 1 = 1715.58 loss) I1014 06:24:03.725491 20911 solver.cpp:244] Train net output #4: dsn5_loss = 1522.18 ( 1 = 1522.18 loss) I1014 06:24:03.725493 20911 solver.cpp:244] Train net output #5: fuse_loss = 1402.32 ( 1 = 1402.32 loss) I1014 06:24:03.725497 20911 sgd_solver.cpp:106] Iteration 17580, lr = 1e-07 I1014 06:24:27.519074 20911 solver.cpp:228] Iteration 17600, loss = 9605.9 I1014 06:24:27.519105 20911 solver.cpp:244] Train net output #0: dsn1_loss = 1875.91 ( 1 = 1875.91 loss) I1014 06:24:27.519110 20911 solver.cpp:244] Train net output #1: dsn2_loss = 1326.33 ( 1 = 1326.33 loss) I1014 06:24:27.519114 20911 solver.cpp:244] Train net output #2: dsn3_loss = 1165.06 ( 1 = 1165.06 loss) I1014 06:24:27.519117 20911 solver.cpp:244] Train net output #3: dsn4_loss = 962.88 ( 1 = 962.88 loss) I1014 06:24:27.519121 20911 solver.cpp:244] Train net output #4: dsn5_loss = 746.879 ( 1 = 746.879 loss) I1014 06:24:27.519125 20911 solver.cpp:244] Train net output #5: fuse_loss = 771.579 ( 1 = 771.579 loss) I1014 06:24:27.519130 20911 sgd_solver.cpp:106] Iteration 17600, lr = 1e-07 I1014 06:24:52.476837 20911 solver.cpp:228] Iteration 17620, loss = 8696.38 I1014 06:24:52.476866 20911 solver.cpp:244] Train net output #0: dsn1_loss = 550.764 ( 1 = 550.764 loss) I1014 06:24:52.476871 20911 solver.cpp:244] Train net output #1: dsn2_loss = 468.771 ( 1 = 468.771 loss) I1014 06:24:52.476874 20911 solver.cpp:244] Train net output #2: dsn3_loss = 332.824 ( 1 = 332.824 loss) I1014 06:24:52.476878 20911 solver.cpp:244] Train net output #3: dsn4_loss = 270.827 ( 1 = 270.827 loss) I1014 06:24:52.476882 20911 solver.cpp:244] Train net output #4: dsn5_loss = 248.062 ( 1 = 248.062 loss) I1014 06:24:52.476886 20911 solver.cpp:244] Train net output #5: fuse_loss = 233.987 ( 1 = 233.987 loss) I1014 06:24:52.476891 20911 sgd_solver.cpp:106] Iteration 17620, lr = 1e-07 I1014 06:25:16.686174 20911 solver.cpp:228] Iteration 17640, loss = 8792.74 I1014 06:25:16.686202 20911 solver.cpp:244] Train net output #0: dsn1_loss = 34.1634 ( 1 = 34.1634 loss) I1014 06:25:16.686206 20911 solver.cpp:244] Train net output #1: dsn2_loss = 22.7592 ( 1 = 22.7592 loss) I1014 06:25:16.686210 20911 solver.cpp:244] Train net output #2: dsn3_loss = 20.0064 ( 1 = 20.0064 loss) I1014 06:25:16.686213 20911 solver.cpp:244] Train net output #3: dsn4_loss = 17.8031 ( 1 = 17.8031 loss) I1014 06:25:16.686218 20911 solver.cpp:244] Train net output #4: dsn5_loss = 22.2257 ( 1 = 22.2257 loss) I1014 06:25:16.686220 20911 solver.cpp:244] Train net output #5: fuse_loss = 16.1762 ( 1 = 16.1762 loss) I1014 06:25:16.686224 20911 sgd_solver.cpp:106] Iteration 17640, lr = 1e-07 I1014 06:25:43.914378 20911 solver.cpp:228] Iteration 17660, loss = 9760.44 I1014 06:25:43.914403 20911 solver.cpp:244] Train net output #0: dsn1_loss = 593.461 ( 1 = 593.461 loss) I1014 06:25:43.914408 20911 solver.cpp:244] Train net output #1: dsn2_loss = 652.67 ( 1 = 652.67 loss) I1014 06:25:43.914413 20911 solver.cpp:244] Train net output #2: dsn3_loss = 558.682 ( 1 = 558.682 loss) I1014 06:25:43.914417 20911 solver.cpp:244] Train net output #3: dsn4_loss = 584.638 ( 1 = 584.638 loss) I1014 06:25:43.914420 20911 solver.cpp:244] Train net output #4: dsn5_loss = 567.683 ( 1 = 567.683 loss) I1014 06:25:43.914424 20911 solver.cpp:244] Train net output #5: fuse_loss = 409.183 ( 1 = 409.183 loss) I1014 06:25:43.914428 20911 sgd_solver.cpp:106] Iteration 17660, lr = 1e-07 I1014 06:26:08.589584 20911 solver.cpp:228] Iteration 17680, loss = 10239.2 I1014 06:26:08.589618 20911 solver.cpp:244] Train net output #0: dsn1_loss = 873.055 ( 1 = 873.055 loss) I1014 06:26:08.589623 20911 solver.cpp:244] Train net output #1: dsn2_loss = 729.568 ( 1 = 729.568 loss) I1014 06:26:08.589627 20911 solver.cpp:244] Train net output #2: dsn3_loss = 403.758 ( 1 = 403.758 loss) I1014 06:26:08.589632 20911 solver.cpp:244] Train net output #3: dsn4_loss = 310.149 ( 1 = 310.149 loss) I1014 06:26:08.589635 20911 solver.cpp:244] Train net output #4: dsn5_loss = 251.903 ( 1 = 251.903 loss) I1014 06:26:08.589639 20911 solver.cpp:244] Train net output #5: fuse_loss = 293.253 ( 1 = 293.253 loss) I1014 06:26:08.589645 20911 sgd_solver.cpp:106] Iteration 17680, lr = 1e-07 I1014 06:26:33.351687 20911 solver.cpp:228] Iteration 17700, loss = 10091.8 I1014 06:26:33.351716 20911 solver.cpp:244] Train net output #0: dsn1_loss = 422.93 ( 1 = 422.93 loss) I1014 06:26:33.351722 20911 solver.cpp:244] Train net output #1: dsn2_loss = 395.524 ( 1 = 395.524 loss) I1014 06:26:33.351725 20911 solver.cpp:244] Train net output #2: dsn3_loss = 337.67 ( 1 = 337.67 loss) I1014 06:26:33.351729 20911 solver.cpp:244] Train net output #3: dsn4_loss = 291.185 ( 1 = 291.185 loss) I1014 06:26:33.351732 20911 solver.cpp:244] Train net output #4: dsn5_loss = 320.587 ( 1 = 320.587 loss) I1014 06:26:33.351737 20911 solver.cpp:244] Train net output #5: fuse_loss = 286.616 ( 1 = 286.616 loss) I1014 06:26:33.351740 20911 sgd_solver.cpp:106] Iteration 17700, lr = 1e-07 I1014 06:26:57.826781 20911 solver.cpp:228] Iteration 17720, loss = 9518.21 I1014 06:26:57.826809 20911 solver.cpp:244] Train net output #0: dsn1_loss = 6584.81 ( 1 = 6584.81 loss) I1014 06:26:57.826814 20911 solver.cpp:244] Train net output #1: dsn2_loss = 5987.27 ( 1 = 5987.27 loss) I1014 06:26:57.826817 20911 solver.cpp:244] Train net output #2: dsn3_loss = 5306.81 ( 1 = 5306.81 loss) I1014 06:26:57.826822 20911 solver.cpp:244] Train net output #3: dsn4_loss = 4698.7 ( 1 = 4698.7 loss) I1014 06:26:57.826825 20911 solver.cpp:244] Train net output #4: dsn5_loss = 4064.94 ( 1 = 4064.94 loss) I1014 06:26:57.826828 20911 solver.cpp:244] Train net output #5: fuse_loss = 3932.76 ( 1 = 3932.76 loss) I1014 06:26:57.826833 20911 sgd_solver.cpp:106] Iteration 17720, lr = 1e-07 I1014 06:27:22.723006 20911 solver.cpp:228] Iteration 17740, loss = 9918.07 I1014 06:27:22.723031 20911 solver.cpp:244] Train net output #0: dsn1_loss = 1215.6 ( 1 = 1215.6 loss) I1014 06:27:22.723034 20911 solver.cpp:244] Train net output #1: dsn2_loss = 653.785 ( 1 = 653.785 loss) I1014 06:27:22.723038 20911 solver.cpp:244] Train net output #2: dsn3_loss = 501.922 ( 1 = 501.922 loss) I1014 06:27:22.723042 20911 solver.cpp:244] Train net output #3: dsn4_loss = 439.097 ( 1 = 439.097 loss) I1014 06:27:22.723045 20911 solver.cpp:244] Train net output #4: dsn5_loss = 373.711 ( 1 = 373.711 loss) I1014 06:27:22.723048 20911 solver.cpp:244] Train net output #5: fuse_loss = 395.55 ( 1 = 395.55 loss) I1014 06:27:22.723052 20911 sgd_solver.cpp:106] Iteration 17740, lr = 1e-07 I1014 06:27:49.926905 20911 solver.cpp:228] Iteration 17760, loss = 11213.8 I1014 06:27:49.926935 20911 solver.cpp:244] Train net output #0: dsn1_loss = 5676.11 ( 1 = 5676.11 loss) I1014 06:27:49.926940 20911 solver.cpp:244] Train net output #1: dsn2_loss = 3981.07 ( 1 = 3981.07 loss) I1014 06:27:49.926944 20911 solver.cpp:244] Train net output #2: dsn3_loss = 3783.93 ( 1 = 3783.93 loss) I1014 06:27:49.926947 20911 solver.cpp:244] Train net output #3: dsn4_loss = 3780.37 ( 1 = 3780.37 loss) I1014 06:27:49.926952 20911 solver.cpp:244] Train net output #4: dsn5_loss = 3555.2 ( 1 = 3555.2 loss) I1014 06:27:49.926955 20911 solver.cpp:244] Train net output #5: fuse_loss = 3312.95 ( 1 = 3312.95 loss) I1014 06:27:49.926959 20911 sgd_solver.cpp:106] Iteration 17760, lr = 1e-07 I1014 06:28:16.035239 20911 solver.cpp:228] Iteration 17780, loss = 11496 I1014 06:28:16.035266 20911 solver.cpp:244] Train net output #0: dsn1_loss = 1.53796 ( 1 = 1.53796 loss) I1014 06:28:16.035270 20911 solver.cpp:244] Train net output #1: dsn2_loss = 1.22264 ( 1 = 1.22264 loss) I1014 06:28:16.035274 20911 solver.cpp:244] Train net output #2: dsn3_loss = 2.04265 ( 1 = 2.04265 loss) I1014 06:28:16.035277 20911 solver.cpp:244] Train net output #3: dsn4_loss = 1.4683 ( 1 = 1.4683 loss) I1014 06:28:16.035280 20911 solver.cpp:244] Train net output #4: dsn5_loss = 1.30301 ( 1 = 1.30301 loss) I1014 06:28:16.035284 20911 solver.cpp:244] Train net output #5: fuse_loss = 1.28767 ( 1 = 1.28767 loss) I1014 06:28:16.035287 20911 sgd_solver.cpp:106] Iteration 17780, lr = 1e-07 I1014 06:28:40.988015 20911 solver.cpp:228] Iteration 17800, loss = 11246.3 I1014 06:28:40.988041 20911 solver.cpp:244] Train net output #0: dsn1_loss = 6775.97 ( 1 = 6775.97 loss) I1014 06:28:40.988046 20911 solver.cpp:244] Train net output #1: dsn2_loss = 6549.69 ( 1 = 6549.69 loss) I1014 06:28:40.988049 20911 solver.cpp:244] Train net output #2: dsn3_loss = 5554.22 ( 1 = 5554.22 loss) I1014 06:28:40.988054 20911 solver.cpp:244] Train net output #3: dsn4_loss = 4851.64 ( 1 = 4851.64 loss) I1014 06:28:40.988057 20911 solver.cpp:244] Train net output #4: dsn5_loss = 4233.41 ( 1 = 4233.41 loss) I1014 06:28:40.988060 20911 solver.cpp:244] Train net output #5: fuse_loss = 4581.8 ( 1 = 4581.8 loss) I1014 06:28:40.988066 20911 sgd_solver.cpp:106] Iteration 17800, lr = 1e-07 I1014 06:29:04.613157 20911 solver.cpp:228] Iteration 17820, loss = 10920.4 I1014 06:29:04.613184 20911 solver.cpp:244] Train net output #0: dsn1_loss = 333.079 ( 1 = 333.079 loss) I1014 06:29:04.613189 20911 solver.cpp:244] Train net output #1: dsn2_loss = 225.963 ( 1 = 225.963 loss) I1014 06:29:04.613193 20911 solver.cpp:244] Train net output #2: dsn3_loss = 165.173 ( 1 = 165.173 loss) I1014 06:29:04.613196 20911 solver.cpp:244] Train net output #3: dsn4_loss = 176.134 ( 1 = 176.134 loss) I1014 06:29:04.613200 20911 solver.cpp:244] Train net output #4: dsn5_loss = 164.467 ( 1 = 164.467 loss) I1014 06:29:04.613204 20911 solver.cpp:244] Train net output #5: fuse_loss = 151.299 ( 1 = 151.299 loss) I1014 06:29:04.613209 20911 sgd_solver.cpp:106] Iteration 17820, lr = 1e-07 ^AI1014 06:29:28.387291 20911 solver.cpp:228] Iteration 17840, loss = 10571.1 I1014 06:29:28.387316 20911 solver.cpp:244] Train net output #0: dsn1_loss = 0.766708 ( 1 = 0.766708 loss) I1014 06:29:28.387321 20911 solver.cpp:244] Train net output #1: dsn2_loss = 0.572542 ( 1 = 0.572542 loss) I1014 06:29:28.387325 20911 solver.cpp:244] Train net output #2: dsn3_loss = 0.343473 ( 1 = 0.343473 loss) I1014 06:29:28.387327 20911 solver.cpp:244] Train net output #3: dsn4_loss = 0.394873 ( 1 = 0.394873 loss) I1014 06:29:28.387331 20911 solver.cpp:244] Train net output #4: dsn5_loss = 0.33945 ( 1 = 0.33945 loss) I1014 06:29:28.387334 20911 solver.cpp:244] Train net output #5: fuse_loss = 0.290347 ( 1 = 0.290347 loss) I1014 06:29:28.387338 20911 sgd_solver.cpp:106] Iteration 17840, lr = 1e-07 Why is thatI1014 06:29:53.370223 20911 solver.cpp:228] Iteration 17860, loss = 11081.8 I1014 06:29:53.370249 20911 solver.cpp:244] Train net output #0: dsn1_loss = 7151.27 ( 1 = 7151.27 loss) I1014 06:29:53.370254 20911 solver.cpp:244] Train net output #1: dsn2_loss = 5249.87 ( 1 = 5249.87 loss) I1014 06:29:53.370257 20911 solver.cpp:244] Train net output #2: dsn3_loss = 4377.98 ( 1 = 4377.98 loss) I1014 06:29:53.370260 20911 solver.cpp:244] Train net output #3: dsn4_loss = 4299.77 ( 1 = 4299.77 loss) I1014 06:29:53.370263 20911 solver.cpp:244] Train net output #4: dsn5_loss = 3845.45 ( 1 = 3845.45 loss) I1014 06:29:53.370267 20911 solver.cpp:244] Train net output #5: fuse_loss = 3858.16 ( 1 = 3858.16 loss) I1014 06:29:53.370270 20911 sgd_solver.cpp:106] Iteration 17860, lr = 1e-07

yun-liu commented 5 years ago

If it is trained on PASCAL+BSDS500 dataset, I think the output loss is normal. Due to the significant difference between various images for edge detection, the loss usually has a fluctuation. You can test your trained model to see if it is well trained.

WWWJL commented 5 years ago

I tested my tarined model,and Verified this model is valid,thank you

WWWJL commented 5 years ago

hi,recently I used my own dataset to train the model and found that the LOSS value did not fall well. I think my groundtruth-images may not very good, which affecting the training effect. I want to ask you what method is the groundtruth-image of the picture produced? I thought by canny,but now i think i'm wrong.

frankie-yanfeng commented 5 years ago

@WWWJL https://www2.eecs.berkeley.edu/Research/Projects/CS/vision/bsds/

They descries how to make ground truth in above link. Actually, they built a java based tool to complete this task.