search

qqwweee / keras-yolo3

A Keras implementation of YOLOv3 (Tensorflow backend)
MIT License
7.14k stars 3.45k forks source link

Epoch 00063: ReduceLROnPlateau reducing learning rate to 9.999999974752428e-08. Epoch 00063: early stopping #615

Open li0128 opened 4 years ago

li0128 commented 4 years ago

17/17 [==============================] - 572s 34s/step - loss: 35.4391 - val_loss: 36.9989 Epoch 48/50 17/17 [==============================] - 573s 34s/step - loss: 35.9728 - val_loss: 36.2156 Epoch 49/50 17/17 [==============================] - 570s 34s/step - loss: 34.2568 - val_loss: 39.0358 Epoch 50/50 17/17 [==============================] - 562s 33s/step - loss: 34.6852 - val_loss: 36.1666 Unfreeze all of the layers. Train on 549 samples, val on 60 samples, with batch size 32. Epoch 51/100 17/17 [==============================] - 1879s 111s/step - loss: 27.4504 - val_loss: 26.7586 Epoch 52/100 17/17 [==============================] - 1905s 112s/step - loss: 24.5947 - val_loss: 24.0862 Epoch 53/100 17/17 [==============================] - 1866s 110s/step - loss: 23.5987 - val_loss: 20.1109 Epoch 54/100 17/17 [==============================] - 1865s 110s/step - loss: 22.9273 - val_loss: 22.6067 Epoch 55/100 17/17 [==============================] - 1855s 109s/step - loss: 22.5129 - val_loss: 25.2840 Epoch 56/100 17/17 [==============================] - 1858s 109s/step - loss: 21.9349 - val_loss: 20.8508 Epoch 57/100 17/17 [==============================] - 1855s 109s/step - loss: 21.9016 - val_loss: 23.6031

Epoch 00057: ReduceLROnPlateau reducing learning rate to 9.999999747378752e-06. Epoch 58/100 17/17 [==============================] - 1853s 109s/step - loss: 21.6956 - val_loss: 21.5241 Epoch 59/100 17/17 [==============================] - 1854s 109s/step - loss: 21.7020 - val_loss: 21.3965 Epoch 60/100 17/17 [==============================] - 1856s 109s/step - loss: 21.5508 - val_loss: 23.7292

Epoch 00060: ReduceLROnPlateau reducing learning rate to 9.999999747378752e-07. Epoch 61/100 17/17 [==============================] - 1857s 109s/step - loss: 21.1803 - val_loss: 22.2895 Epoch 62/100 17/17 [==============================] - 1858s 109s/step - loss: 21.3319 - val_loss: 23.7608 Epoch 63/100 17/17 [==============================] - 1857s 109s/step - loss: 21.6261 - val_loss: 26.9629

Epoch 00063: ReduceLROnPlateau reducing learning rate to 9.999999974752428e-08. Epoch 00063: early stopping

The training stopped. How to slove the problem and let the training continue. Thank you!

israelpereira55 commented 4 years ago

The early stopping is caused by a number of epochs without improvement on val_loss. You can increase the patience to let the train go further.

It's on line 39 in train.py early_stopping = EarlyStopping(monitor='val_loss', min_delta=0, patience=10, verbose=1)