CHBMIT datastet- Bad AUC result for patient chb3

[Afef00]

When I was trying to run the code for the patient 3 in the CHBMIT dataset the training gives me an AUC of 0.307 . At the last epoch the training accuracy was 98.36% however the validation accuracy was 29.31%. I was wondering why the training stopped at this bad results and does not keep training?

[truongduynhan]

Hi Afel00, There was an early stopping that monitors both training loss and validation loss. In your case, it is likely that the monitor saw the model overfitting so it stopped the training.

[truongduynhan]

Hi Afel00,

Can you show the whole history of training? Just in case, I re-run for CHBMIT patient 3 and this is what I got. Train on 3574 samples, validate on 1192 samples Epoch 1/50 2021-10-13 13:54:40.103201: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10.0 2021-10-13 13:54:40.637237: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7

• 23s - loss: 0.3755 - accuracy: 0.8276 - val_loss: 0.7650 - val_accuracy: 0.5512 Epoch 2/50
• 9s - loss: 0.1902 - accuracy: 0.9256 - val_loss: 1.0192 - val_accuracy: 0.5612 Epoch 3/50
• 8s - loss: 0.1221 - accuracy: 0.9550 - val_loss: 0.9407 - val_accuracy: 0.6820 Epoch 4/50
• 8s - loss: 0.0647 - accuracy: 0.9757 - val_loss: 0.9588 - val_accuracy: 0.7156 Epoch 5/50
• 8s - loss: 0.0366 - accuracy: 0.9863 - val_loss: 1.0670 - val_accuracy: 0.7181 Epoch 6/50
• 8s - loss: 0.0266 - accuracy: 0.9916 - val_loss: 0.7829 - val_accuracy: 0.7836 Epoch 7/50
• 9s - loss: 0.0324 - accuracy: 0.9868 - val_loss: 1.8023 - val_accuracy: 0.6594 Epoch 8/50
• 9s - loss: 0.0196 - accuracy: 0.9938 - val_loss: 0.7990 - val_accuracy: 0.7844 Epoch 9/50
• 9s - loss: 0.0153 - accuracy: 0.9958 - val_loss: 1.3199 - val_accuracy: 0.7257 Epoch 10/50
• 9s - loss: 0.0145 - accuracy: 0.9964 - val_loss: 1.1505 - val_accuracy: 0.7131 Epoch 11/50
• 9s - loss: 0.0164 - accuracy: 0.9950 - val_loss: 2.0753 - val_accuracy: 0.6703 Epoch 12/50
• 9s - loss: 0.0088 - accuracy: 0.9975 - val_loss: 2.0546 - val_accuracy: 0.6829 Epoch 13/50
• 9s - loss: 0.0092 - accuracy: 0.9969 - val_loss: 1.4302 - val_accuracy: 0.7324 Epoch 14/50
• 9s - loss: 0.0091 - accuracy: 0.9955 - val_loss: 1.1846 - val_accuracy: 0.7341 Epoch 15/50
• 9s - loss: 0.0041 - accuracy: 0.9978 - val_loss: 1.6080 - val_accuracy: 0.6988 Epoch 16/50
• 9s - loss: 0.0078 - accuracy: 0.9969 - val_loss: 2.0751 - val_accuracy: 0.6711 Epoch 17/50
• 9s - loss: 0.0050 - accuracy: 0.9986 - val_loss: 1.7395 - val_accuracy: 0.6913 Test AUC is: 0.9930037313432836