qubvel / segmentation_models

Segmentation models with pretrained backbones. Keras and TensorFlow Keras.
MIT License
4.76k stars 1.03k forks source link

Odd performance stats #358

Open asmagen opened 4 years ago

asmagen commented 4 years ago

I'm training a multi-class segmentation model based on adaptation of the tutorial from segmentation_models and the performance stats seem odd in the sense that while the predictions on the test set appear to be pretty good. Performance and example predictions on the test set: Screen Shot 2020-06-14 at 10 35 23 AM Screen Shot 2020-06-14 at 10 35 38 AM (1) the iou score seems to be better on the validation set (annotated in the figure as test) relative to the train set, which is unexpected, and (2) the loss on the validation set seems to not decrease by much, while the loss on the train set is decreasing, suggesting potential overfitting (but the performance on the test set is not bad) Screen Shot 2020-06-14 at 10 35 17 AM

Any ideas for what might be happening?

JordanMakesMaps commented 4 years ago

Do you think it could be how you're splitting your data up? (1) implies underfitting, so are you applying a lot of augmentation?