Predicted images are getting worse while the metrics are still improving

[Gpetrak]

I'm trying to train a U-net with VGG16 as a backbone in order to recognize 4 classes: sky, rocks, trees and background in a dataset of about 10000 images. I'm using categorical crossentropy loss function and IOU / F1-score as metrics. The weird thing is that if I train the model for 20 epochs, the loss function and metrics are improving smoothly with the corresponding validation values. The model reaches about 90% of accuracy (with loss function about 0.02) and it seems that the network doesn't overfit. However, after the training, the predicted images are black. On the other hand, when if I train the model for about 5-7 epochs where the loss function had a value of about 0.20 and IOU/F1-score about 0.40, the predicted images are pretty much better, achieving an IoU of about 0.80 - 0.90.

How would you interpret this ? Is the issue related with the unbalanced size of the classes in the image ? I mean that sky and background, cover much bigger area in the images than rocks and trees. Any thoughts ?

[immuhammadadil]

Getting the same issue

[nataliameira]

Hello @Gpetrak , @immuhammadadil

Did you manage to solve this problem?

[Gpetrak]

Hello @nataliameira,

No, unfortunately I didn't solve the problem with segmentation_models. The problem that I had, solved when I developed the model on my own.