Low loss but still getting poor segmentation

[MBronars]

I am fine-tuning a model to detect neural stem cells. The cell shape is not very consistent from picture to picture and the cells will often go out of focus. The loss and intersection over union are both pretty good (.15 and .7 respectively) but the segmentation is very poor (under .1). When I try to segment using the trained model it either picks up on nothing or partially recognizes a few cells. I am confused why the performance is so good on the other two metrics and yet it still fails to segment. Do you have any suggestions for how I can get better segmentation results? Thanks!

[ThorstenFalk]

IoU is at 0.7, this is OK, 0.9 would be human level performance. If IoU and F1 segmentation deviate a lot from each other this indicates massive over/under-segmentation, i.e. adjacent cells merge or cells get segmented into multiple segments.

Out-of-Focus parts of cells are tricky to annotate consistently and then to segment properly. Can you post an example validation image with annotations and its corresponding segmentation?

[MBronars]

Here are a few images:

These are the settings I used for fine-tuning the current model.

Here are the graphs from the training.

Here is a validation image of a simple cell, works pretty well.

These are validation images of more difficult cells. Does not have good segmentation.

Segmentation results are much better for this one, but this is one of the training images, not validation.

[ThorstenFalk]

Even on the training image I would call the segmentation at best poor... Your training images seem to be quite diverse, looking at the repetitive pattern in the loss curve varying by more than factor 20. 2000 iterations are not very many especially with out-of-focus objects that must be rejected. Maybe the network just needs a little more time? The checkerboard-like segmentation also indicates that the model is not yet stabilized.

[MBronars]

I will try it again with more iterations and see how it goes. Would it help to re-finetune this partially trained model or is it best to always start from your pretrained 2d_cell_net_v0 model?

[ThorstenFalk]

You can safely continue from your model.