Dice coefficient no change during training

[Bienqq]

Hello,

I am trying to use Your implementation of UNet in segmentation of medical images (1 class, cell surface or background)

I have changed only paths to images/masks directory

I have following images of cells under the microscope:

1376x1038 RGB image

And for each image I have corresponding mask - it is binary image with one channel which has only black/white pixels

However during further training Dice coefficient does not change at all it has very low value all the training time

For example: INFO: Validation Dice Coeff: 2.9824738678740914e-08

Could You please help? Thanks

[milesial]

Hi, that's strange. How many images are you using for validation rounds ? Did you check that your masks only have 0-1 values and not 0-255 ?

[Bienqq]

Hello, I have small dataset only 40 images so when val=10% it will be 36 train images and 4 validation images. Could this be the reason?

I have checked mask values ( in PyCharm SciView) and it has only values 0 or 1 saved as float points number. It seems to be correct.

[milesial]

Maybe if you have very few validation images and all the predictions are wrong, then you would get the same value everytime (which is not quite 0 because of an epsilon in the computation).

Your value is basically zero, so that could be it, maybe the model's output is always zeros because it has not learned anything yet.

Can you try manually checking the predicted masks and the true masks in the validation round ? If there is some overlap, then the dice coefficient should be higher.

[KarlCMB]

I experience the same issue. Using a custom dataset as well as the one with cars, I only receive white background. Dice Coefficient changes from permanently close to zero up to 93 for all epochs. Epoch 1/5: 11%|██████████████████▉ | 10/90 [02:59<23:07, 17.35s/img, loss (batch)=0.82INFO: Validation Dice Coeff: 0.9046485900878907 Epoch 1/5: 22%|█████████████████████████████████████▌ | 20/90 [05:23<12:09, 10.42s/img, loss (batch)=0.211INFO: Validation Dice Coeff: 0.9046485900878907 Epoch 1/5: 33%|████████████████████████████████████████████████████████▎ | 30/90 [07:49<11:28, 11.48s/img, loss (batch)=0.561INFO: Validation Dice Coeff: 0.9046485900878907 Epoch 1/5: 34%|██████████████████████████████████████████████████████████▏ | 31/90 [08:34<20:06, 20.45s/img, loss (batch)=0.275 I checked TensorBoard and get the white only prediction there as well during validation. Any tips?

[Branikas]

Hi guys, I ve had similar results when trying to train small datasets at first. It is extremely hard to train the network with a few dozens of images, specially when your image has high resolution. What I suggest you do is take patches of the images. For example I used shifting windows that overlap lets say of size 48 by 48 pixels (5K patches per image) and used those as inputs. If you have an unbalanced dataset (many pixels are background and only a few the target class) you can do a sampling of the patches so that you have lets say 50% of the patches having pixels belonging to the positive class. Hope that helps.

[yyyzzzhao]

hi, has this problem been resolved? I encounter this problem too. ~ 250 tranining images and 25 validation images, 5000 training steps. the dice of validation set is always small (~1e-8) and the output image is always all black.

[loveq007]

So many people have encountered this phenomenon,Perhaps there is a problem with the training strategy in the code