deeplabv3plus for binary segmentation

[tirk999]

I am new to Keras so sorry if the question is silly.

I found here https://keras.io/examples/vision/deeplabv3_plus/ the deeplabv3+ model to perform multiclass semantic segmentation. I need to adapt this code to another purpose, because I need to perform binary semantic segmentation on medical images. Is it correct to change from

NUM_CLASSES = 20

to NUM_CLASSES = 1?

If I put NUM_CLASSES = 2, I get an error about mismatch between logits and labels.

About the loss function, the code line is loss = keras.losses.SparseCategoricalCrossentropy(from_logits=True)

I thought to change it to

loss = keras.losses.BinaryCrossentropy(from_logits=True)

but loss becomes negative. Should I add something else?

Thank you!

Edit: the deeplabv3+ for multiclass semantic segmentation uses keras.activations.linear(x) in the last layer. For my purpose, should I use softmax instead of keras.activations.linear(x) with BinaryCrossEntropy and put from_logits=False?

[sayannath]

@soumik12345 can help you out with this.

[tirk999]

@soumik12345 I have some fresh news: I put

• NUM_CLASS = 1
• I added a sigmoid as the activation function in the last layer of the net
• I put loss = keras.losses.BinaryCrossentropy(from_logits=False)

My data is made by images of cells and their ground truth (binary masks). I normalized both in range [0-1] Modifying accordingly the part about model.predict() ecc, I actually get a mask as an output (not perfectly binary). So far, I used a small dataset of 1000 images for training and 4 for validation (it's just a small part of a bigger dataset) to understand if things were going okay. Loss is positive and decreases, accuracy increases instead of decreasing (which happened before). Do the changes sound reasonable to you?

The part that still confuses me somehow, in theory, is the NUM_CLASS = 1 instead of 2. In the example of multiclass segmentation, I read 19 classes of objects (if I recall correctly) so I suppose 20 included the background, but I think the encoding of ground truths was different than in my case. I didn't try to run the code after my changes with NUM_CLASS = 2, but I supposed that it wouldn't run anyway (because the problem was about logits and labels not matching) I will try tomorrow though. Thank you!

[soumik12345]

Hi @tirk999 , for binary segmentation, it's preferable to keep NUM_CLASS = 1 since you're trying to predict a binary mask that represents a single class against the background. If you wish to predict a one-hot-encoded segmentation mask (in the current context, setting NUM_CLASS = 2), only then use softmax activation along with sparse categorical cross-entropy loss otherwise use sigmoid activation along with binary cross-entropy.

[tirk999]

Hi @tirk999 , for binary segmentation, it's preferable to keep NUM_CLASS = 1 since you're trying a binary mask that represents a single class against the background. If you wish to predict a one-hot-encoded segmentation mask, only then use softmax activation along with sparse categorical cross-entropy loss otherwise use sigmoid activation along with binary cross-entropy.

Perfect, thank you for solving my doubt, this was the thing bothering me. Thank you again!

[soumik12345]

@tirk999 happy to help :) Please close the issue if you're doubt has been cleared.

[felipetobars]

Did you get values between 0 and 1 in te new bin mask? Or only 0 or 1?

[tirk999]

Did you get values between 0 and 1 in te new bin mask? Or only 0 or 1?

Hi Felipe, I got values between 0 and 1 as output from the neural network. Then I binarized to obtain only 0 or 1.

[felipetobars]

Did you get values between 0 and 1 in te new bin mask? Or only 0 or 1?

Hi Felipe, I got values between 0 and 1 as output from the neural network. Then I binarized to obtain only 0 or 1.

Thanks, Can I ask you... how big was your data set and if you showed overfitting with the Resnet50 backbone model? Did you apply any regularization?

[tirk999]

Did you get values between 0 and 1 in te new bin mask? Or only 0 or 1?

Hi Felipe, I got values between 0 and 1 as output from the neural network. Then I binarized to obtain only 0 or 1.

Thanks, Can I ask you... how big was your data set and if you showed overfitting with the Resnet50 backbone model? Did you apply any regularization?

I used about 8500 images for training and about 2100 for validation. Moreover, I applied real time data augmentation, but if I remember correctly the network worked even before real time data augmentation. In my case, the ResNet50 backbone worked just fine and I did not have any overfitting problems; I didn't apply any regularization.

[felipetobars]

Did you get values between 0 and 1 in te new bin mask? Or only 0 or 1?

Hi Felipe, I got values between 0 and 1 as output from the neural network. Then I binarized to obtain only 0 or 1.

Thanks, Can I ask you... how big was your data set and if you showed overfitting with the Resnet50 backbone model? Did you apply any regularization?

I used about 8500 images for training and about 2100 for validation. Moreover, I applied real time data augmentation, but if I remember correctly the network worked even before real time data augmentation. In my case, the ResNet50 backbone worked just fine and I did not have any overfitting problems; I didn't apply any regularization.

okay, got it. My data set is not that large, do you remember what loss and accuracy values your models reached?

[Harshu2032000]

i'm also doing binary segmentation on retina images to detect the blood vessels using deeplabv3 but i'm getting output in ranging from 0 to 255 why is it so?

[cmk-30]

Hi @tirk999 , for binary segmentation, it's preferable to keep NUM_CLASS = 1 since you're trying to predict a binary mask that represents a single class against the background. If you wish to predict a one-hot-encoded segmentation mask (in the current context, setting NUM_CLASS = 2), only then use softmax activation along with sparse categorical cross-entropy loss otherwise use sigmoid activation along with binary cross-entropy.

Hey Soumik, I was trying the same. I couldnt find any linear activation layer as mentioned, in the last layer. In the output (final) Conv2D, I tried adding activation=sigmoid but that did not make the loss positive either. Where are you suggesting to use the sigmoid activation exactly?