Segmentation Dataset Verification Fails

[AmrataRamchandani]

Hi, I am working on UC Merced Landuse dataset for segmentation, I have segmented images which are psuedo colored, and they are annotated according to the class labels

But still, I am getting the following error AssertionError: The pixel values of seg image should be from 0 to 16 . Found pixel value 128

I have checked the image by converting it to numpy array, the max value is no where 128, infact max pixel value is 11 in that particular image

[Adelost]

What output do you get from np.unique if you do "unique, counts = np.unique(seg_image, return_counts=True)"?

[Adelost]

Are you sure the path you are giving to the train method only contains one image and not multiple images?

You can also try setting "verify_dataset=False", though I would not recommend it since it obviously detects something. Perhaps you can set a breakpoint in "the verify_segmentation_dataset" method and look what it is finding? It looks like it does a np.max() on all images.

[sainatarajan]

@Adelost Even I am having this same issue. I have 3 + 1(background) classes in total. This code: unique, counts = np.unique(image1, return_counts=True) returns [0 1 2 3] [12371241 1835322 525945 492]. But still I get the error: AssertionError: The pixel values of seg image should be from 0 to 3 . Found pixel value 4. But there is no pixel with value as 4.

[sainatarajan]

So I have a workaround with this. When I set my n_classes parameter when defining the model to 2 more than the actual number of classes(incl. background), the training starts normally and also the predictions are good. @divamgupta can you tell what could be the reason for this?

[Adelost]

@sainatarajan Ok, strange. Yeah pixel-values seem fine from your output. If it works with n_classes set to 5 I would have expected at least one image have the pixel value 4.

Did you check all of your images (in case there is one that is corrupt among them)? If so, could you perhaps share one of the images that it is rejecting. Which version of the repo are you using?

You could also try to run all of your images through a np.max(), and check if anyone is returning a 4, since that is my understanding of what the source code is doing under the hood.

[sainatarajan]

@Adelost So I tried to do np.unique() on my previous set of images and found that 10-20% of the images had values more than n_classes. That is, some pixel values are more than the actual number that should be.

There is a long story to this issue which I had discovered recently and I have not yet figured out a decent solution to solve this issue. My annotations (polygon coordinates) are read from a .json file and are passed to OpenCV to draw the contours polygons as masks. In this way, I create my ground truths for my images. The problem is that some of the pixels near the border of a polygon have values that are neither the pixel value the polygon should have nor the background value. The values are a random color similar to the mixing of two-pixel values. Ex: Blue + Green = Yellow.

I tried PIL, imageio and another library to write the images to disk along with OpenCV. The number of pixels that have improper class values varied when I used all these libraries. Some of these pixels were even inside the polygon mask of an image.

What I have done until now is to replace the values of those pixels to black while constructing the ground truths from the .json file.

[Adelost]

@sainatarajan Ok great, then we know that the problem is caused by the artifact pixels.

Are you saving the masks as .jpg? If so you should switch to .png as jpeg creates small image artifacts similar to what you are describing.

Or are the images artifacts created by cv2 when you draw the mask-contours? I do not know OpenCV that well myself, but maybe there is a anti-aliasing setting that needs to be turned off in order to conserve the edges.

[sainatarajan]

@Adelost Yes, I was saving those masks as .jpg files. I had switched over to .png sometime back and now it's okay. The artifacts don't appear when I use .png files. It seems this is due to the jpeg lossy compression algorithms. The artifacts were created by cv2.imwrite() when I save the image as .jpg file after drawing the contours. Either perform thresholding techniques to remove those pixels, but it can become difficult when more colors are involved or as you said one can use anti-aliasing turned off (I have not tried this) or the best thing to do is to save as png file.

[Adelost]

Great, problem solved then. :)

[JaledMC]

I had the same issue than @AmrataRamchandani, making my dataset with Labelme, because it uses Pillow to save images, while unet model from @divamgupta uses opencv to load them. If you run labelme2voc.py from Labelme, the obtained annotation is something like this:

At first sight, it's obvious that something is wrong: if you have, for example, 12 classes in your dataset, your maximum pixel intensity is 11 (one value per class). But this red circle is so bright (in fact, 128). Moreover, in images with 2 different classes, there are different colors, revealing that Pillow stores data in the three RGB channels:

Loading this images with cv2.imread and np.max, it can be seen that each region use a different channel with 128 pixel values. However, with PIL.Image, max returned value is 2! :o

The solution? Save images with opencv. If you use Labelme, in labelme2voc.py, override lblsave function with:

def lblsave(filename, lbl):
    if osp.splitext(filename)[1] != '.png':
        filename += '.png'
    # Assume label ranses [-1, 254] for int32,
    # and [0, 255] for uint8 as VOC.
    if lbl.min() >= -1 and lbl.max() < 255:
        # lbl_pil = PIL.Image.fromarray(lbl.astype(np.uint8), mode='L')
        colormap = label_colormap(255)
        cv2.imwrite(filename, lbl)
    else:
        raise ValueError(
            '[%s] Cannot save the pixel-wise class label as PNG. '
            'Please consider using the .npy format.' % filename
        )

Hope this helps.

[jmzelectronica]

@JaledMC I am getting the same issue, I am using LabelMe for convert my json file to mask, do you have the full code for convert it?

[JaledMC]

Hi @jmzelectronica, and excuse my long delay,

Once you have clone Labelme, follow their Semantic segmentation tutorial to tag your dataset. Once you have been finished, you have to put your folder with the images and annotations in labelme/examples/semantic_segmentation. Before run labelme2voc.py, one easy (but dirty way), to use opencv and solve the problem is pasting this code at the beginning of the file:

import cv2
def lblsave(filename, lbl):
    if osp.splitext(filename)[1] != '.png':
        filename += '.png'
    # Assume label ranses [-1, 254] for int32,
    # and [0, 255] for uint8 as VOC.
    if lbl.min() >= -1 and lbl.max() < 255:
        # lbl_pil = PIL.Image.fromarray(lbl.astype(np.uint8), mode='L')
        colormap = label_colormap(255)
        cv2.imwrite(filename, lbl)
    else:
        raise ValueError(
            '[%s] Cannot save the pixel-wise class label as PNG. '
            'Please consider using the .npy format.' % filename
        )

And then change the line 85 from: labelme.utils.lblsave(out_png_file, lbl) to
lblsave(out_png_file, lbl)

Hope it helps. If you have any question, feel free to ask again

[A-V-Jagannathan]

I had the same issue but resolved it. my segmentation mask had pixels of value [0,6] while it should be [0,4]. what i did was take an image containing all classes , changed all pixels with value 1 to 0, 2 to 0 and so on... then eventually found pixel 2 and 4 were actually supposed to be 0. so i changed every pixel with value 2,4->0 ; 3->2; 5-> 3 ;6->4 , on every image by runnin loops on img array and by using os.listdir(), i was able to access images one by one and change them using a function.