SuperpixelSEEDS/SLIC: fatal error on little matrices

[Daniel-Alievsky]

I detected that SuperpixelSEEDS does not work on very little matrices (5x5, 10x10 etc.). It either leads to fatal error (EXCEPTION_ACCESS_VIOLATION) or causes the computer to freeze. The similar problem is for superpixelSLIC; maybe other superpixel algorithms have analogous bug (I didn't test them).

Below is a very simple test in Java; I believe it can be easily reproduced in C++.

public class SuperpixelLittleMatBug {
    public static void main(String[] args) {
        opencv_core.setUseOpenCL(true);
        if (args.length < 4) {
            System.out.printf("Usage: %s source_image target_image width height%n", SuperpixelLittleMatBug.class);
            return;
        }
        final String sourceFile = args[0];
        final String targetFile = args[1];
        final int width = Integer.parseInt(args[2]);
        final int height = Integer.parseInt(args[3]);
        final opencv_core.Mat source = opencv_imgcodecs.imread(sourceFile);
        final opencv_core.Size size = new opencv_core.Size(width, height);
        opencv_imgproc.resize(source, source, size);

        opencv_core.Mat result = new opencv_core.Mat();
        if (true) {
            System.out.println("Creating SuperpixelSEEDS...");
            final opencv_ximgproc.SuperpixelSEEDS superpixelSEEDS = opencv_ximgproc.createSuperpixelSEEDS(
                    source.cols(),
                    source.rows(),
                    source.channels(),
                    100,
                    4,
                    4,
                    5,
                    false);
            System.out.println("Iterating...");
            superpixelSEEDS.iterate(source, 4);
            System.out.println("Getting controur mask...");
            superpixelSEEDS.getLabelContourMask(result, true);
        } else {
            System.out.println("Creating createSuperpixelSLIC...");
            final opencv_ximgproc.SuperpixelSLIC superpixelSLIC = opencv_ximgproc.createSuperpixelSLIC(
                    source, opencv_ximgproc.SLICO, 10, 10.0f);
            System.out.println("Iterating...");
            superpixelSLIC.iterate(10);
            System.out.println("Getting controur mask...");
            superpixelSLIC.getLabelContourMask(result, true);
        }

        System.out.println("Writing...");
        opencv_imgcodecs.imwrite(targetFile, result);
    }
}

If width and height are extremely small, for example, 16x16, iterate method leads to fatal error in iterate() method:

EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x00007ffbc479c51a, pid=10444, tid=0x0000000000002504 Problematic frame: C [opencv_ximgproc401.dll+0xac51a]

Smaller size 8x8 leads to hung: the test prints "Creating SuperpixelSEEDS..." and freezes. SuperpixelSLIC leads to EXCEPTION_ACCESS_VIOLATION on image 5x5.

[cbalint13]

• Running superpixel segmentation for so small images 10x10 is non-sense. I am sure it leads to some NULL lists right at initialization of superpixel "growing" regions. It will have nothing to grow on such small space.

[cbalint13]

Or, by "very little matrices" you mean input image size, right ? If yes, see my previous answer.

Otherwise segmenting normal images into small e.g. 5x5 regions (superpixels) works fine, even with 3x3 regions.

[Daniel-Alievsky]

Yes, I tested with images 8x8, 16x16, 5x5. Of course, it is usually not necessary, but an application server must not crash if the user uploads such an image for processing. If your algorithm cannot do anything useful with very little image, it should return some "trivial" result, like a matrix filled by 0, but not lead to freezing computer or system crash.

[cbalint13]

@Daniel-Alievsky ,

You found a narrow case, is hard to predict what algorithm would do on such extreme scenarios.

I am sure that in some case (high color variance) even these small images will run but in other not.

The simplest way is to not allow such images, it would generate zero amount of superpixels anyway.

[Daniel-Alievsky]

The simplest way is to not allow such images, it would generate zero amount of superpixels anyway.

It is exactly that I offer you to do: check that image is small for your algorithm and return trivial result in this case. Other solution: you can include in the documentation the warning, that images less (for example) 32x32 pixels will lead to system crash, and provide a guarantee that if cols>=32 and rows>=32 errors impossible. But I believe it is much better just NOT to process such images inside your class, than to complicate documentation.

I can explain how such images appeared in our application. Your superpixel algorithms are relatively slow while processing large images (our standard frames are 2000x2000 or 5000x5000 - tiles of the panorama ~50000x50000). So, I've added little additional feature: before calling superpixel I compress the image in N times, and after calling superpixel resize the result it to original sizes. For many solutions, results are still suitable when N=2, 3 or even 5 (segmentation of reduced picture 1000x1000 instead of 5000x5000).

Of course, N is the customization parameter available for user. We created test suit of little pictures for illustrating the idea of solution, and N stayed relatively big (as for large images). As a result, the server has been frozen.