confused about addBinaryData step in image classification

[aubreyyan]

Here is the test code we are working with just as a start. The end use case for weka segmentation is as part of a backend service that can handle image classification requests from a frontend application, so trainablesegmentation would just handle the processing from a backend Spring Boot service. (just mentioning this so if there are any foundational issues with what we're trying to do, we get that out of the way early)

import ij.IJ;
import ij.gui.Roi;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import trainableSegmentation.WekaSegmentation;

import java.io.IOException;

@SpringBootApplication
public class AutomatedClassificationApplication {

    public static void main(String[] args) throws IOException {
        WekaSegmentation ws = new WekaSegmentation(IJ.openImage("C:\\Users\\Aubrey\\Downloads\\Untitled.JPG"));
        ws.setNumOfClasses(1);
        ws.setClassLabels(new String[] {"class 1", "class 2"});
        ws.addExample(0, new Roi(50, 50, 50, 50), 1);
        ws.addExample(1, new Roi(100, 100, 50, 50), 1);
        ws.addBinaryData(IJ.openImage("C:\\Users\\Aubrey\\Downloads\\Untitled.JPG"), IJ.openImage("C:\\Users\\Aubrey\\Downloads\\Untitled.JPG"), "class 1", "class 2");
        ws.trainClassifier();
        ws.applyClassifier(0, false);
//        new GetObjectRequest("", "");

        SpringApplication.run(AutomatedClassificationApplication.class, args);
    }

}

Obviously the final version of the code won't look like this but we're just trying to understand the ImageJ api at the moment so main is our vehicle for doing so.

Terminal Output

``` Initializing loaded data... Added 239766 instances of 'class 1'. Added 0 instances of 'class 2'. Training dataset updated (239766 instances, 77 attributes, 1 classes). Creating feature stack... Updating features of slice number 1... Feature stack array is now updated (1 slice(s) with 76 feature(s), took 4566ms). Loading Weka properties... WARNING: core mtj jar files are not available as resources to this classloader (jdk.internal.loader.ClassLoaders$AppClassLoader@16f65612) Training input: # of pixels selected as class 1: 2500 Creating training data took: 1775ms Merging data... Finished: total number of instances = 242266 Training classifier... FastRandomForest of 200 trees, each constructed while considering 2 random features. Out of bag error: 0% Finished training in 12010ms Classifying whole image using 16 thread(s)... Classifying whole image data took: 153ms Finished segmentation of whole image. ```

So I can confirm that this code "works", we just don't quite understand it.

We have looked into your website for some help https://imagej.net/plugins/tws/scripting#Example:_define_your_own_features

but I don't understand the nature of addBinaryData. It seems like we want to create tif files somehow that are just black and white, where "white pixels belong to one class and black pixels belong to the other class". But it is my understanding of ImageJ that images can have more than 2 classes right?

So in the case where there are more than 2 classes, but white and black can only work for the first two, how do we accomplish this step of the script?

Also, is there any example of how to produce such a tif file(s)? I'm also confused why it's even necessary if we've already provided example bounds with class assignment, how does the tif file, assuming it's just visually representing those examples in black and white, add any value?

Thanks so much

[iarganda]

Hello @aubrey-y,

addBinaryData is indeed expecting two images: the input (grayscale or RGB) image and its corresponding binary label image. The binary label image in this case is just black and white to assign white pixels to one class and white ones to the other class. So yes, this is used when there is only two classes.

For more classes, you should use any of the addLabeledData methods, for instance this one: https://javadoc.scijava.org/Fiji/trainableSegmentation/WekaSegmentation.html#addLabeledData-ij.ImagePlus-ij.ImagePlus-

That method receives the input image and a label image. The pixel values of the label image are the indexes of the classes they belong to. That way, you can use more than two classes.

That being said, you should set the number of classes to the right number. In your code, you are setting it to 1 :)

I hope this helps!

[aubreyyan]

I'm assuming that addBinaryData provides some advantages compared to addLabeledData, given of course there are only two classes?

[iarganda]

It is just another method to do it. Binary classification is usually very popular, so it has its own methods.

[aubreyyan]

The pixel values of the label image are the indexes of the classes they belong to. That way, you can use more than two classes.

oh and can you explain this a bit more? pixel values are (r,g,b) right? assuming the indexes of my class are 0, 1, 2 what pixel values would they correspond to?

[iarganda]

The input image can be grayscale or RGB, but the label image should be a grayscale image, so every value corresponds to an index.