Loading data fails (silently)

[christianrickert]

Problem description

When loading saved data (ARFF) into a Trainable Weka Segmentation window, neither the image overlay (center of plugin window) nor the class lists ("Labels" section on the right) are updated. Basically, you can't revisit previous classification sessions and refine data input for better classification models.

Expected behaviour

The current documentation states that it is possible to "load the data (in Weka format) from previous traces on the same or other image or stack. Again, the plugin will check and force the consistency between the loaded data and the current image, features and classes."

Software versions

• (Fiji Is Just) ImageJ 2.1.0/1.53f; Java 1.8.0_172 [64-bit]; Windows 10 10.0
• Trainable Weka Segmentation v3.2.34

Short demo

(ImageJ macro)

// set up variables
cwd = "C:\\Users\\Christian Rickert\\Desktop";
arff = "data.arff";

// load image
run("Blobs (25K)");

// run Weka plugin
run("Trainable Weka Segmentation");
selectWindow("Trainable Weka Segmentation v3.2.34");

// create example data
makeRectangle(13, 38, 33, 37);
call("trainableSegmentation.Weka_Segmentation.addTrace", "0", "1");
makeRectangle(80, 133, 22, 55);
call("trainableSegmentation.Weka_Segmentation.addTrace", "0", "1");
makeRectangle(181, 113, 26, 26);
call("trainableSegmentation.Weka_Segmentation.addTrace", "1", "1");
makeRectangle(91, 31, 31, 23);
call("trainableSegmentation.Weka_Segmentation.addTrace", "1", "1");

// save example data
call("trainableSegmentation.Weka_Segmentation.saveData", cwd + File.separator + arff);
waitForUser("Now deleting and loading data...");

// delete example data
call("trainableSegmentation.Weka_Segmentation.deleteTrace", "0", "1", "0");
call("trainableSegmentation.Weka_Segmentation.deleteTrace", "0", "1", "0");
call("trainableSegmentation.Weka_Segmentation.deleteTrace", "1", "1", "0");
call("trainableSegmentation.Weka_Segmentation.deleteTrace", "1", "1", "0");

// load example data
call("trainableSegmentation.Weka_Segmentation.loadData", cwd + File.separator + arff);

Log output

Creating feature stack...
Updating features of slice number 1...
Feature stack is now updated.
Training input:
# of pixels selected as class 1: 2431
# of pixels selected as class 2: 1389
Writing training data: 3820 instances...
Saved training data: C:\Users\Christian Rickert\Desktop\data.arff
Loading data from C:\Users\Christian Rickert\Desktop\data.arff...
Loading data from C:\Users\Christian Rickert\Desktop\data.arff...
Read class name: class 1
Read class name: class 2
Field of view: max sigma = 16.0, min sigma = 0.0
Membrane thickness: 1, patch size: 19
Read class name: class 1
Read class name: class 2
Loaded data: 3820 instances (77 attributes)

Notice that the data is actually loaded twice - despite a single call to trainableSegmentation.Weka_Segmentation.loadData

The Trainable Weka Segmentation has been such an important tool in my current workflow that I would love to be able to revisit previous classifications to refine my models.

Please let me know, if I can be of assistance with debugging or testing updated TWS versions.

Chris

[iarganda]

Dear @christianrickert ,

This behavior is intended. First, the ARFF files store the feature vectors of the pixels associated to your training traces, not the ROI selections because ARFF do not store that information. Second, when loading the ARFF file the class labels do not get automatically changed to the ones in the file in order to avoid over-writing the current number and names of classes. If they do not correspond, you will get a message like this:

Finally, the data does not get loaded twice, it is the messages which is repeated. I can fix that ;-)

Regarding saving/loading the traces, you may want to use the ROI manager to store them and load them back.

I hope this helps!

[christianrickert]

Hi @iarganda

This behavior is intended. First, the ARFF files store the feature vectors of the pixels associated to your training traces, not the ROI selections because ARFF do not store that information.

I was afraid you might say: "intended behavior". :smile:

Second, when loading the ARFF file the class labels do not get automatically changed to the ones in the file in order to avoid over-writing the current number and names of classes.

As far as I can tell, the model takes care of the class labels. Did I say otherwise unintentionally?

Regarding saving/loading the traces, you may want to use the ROI manager to store them and load them back.

This manual approach is very, very tedious and error-prone. The workflow for manually saving traces is roughly this:

(pseudocode)

// saving
for every class:
    for every trace:
        select trace from TWS class
            add trace to ROI Manager selections
    save ROI Manager class selections (zip)

// loading
for every class:
    load ROI Manager class selections (zip)
    for every trace:
        select trace from ROI Manager
            add trace to TWS class

If we assume 25 traces for three classes, that's 150+ mouse clicks to save and 150+ mouse clicks to load the traces between sessions... instead of two button clicks.

In addition, the TWS GUI only allows to select individual traces at a time (no bulk selection from a class) and there's a visual offset when highlighting an individual trace from the classes (image is scaled, selection is not):

Unfortunately, I don't see a way to do this programmatically with a macro: Weka_segmentation.java

    // Macro recording constants (corresponding to  
    // static method names to be called)
    /** name of the macro method to add the current trace to a class */
    public static final String ADD_TRACE = "addTrace";
    /** name of the macro method to delete the current trace */
    public static final String DELETE_TRACE = "deleteTrace";

Do you think it would be possible in the future to add

    /** name of the macro method to select a trace */
    public static final String SELECT_TRACE = "selectTrace";

to the macro methods?

[mlt]

Can we reopen this? This is totally goes against POLA (the principle of least astonishment) for new users when you spend some time settings things up just to realize it can't be adjusted later.

If ARFF doesn't store this information, there should be another file that stores all that along with a reference to ARFF.

[amusa22]

Can we reopen this? This is totally goes against POLA (the principle of least astonishment) for new users when you spend some time settings things up just to realize it can't be adjusted later.

If ARFF doesn't store this information, there should be another file that stores all that along with a reference to ARFF.

As a 'relatively new user' myself, I can confirm it was quite astonishing.

[imagesc-bot]

This issue has been mentioned on Image.sc Forum. There might be relevant details there:

https://forum.image.sc/t/load-and-save-traces-in-trainable-weka/80201/1