Spindle: Exclude bright non-mitotic DNA objects from DNA mask

[TobiasKletter]

Bright DNA signals in close proximity to metaphase plate are often included into DNA mask, leading to wrong downstream measurements.

[tischi]

Ok! Makes sense.... just keep a few example data sets that show this problem and next time you come we will check whether we can fix this.

[tischi]

Hi @n-dempe, Could you please post here a download link to one (just one) representative example image file where there is extra DNA detected (leading, among other things) to overly long spindle lengths.

[tischi]

Hi Nils, Could you somehow manage to put a down-load link to one example file here? As said, having such information in e-mails makes it very hard to find back later...

[TobiasKletter]

https://www.dropbox.com/s/4t37ilgy5ri2rnc/190413-HeLa_WT_Hoechst_500nMSiRTubulin_MG132_60x_029Cropped_2.tif?dl=0

[tischi]

This is an interesting image. Is that later in the development?

1. What are the bright dots in the tubular channel?

2. Is it normal that the spindle tubulin is quite dim? It seems not brighter than the non-spindle tubulin in the other cells...I think in the other images it is not like this, is it?

[TobiasKletter]

It's not a cell from our stem cell system, it's a HeLa Cell stained with SiR Tubulin and Hoechst dyes. That's also why there's not that much of a difference between spindle microtubules and the non-spindle filaments... The morphology is also a different compared to the mouse stem cells - much more "round" :) The dots in the tubulin channel could be some non-dissolved aggregates from the dye, not sure though.

[TobiasKletter]

@tischi Related issue that is super annoying to me but also super specific to my experiments, so it might not be of the highest priority in the greater scheme of things:

In many of my images there is a lot of junk from dead cells in the DNA channel and usually brighter than the actual mitotic DNA:

https://www.dropbox.com/s/fn12oftuzddmgx3/Apoptsis-HighZoom--W0000--P0002-T0322.tif?dl=0

So, not surprisingly, the dead-cell DNA is "chosen" as the main DNA object, leading to faulty downstream measurements:

https://www.dropbox.com/s/5k2yz35pqmw82cy/Apoptosis-HighZoom--W0000--P0002-T0322-out.tif?dl=0

My only idea to solve this would be to request that, next to the DNA object, the spindle object in the other channel has to be at the right position as well (to license the DNA plate as such) but that would be a chicken and egg problem, I guess? Or could this be an iterative process, going back and forth between the channels and looking for the actual spindle?

[TobiasKletter]

Concatenated examples: z as t: https://www.dropbox.com/s/72i6qjdy2b4ne41/Apoptotic-concat-1.tif?dl=0 z as z: https://www.dropbox.com/s/zwhzvs726hexjr0/Apoptotic-concat-2.tif?dl=0

[tischi]

[TobiasKletter]

@tischi CATS-based spindle detection: Just wanted to add to our discussion that the CATS detection step might in general be super useful also for users who don't have cropped and central spindles in their images in the first place... And also useful possibly regarding multiple spindles in the field of view?

[tischi]

@TobiasKletter It looks like we did not solve this yet, as this is causing the crash in the examples that you posted in the other thread:

[tischi]

@TobiasKletter Are you already using CATS for finding the metaphase cells during the microscopy? Maybe we would also need to use it here...

[tischi]

@TobiasKletter I could try using CATS within our workflow. I would need example images from you with

1. exactly 2 channels, DAPI and Tubulin (always in the same order)
2. an isotropic voxel size of 0.25 um (below is an IJ macro to do this)
3. ideally they would also all have the same number of pixels in x,y,z.
4. if you manage (3) it would be great if you could combine them into a "fake movie" (5D Hyperstack). That would make the training of CATS easier.

5-10 images would be sufficient to give it a go.

run("Close All");

targetVoxelSize = 0.25 // um

inputDir = "/Users/tischer/Desktop/3d_Plugin_Problematic_Images";

files = newArray(1);
files[ 0 ] = "R1ENestin(h)mCherry_doubletag_Unfacs_Tubb3KO_clone16_046_Cropped_1_ExceptionDuringComputation.tif";
//files[ 0 ] = "aaa.tif";

for (i = 0; i < files.length; i++) 
{
    path = inputDir + "/" + files[ i ];
    print( "Opening: " + path );
    open( path ); 
    rename( "image" );
    scale( "image" );
}

function scale( name ) {
    selectWindow( name );

    getVoxelSize(width, height, depth, unit);
    scaleX = 1.0 * width / targetVoxelSize;
    scaleY = 1.0 * height / targetVoxelSize;
    scaleZ = 1.0 * depth / targetVoxelSize;
    getDimensions(width, height, channels, slices, frames);
    newSlices = slices * scaleZ;
    print( scaleZ );
    run("Scale...", "x=&scaleX y=&scaleY z=&scaleZ depth=&newSlices interpolation=Bilinear average create");
    rename( "scaled" );
    selectWindow( name ); close( );
    selectWindow( "scaled" );
    rename( name );
}

[TobiasKletter]

@TobiasKletter Are you already using CATS for finding the metaphase cells during the microscopy? Maybe we would also need to use it here...

No, I started the training but the predictions were not good enough...Trained on max projections though, still need to test whether it's doing better when using the small stack we're actually recording.

5-10 images would be sufficient to give it a go.

On it!

[TobiasKletter]

@tischi https://www.dropbox.com/s/rv8lngp10h00err/Isotropic_spindles_for_CATS.tif?dl=0

[tischi]

You think this has highest priority or something else of the open issues?

[TobiasKletter]

I'd say it's a tie with "Spindle Length overestimation"

[tischi]

ok. I'll see...

do you know the algorithm that you guys used to automatically find those cells on the microscope? apparently is worked well enough to find them ?!

[TobiasKletter]

We're at the moment using an Ilastik-model (random forest) that looks at max projections of the tubulin channel only.

But didn't we find this "one-decision" tree computation in CATS that worked really well for recognising tubular shapes such as chromatin arms?

[tischi]

Yes. Should be no problem! I was just wondering whether you guys had something without machine learning.

[TobiasKletter]

Before Ilastik it was: Max proj of tubulin signal -> median filter -> fixed threshold for segmentation -> feature extraction of segmented objects -> decision tree to say "true metaphase" or "other"

[tischi]

@TobiasKletter

1. Did you sample the images to 0.25 isotropic before trying the classifier?
2. Classifier requirements are: 0.25 isotropic pixel size, ch1=tubulin, ch2=dna, bitdepth=8
3. Does the classifier work for you those images? https://www.dropbox.com/s/rv8lngp10h00err/Isotropic_spindles_for_CATS.tif?dl=0
4. If all above are a yes, could you send me another 10, composed of some of the examples that did not work for you? As diverse as possible please in terms of intensities.

[TobiasKletter]

I forgot to do the re-sampling...

This is how it looks now:

1. Seems to have some problems with super-bright and super-dim samples, but the rest looks promising! I selected 6 new samples from the hole intensity spectrum end concatenated them with the original training set: https://www.dropbox.com/s/3hrcrj1ikjfxeoi/Isotropic_spindles_for_CATS_20191202.tif?dl=0

2. Right now, it's only looking at the DNA signal, isn't it? Should we include the other channel, or maybe only use the spindle channel, since it's more predictable?

[TobiasKletter]

@tischi 16-bit set: https://www.dropbox.com/s/l8oid445lk4ncyu/16bit_Isotropic_Spindles_for_CATS_20191202.tif?dl=0

[tischi]

Thanks!

1. From which microscope was the first (now extended) data set that you posted here?
2. The 16 bit one is from a spinning disc, correct?

Regarding your question:

Nope, it is looking at both channels; I am still confused whether we should restrict it to only look at the tubulin channel. Somehow I feel we should keep looking at both...

[TobiasKletter]

From which microscope was the first (now extended) data set that you posted here?

From the Zeiss LSM 800

The 16 bit one is from a spinning disc, correct?

Yes

Nope, it is looking at both channels; I am still confused whether we should restrict it to only look at the tubulin channel. Somehow I feel we should keep looking at both...

Weird, because in the output probabilities it kind of looks like it only tries to segment the DNA, no? But yes I also think that looking at both should be a good strategy.

[tischi]

Weird, because in the output probabilities it kind of looks like it only tries to segment the DNA, no? But yes I also think that looking at both should be a good strategy.

This is on purpose, because I trained it t find the metaphase plate region. This is enough to roughly know where the cell is. I think I only need the rough centre of the cell, to exclude the other, dying cells, etc. My plan was to exclude all pixels that are more than X micrometer away from the centre of the segmented structure.

[TobiasKletter]

@tischi Finished training an ilastik model that I'm pretty happy with. For now, I only trained on the 8-bit dataset. It needs both DNA and tubulin channels (works much better than DNA-only). In the first stage, it classifies "Metaphase DNA", "Background", "Apoptotic", "Spindle" and "interphase DNA", in the second stage it only decides between good / bad. Here is the link to the file: https://www.dropbox.com/s/euccqgtxkl5uet3/20191206_DNA_Segmentation_2Ch.ilp?dl=0

(The training set should be embedded into the file. I used Ilastik 1.3.2. in case that matters)

Here's a little summary of the quality control so far. I sorted the images of my subsample into 4 categories according to their gray values in the mitotic DNA manually set a threshold for segmentation (average of 3) and measured the volume of the resulting binary. This seemed to work well for categories 1 - 3, but overestimated overexposed images. The panel in the middle shows the volumes as quantified by the current version of the plugin. The right panel shows the output of the ilastik model, which seems to cope well and with all brightnesses and also shows less spread than the other two.

There seems to be a small trend towards larger volumes in brighter DNA but this could still be small sample size (category 3 is only 3 images...) so I'm going to test a few more (and also 16-to-8bit converted ones).

[tischi]

Can you please try whether your project file also works with ilastik 1.3.3? I tried to run ilastik 1.3.3 and it crashed. I'll send you the file by mail.

[TobiasKletter]

Can you please try whether your project file also works with ilastik 1.3.3? I tried to run ilastik 1.3.3 and it crashed. I'll send you the file by mail.

For me it works fine on both versions. The problem seems to be that I used files that were already re-sampled to 0.25 µm voxels. I uploaded some of these here: https://www.dropbox.com/s/lrslhp7wtk37vwu/Isotropic_input_for_Ilastik_DNA.zip?dl=0

[tischi]

ah, and the file I sent to you by mail was not re-sampled?!

then it was my mistake!

[TobiasKletter]

Hi @tischi Small progress, here is some fiji-embedded Ilastik classification:

rootDir = "/Volumes/TOB_WD1/Ilastik/Ilastik_model_DNA";

inDir = rootDir + "/" + "FIJI_Input";
outDir = rootDir + "/" + "FIJI_Output";

project = rootDir + "/" + "20200121_DNA_Segmentation_1Ch.ilp";
outputtype = "Probabilities"; 

print("Classifier file: "+project);

fileList = getFileList(inDir);

setBatchMode(true);
for (i = 0; i < fileList.length; i++) {
    fileName = inDir + "/" + fileList[i];
    classifierArgs = "projectfilename=" + project + " inputimage=" + fileName + " pixelclassificationtype=" + outputtype; 
    open(fileName);
    print("Processing next: " + fileName);
    run("Run Pixel Classification Prediction", classifierArgs);
    close(fileList[i]);
    run("Duplicate...", "duplicate channels=1");
    saveAs("tiff", outDir+"/ProbabilityPrediction_"+fileList[i]);
    print("Saved probability prediction to :" + outDir);
    run("Close All"); 
}
setBatchMode(false);
print("Finished analysis.")

It works nicely with classifiers that take input images with 1 channel only. The classifier (see posts above) that segments the best, however, uses both channels and this crashes for some reason (with or without macro...). I posted the issue on the forums, let's see...: https://forum.image.sc/t/ilastik-pixel-classification-prediction-in-fiji/33182

[TobiasKletter]

@tischi Good news: The ilastik FIJI plugin was updated and the 2-channel project (https://www.dropbox.com/s/euccqgtxkl5uet3/20191206_DNA_Segmentation_2Ch.ilp?dl=0) can now be used and is working within the macro I posted. Sadly, it's a bit slow....

[tischi]

Is above file still up to date?

[tischi]

...and does the first channel need to be DNA or tubular?

[TobiasKletter]

Channel 1 = Tubulin, Channel 2 = DNA

[TobiasKletter]

Concerning the problematic case here:

it seems to be confused by some unusually bright interphase neighbour nucleus:

I could try to fix this by re-training, but wouldn't it generally make sense to exclude border objects? (or, when in doubt, take the object that's closer to the center?)

[tischi]

I got it running now, but it is horribly slow. So slow that I cannot really debug/develop the down-stream part.

1. Is there any way to increase speed of the ilastik classification? Use less features? Use less trees?
2. Maybe it is worth trying the 3D Weka Segmentation in Fiji?
3. @haesleinhuepf how far are you with GPU accelerated trainable segmentation? 😄

[haesleinhuepf]

Hey @tischi ,

3. @haesleinhuepf how far are you with GPU accelerated trainable segmentation? 😄

applyOCLWekaModel lives in CLIJx at the moment. For now it can only be applied to 2D data. You can test it by activating the clij2 update site in Fiji. Last tests showed a speedup by about factor 450 compared to classic Weka Trainable Segmentation. 😎

To give it a try: https://github.com/clij/clij-advanced-filters/blob/master/src/main/macro/weka.ijm#L92-L105

Let me know if it helps!

Cheers, Robert

[TobiasKletter]

CLIJ + Weka 2D could be nice for our online detection ... :)

Is there any way to increase speed of the ilastik classification? Use less features? Use less trees?

I can try to set up a fast/minimal ilastik workflow, but I expect that to work only with really nice, un-messy images :( Same goes for 3D Weka... I'll give them a shot nevertheless and post an update here!

[tischi]

@haesleinhuepf

1. Did you already implement the RF on GPU?
2. Making it work on 3D data should be in reach?!

[haesleinhuepf]

• Did you already implement the RF on GPU?

Training runs on the CPU using Weka. After training the model is translated to opencl code.

2. Making it work on 3D data should be in reach?!

In reach, yes. I'd estimate a workload of days or weeks. The problem is limited memory in typical GPUs.

Alternative: Program a custom workflow which generates Feature-stacks on the GPU, pulls them to the CPU and then processes them slice-by-slice on the GPU. A basic script doing that should be programmable in a day.

What options do you use in your use-case for feature-stack generation? Which filters?