Nuclear and whole cell segmentation masks identical

[megleila]

Hi VanValen lab,

I have been struggling to identify why my nuclear and whole cell segmentation masks are identical for my IMC data. I have been obtaining my files directly through a cloud-based Jupyter notebook using source code with the help of a bioinformatics specialist.

I use a phospholipid marker plus immune cell cell surface markers together to identify membrane, and two DNA channels identify DNA. I am working in kidney tissue.

Code screenshots below along with a portion of the whole cell segmentation and nuclear segmentation outlines:

[megleila]

After wondering about this for a long time I think I just came across the answer on your site -- "Finally, if your data doesn't include in a strong membrane marker, the model will default to just predicting the nuclear segmentation, even for whole-cell mode. If you'd like to add a manual pixel expansion after segmentation, you can do that using the pixel_expansion argument. This will universally apply an expansion after segmentation to each cell". What defines a strong membrane marker that is different than our markers?

[ngreenwald]

Hi @megleila, that is definitely an issue that comes up. However, when poor membrane staining is the root cause, we generally see the two segmentations looking quite similar to each other, not literally identical as you showed here. I'm wondering if somewhere in your prediction pipeline, the inputs are getting swapped, so the same thing is being predicted both times?

Can you rerun your notebook above, but using the example data from the application notebook. This will allow us to confirm that everything is working correctly for your setup.

[megleila]

The bioinformatics analyst on our team did this and obtained identical outputs. I just looked more carefully at our code and cleaned up some areas to match the source code and now am having a different issue. With all default parameters set, whole cell segmentation and nuclear segmentation are now different, but whole cell segmentation actually looks to be masking smaller objects than the nuclear mask -- and nuclear mask seems to be masking much larger than the nucleus. I've attached the script. If you have any ideas why this might be happening, would be really helpful.

If whole cell segmentation simply isnt working well for our data with our membrane markers for whatever reason, we can use pixel expansion of the nuclear segmentation. But I am a little perplexed by the nuclear segmentation as well, as it is outlining well beyond nuclear borders for many cells.

Nuclear:

Whole cell:

Mesmer_for_Megan.zip

[ngreenwald]

Hi @megleila, Same suggestion as above. I think the first step is to make sure your code reproduces the outputs from the example dataset. If you're seeing that the whole-cell segmentation on the example dataset looks good, but the whole-cell segmentation on your data looks off, then it's likely to be an issue with the staining.

However, it sounds to me that it's more likely to be a swap between the outputs somewhere.