Cerberus tile inference `pclass_map` PNG files empty

[jjhbw]

Hi TIA team, thanks for this impressive work! We are eager to use it for the model interpretability chapter of our new paper.

I am working on getting Cerberus inference to work on my lab's tile dataset. We are feeding it 500x500 pixel tiles that have been downscaled to ~0.5 MPP. The nuclei, gland, and lumen predictions look fine.

We are also very interested in the patch-level tissue type classification of Cerberus. These predictions are written to the pclass directory, right? The pclass output PNG files that are produced look empty (all black). In an attempt to debug this, I tried changing the below lines to output the pclass_map as a .mat instead.

https://github.com/TissueImageAnalytics/cerberus/blob/4b72cff5584ff60a441678f524d2281994b34ee5/infer/tile.py#L286-L290

The 500x500 pclass_map array seems to contain a bunch of floats and the output of plt.show(pclass_map) looks like this:

These don't look like tissue type predictions, so I think i'm misunderstanding something fundamental here. Can you point me in the right direction?

[simongraham]

Hi @jjhbw

Thanks for raising this! I have now made and merged a PR to fix this. Please give it a go and verify that you are happy with it 👍🏼 .

[jjhbw]

Thanks a lot for taking the time! Much appreciated.

When I re-implement my workaround (writing to .mat instead of .png, see my first post) the pclass matrix now looks like the below:

This seems to be what we want, right? Thanks a lot!

However, the script still writes these matrices as png files, resulting in completely black images. This is likely because the matrix contains integer class labels which need to be mapped to an RGB color scheme first.

Just an idea: I don't think writing to png is necessarily the best way to output these matrices. Formats like .mat or .npz are easier to use in downstream analysis. Users can always turn these maps into images themselves by applying a color scheme, but the other way around is more difficult and would require modification of your code.

[simongraham]

Yes the above is the sort of thing that you should expect. Of course it looks quite blocky when you are using small image tiles as input, but when considering a WSI you will be able to see more noticeable patterns.

Yes when writing with opencv, an array is expected to be between 0 and 255, so low values will look black. I am happy to use mat file to keep it consistent with the rest of the code. I will make another PR to address this.

[jjhbw]

Great, makes sense! Thanks a lot for your time! 👍 👍