[HistoSSL/heatmaps] MIL model attention heatmaps display bright lines when use histoSSL as the feature extractor

[sz3029]

Feature

The newly added pre-trained histoSSL model is very helpful. However, I have observed bright lines on the heatmaps, they are common on instances with 20x and 40x mag levels, and visible with 10x. The lines were not from the image itself, and were not present on heatmaps trained by models using other feature extractors (resnet50, etc. al.). I'm wondering if there's something to do with the heatmap methods. Below are some examples:

• Heatmap examples for 20x, 40x, feature extractor = histoSSL, a true negative case:

• Heatmap examples for 20x, 40x, feature extractor = resnet50, a true negative case:

(Tiles are from the same source)

Pitch

No bright lines on heatmaps.

Alternatives

1. Editing the heatmap methods (if the problem is due to how the heatmap is ploted)
2. Some normalization of the scores might help (log transformation, etc. al.)

[jamesdolezal]

Hmm - first guess is that this looks like focus banding issues (some slide scanners have a propensity to generate out of focus regions in repeating bands, as is shown here). What do you see when you zoom in at high resolution at these transition points? Is there anything obvious to the naked eye with respect to focus, tiling, etc?

[sz3029]

Thank you for the quick reply. I was suspecting that when I first saw the heatmaps. But I inspected the scans at 20x and 40x and I didn't see visible bands. But if you haven't seen this before then it might be a scanner issue :(

at ~5x:

at ~40x:

[jamesdolezal]

Hmm. Next step: what do deepfocus predictions look like at each magnification? This can sometimes highlight subtle focus issues that are difficult to see.

import numpy as np
import slideflow as sf
import matplotlib.pyplot as plt
import matplotlib.cm as cm
from PIL import Image

def plot(array):
    """Plot deepfocus predictions with an inferno colormap."""
    array = (cm.inferno(results)*255).astype(np.uint8)
    Image.fromarray(array).show()

# Load a slide, at any magnification
wsi = sf.WSI(path, tile_px=299, tile_um=302)

# Prepare deepfocus at 20x magnification
deepfocus = sf.slide.qc.DeepFocus(tile_um='20x')

# Calculate focus predictions at 20x
preds = deepfocus(wsi, threshold=False)

# Plot results
plot(preds)

I'd recommend trying this at various magnifications, 10x-40x, to see if there is any banding.

[sz3029]

Hi James, thank you for the advice. I run with the code above on several images. I think for mag. level >20x there are some possible bands showing up on the deepfocus trained image.

I think I will find some way to normalize it. Thank you for your help.