[BUG] Unable to generate heatmaps when stride_div > 1 for MIL models

[sz3029]

Description

Hi I'm having this heatmap coordinate alignment error when I tried to train the CLAM model with stride_div = 2 using slideflow.Project.train_mil and eval_mil. I have attached the error message below. It worked as expected when I use the default value stride_div = 1.

(Thank you for the recent release on version 2.2.0, that is very helpful.)

Traceback (most recent call last):
  File "/home/user/.conda/envs/slideflow2-2/lib/python3.9/site-packages/slideflow/util/__init__.py", line 1072, in map_values_to_slide_grid
    idx = loc_grid_dict[tuple(wsi_dim)]
KeyError: (52488, 22842)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/projects/user/slideflow_experiments/slideflow_test_dataset.py", line 231, in <module>
    train_mil(
  File "/home/user/.conda/envs/slideflow2-2/lib/python3.9/site-packages/slideflow/mil/train/__init__.py", line 80, in train_mil
    return _train_mil(config, **mil_kwargs)
  File "/home/user/.conda/envs/slideflow2-2/lib/python3.9/site-packages/slideflow/mil/train/__init__.py", line 156, in _train_mil
    return train_fn(
  File "/home/user/.conda/envs/slideflow2-2/lib/python3.9/site-packages/slideflow/mil/train/__init__.py", line 775, in train_fastai
    generate_attention_heatmaps(
  File "/home/user/.conda/envs/slideflow2-2/lib/python3.9/site-packages/slideflow/mil/eval.py", line 757, in generate_attention_heatmaps
    sf.util.location_heatmap(
  File "/home/user/.conda/envs/slideflow2-2/lib/python3.9/site-packages/slideflow/util/__init__.py", line 1147, in location_heatmap
    masked_grid = map_values_to_slide_grid(
  File "/home/user/.conda/envs/slideflow2-2/lib/python3.9/site-packages/slideflow/util/__init__.py", line 1074, in map_values_to_slide_grid
    raise errors.CoordinateAlignmentError(
slideflow.errors.CoordinateAlignmentError: Error plotting value at location (52488, 22842) for slide /projects/user/main_image_dataset_632/206 AVD-96SKY-8367A H&E.svs. The heatmap grid is not aligned to the slide coordinate grid. Ensure that tile_px (got: 256) and tile_um (got: 256) match the given location values. If you are using data stored in TFRecords, verify that the TFRecord was generated using the same tile_px and tile_um.

To Reproduce

# 10x
P.extract_tiles(
        tile_px=256,  # size of tile in pixels
        tile_um=256,  # size of tile in micro-meters
        stride_div=2, # 50% overlap
        whitespace_fraction=0.5,  # discard tiles with this fraction of whitespace
        num_threads=4,  # number of threads
        qc="otsu",
        grayspace_fraction=1,
        save_tiles = True,
        report = True,
    )
...
train_mil(
            clam_config,
            train_dataset = dataset_train,
            val_dataset = dataset_val,
            exp_label=exp_label, # used for naming the subdirectory in the {project root}/mil folder
            outcomes=outcome,
            bags = os.path.join(
                "clamfeatures",
                backbone_model,
                normalization_type,
            ), # path to .pt files
            attention_heatmaps=True,
            cmap='magma'
        )

I also tried with P.train_mil() and P.evaluate_mil()but it gave me the same error.

Expected behavior

Generating heatmaps for slides in the validation dataset

Environment:

• Slideflow Version (e.g., 1.0): 2.2.0, cuCIM, torch
• OS (e.g., Ubuntu): Linux
• How you installed Slideflow (pip, source): pip
• Python version: 3.9.13
• CUDA/cuDNN version: 11.3, cuCIM
• GPU models and configuration: Tesla

[jamesdolezal]

Thanks for the bug report! We'll get this fixed shortly.

In the meantime, as a temporary workaround, I would recommend using Slideflow Studio, which can save and export attention heatmaps at arbitrary strides.

[jamesdolezal]

Apologies it took this long to address the issue. This bug has now been fixed in 632fbf7 and applied to master. It will be included in the next patch release in the coming days. Please let us know if you experience any further issues.