search

mahmoodlab / PANTHER

Morphological Prototyping for Unsupervised Slide Representation Learning in Computational Pathology - CVPR 2024
Other
92 stars 11 forks source link

Question Not Issue - Overlapping Tiles? #3

Closed matthew-lee1 closed 4 months ago

matthew-lee1 commented 4 months ago

Hello! Awesome work. I am curious, if in the patching/feature extraction steps I'm using overlapping tiles, let's say tile size 224 and step size 112, how would the visualization work? Is that even possible? In CLAM it seems overlapping tiles can still be used when visualizing the attention scores, I'm wondering if the same thing can be done with PANTHER. Thanks!

andrewsong90 commented 4 months ago

Yes, that's a good point - It should theoretically be possible. We would need to modify PANTHER such that for overlapping regions, it accounts for all cluster assignment probabilities (For patch size 224 of overlap 112, a given region would be assigned 4 different probabilities) - I think just taking average of the probabilities would do.

matthew-lee1 commented 4 months ago

I see. Would it be easiest to implement that at the visualization stage - if not could you point me out to the class where you think implementing this would be easiest? Thanks!

andrewsong90 commented 4 months ago

Yes, I think it would be easiest to implement it at the visualization step, since the PANTHER steps for clustering & assigning cluster probabilities are agnostic to whether the patches overlap or not.

This would be similar in spirit to how typical attention-based heatmaps are generated (e.g., CLAM), where for a given region you average the cluster probabilities assigned by all the patches overlapping with the region, and then assign the final cluster assignment.