search

mahmoodlab / UNI

Towards a general-purpose foundation model for computational pathology - Nature Medicine
Other
348 stars 48 forks source link

How was figure 3e generated in the paper? #13

Open AditMeh opened 8 months ago

AditMeh commented 8 months ago
Screenshot 2024-03-25 at 8 51 32 PM

I used something similar to this to extract the attention scores for the penultimate layer, as explained in the caption for figure 3e. However, I found that the attention maps I'm getting are a lot less "intuitive" compared to the ones shown in this figure.

Was this figure generated with a fine-tuned UNI model on the ROI level task or is it just showing the attention maps of the SSL model (no fine-tuning)?

Also, are the 448^2, 896^2 and 1344^2 attention maps computed by concatenating the attention map for each non-overlapping 224^2 patch together?

PhillipMaire commented 5 months ago

@AditMeh I would love to see your progress with this I've been tasked with replicating this result, any chance you can share your progress?

@Richarizardd @fedshyvana @faisalml @GeorgeBatch could you provide any code one this? it would be super helpful! Fantastic paper!!

GeorgeBatch commented 5 months ago

@PhillipMaire, @AditMeh, I’m just an early adopter of this work - not one of the authors.