search

szc19990412 / TransMIL

TransMIL: Transformer based Correlated Multiple Instance Learning for Whole Slide Image Classification
325 stars 72 forks source link

How to make interpretability and attention visualization？Just like Figure 6 in the paper TransMIL. #21

Closed PerrySkywalker closed 1 year ago

szc19990412 commented 1 year ago

First, get the self attention matrix of nystrom attention (set the return_attn =True). Then take out the column corresponding to the class token, that is, the attention of the class token to the rest of the feature tokens. Finally, we follow the CLAM framework.

PerrySkywalker commented 1 year ago

Ok, thank you very much.

DeVriesMatt commented 1 year ago

Sorry, I still don't understand. The output shape of the nystrom attention (when return_attn=True), is [B, num_head, x, x] (where I don't know what x is). Do you have the code for this?

PerrySkywalker commented 1 year ago

https://github.com/jacobgil/vit-explain

Sorry, I still don't understand. The output shape of the nystrom attention (when return_attn=True), is [B, num_head, x, x] (where I don't know what x is). Do you have the code for this?

https://github.com/jacobgil/vit-explain， watch this.

Rokinluohhh commented 4 months ago

https://github.com/jacobgil/vit-explain

Sorry, I still don't understand. The output shape of the nystrom attention (when return_attn=True), is [B, num_head, x, x] (where I don't know what x is). Do you have the code for this?

https://github.com/jacobgil/vit-explain， watch this.

Do you know how to take out the column corresponding to the class token?

Rokinluohhh commented 4 months ago

https://github.com/jacobgil/vit-explain

Sorry, I still don't understand. The output shape of the nystrom attention (when return_attn=True), is [B, num_head, x, x] (where I don't know what x is). Do you have the code for this?

https://github.com/jacobgil/vit-explain， watch this.

Can you provide your code about visualization part?thanks very much!

PerrySkywalker commented 4 months ago

https://github.com/jacobgil/vit-explain

Sorry, I still don't understand. The output shape of the nystrom attention (when return_attn=True), is [B, num_head, x, x] (where I don't know what x is). Do you have the code for this?

https://github.com/jacobgil/vit-explain， watch this.

Can you provide your code about visualization part?thanks very much!

Ok, I will upload the code to my GitHub in a week.

Rokinluohhh commented 4 months ago

https://github.com/jacobgil/vit-explain

Sorry, I still don't understand. The output shape of the nystrom attention (when return_attn=True), is [B, num_head, x, x] (where I don't know what x is). Do you have the code for this?

https://github.com/jacobgil/vit-explain， watch this.

Can you provide your code about visualization part?thanks very much!

Ok, I will upload the code to my GitHub in a week.

Wow!Thanks a lot!