search

rayleizhu / BiFormer

[CVPR 2023] Official code release of our paper "BiFormer: Vision Transformer with Bi-Level Routing Attention"
https://arxiv.org/abs/2303.08810
MIT License
461 stars 36 forks source link

A question about interpretability #7

Closed sxl845414442 closed 1 year ago

sxl845414442 commented 1 year ago

Hello author, I am a newcomer in this field. I would like to ask if your Biformer has interpretability and where it is reflected.

rayleizhu commented 1 year ago

Can your question be more concrete? I have no idea which interpretability you expect. If you have not read our paper yet, I would suggest you read it first.

sxl845414442 commented 1 year ago

Can your question be more concrete? I have no idea which interpretability you expect. If you have not read our paper yet, I would suggest you read it first.

Thank you for your answer. I would like to know how to visualize the predicted results into a heat map and how to implement the specific code.

rayleizhu commented 1 year ago

I would like to know how to visualize the predicted results into a heat map

(1) Get the routing indices and attention matrix. (2) Create a pseudo color map with extracted indices or attention matrix. (3) Alpha blending image and the pseudo color map.

The visualization demo is on my to-do list. But I'm busy recently, hence it may take months before the release.

and how to implement the specific code.

What do you mean by "specific code"? Which part of the code?

sxl845414442 commented 1 year ago

以及如何实现特定代码。

“特定代码”是什么意思？代码的哪一部分？

I thought the heat map was implemented by code, but I couldn't find which code to run.