search

yaolinli / DeCo

BSD 3-Clause "New" or "Revised" License
11 stars 0 forks source link

R-GAE #2

Open hustzyj opened 1 week ago

hustzyj commented 1 week ago

Hi, Thanks to your Solid work！I want to know how to calculate the R-GAE maps，especially the Query-to-patch. Could you please supply some key codes.

yaolinli commented 1 week ago

Thank you for your interest in our DeCo work. Our current implementation of R-GAE primarily builds upon the code LLaVA v1.5 model (https://github.com/haotian-liu/LLaVA?tab=readme-ov-file) and the ICCV 2021 paper "Generic Attention-model Explainability for Interpreting Bi-Modal and Encoder-Decoder Transformers" (https://github.com/hila-chefer/Transformer-MM-Explainability). The key code still needs to be sorted out, maybe released within the next month.