How to use this project code？

[Zhangwd92]

Upon reading this paper, I've identified several methods that I believe could significantly benefit my project. However, I find myself uncertain about how to integrate these methods into my existing codebase. I believe that your guidance in this matter would be immensely helpful.

Would you be available to provide instructions or assistance on how to effectively utilize the code provided alongside the paper? I am eager to learn and implement these methods to enhance the functionality and performance of my project.

Thank you for considering my request. I look forward to your guidance and support.

[ClementPla]

Hi, Thanks for your interest and sorry for the delayed answer.
I recognize that the code provided is not the most friendly to use.

I've been working to fix a lot of the problems already identified in the paper (faster run time and batch parallelization in particular) and I will be releasing a more user-friendly very soon (probably this week).

All you will need is to have a trained ViT of your own following the implementation of timm: https://github.com/huggingface/pytorch-image-models/blob/main/timm/models/vision_transformer.py

The model must be trained with a class token and for now I'm only planning on implementing Attention Rollout (in the paper I used ViT-LRP for the illustrations, which is probably better, but unfortunately I don't see a clean way to make the relevance propagation generic to all ViTs).

[ClementPla]

Hi again, Thanks for your interest and apologies for the slow response. I have been working on new "friendlier" repository. It includes a new implementation of the algorithm (functionally the same, but more efficient). Please visit: https://github.com/ClementPla/interpretability_toolkit In particular the associated notebook. Please open an issue in the new repository if needed.

[03-134202-096]

Hi, Thanks this one is working: https://github.com/ClementPla/interpretability_toolkit

But this is for Birds, How i fine-tune it for my dataset (Retinal Image dataset or other). Please guide me about it.

[ClementPla]

Hi, I've just posted a notebook for fundus classification that includes generating the interpretability heatmaps:

https://github.com/ClementPla/interpretability_toolkit/blob/main/notebooks/fundus_interpretability.ipynb

You can change the model if needed, I've just released 20+ modern architectures trained on this task:

https://huggingface.co/collections/ClementP/fundus-grading-665e582701ca1c80a0b5797a

[03-134202-096]

Thanks but i want to fine-tune according to my dataset.

Guide me regarding this.

On Tue, Jun 4, 2024, 5:01 AM ClementPla @.***> wrote:

Hi, I've just posted a notebook for fundus classification that includes generating the interpretability heatmaps:

https://github.com/ClementPla/interpretability_toolkit/blob/main/notebooks/fundus_interpretability.ipynb

You can change the model if needed, I've just released 20+ modern architectures trained on this task:

https://huggingface.co/collections/ClementP/fundus-grading-665e582701ca1c80a0b5797a

— Reply to this email directly, view it on GitHub https://github.com/ClementPla/FocusedAttention/issues/1#issuecomment-2146322995, or unsubscribe https://github.com/notifications/unsubscribe-auth/AVZKWPVVO7Z5ZOBRYKOI62LZFT7WLAVCNFSM6AAAAABF4D5D3SVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNBWGMZDEOJZGU . You are receiving this because you commented.Message ID: @.***>