search

mahmoodlab / SurvPath

Modeling Dense Multimodal Interactions Between Biological Pathways and Histology for Survival Prediction - CVPR 2024
95 stars 4 forks source link

How is the interpretability visualization of the WSI in your article implemented? #2

Closed dawalisi closed 6 months ago

dawalisi commented 1 year ago

Your thesis visualisation work is great, could you please share the visualisation code?

lmxmercy commented 1 year ago

They never release the code for visualization

ajv012 commented 1 year ago

Hello,

Thank you for the interest in our work.

We performed the visualizations by reusing the code from CLAM (https://github.com/mahmoodlab/CLAM) and MCAT (https://github.com/mahmoodlab/MCAT). The visualization is done by using the the pathway-to-histology attention (which is outlined more in depth in the paper).

Please let me know if you have any further specific questions about the visualizations.

lmxmercy commented 1 year ago

First of all, I want to apologize for my inappropriate words and thank you for your excellent work.

  1. CLAM has released the code for heatmap visualization, however it’s a multi-classification task, MCAT, PORPOISE, and SurvPath are all multimodal survival prediction problems, they have quite some differences.
  2. What should I do to perform the exquisite visualization in the paper? How to understand pathway-to-histology attention? How can I modify the code in CLAM to do this? After all, MCAT and PORPOISE have done the same work, but none of them have open their code.
  3. How to perform the integrated gradient attribution of pathways? I know maybe it is based on captum, but how? Finally, thank you very much for your contribution to the community, but many of your papers, including visualization and even some exquisite curves, are difficult for beginners, which may be very easy for you. For us, we even need to study for a few months and finally end up without any results. I sincerely hope that you can release this part of the code for everyone to learn. Thank you again
dawalisi commented 1 year ago

你好

感谢您对我们工作的关注。

我们通过重用 CLAM （https://github.com/mahmoodlab/CLAM） 和 MCAT （https://github.com/mahmoodlab/MCAT） 中的代码来执行可视化。可视化是通过使用组织学途径注意力来完成的（本文对此进行了更深入的概述）。

如果您对可视化有任何其他具体问题，请告诉我。

Thank you for your reply, your team did a great job! But how is this visualised after multimodality, which is only CLAM attention

lmxmercy commented 1 year ago

ooops, they never release the code for visualization, that's true, right?

tjwy commented 5 months ago

First of all, I want to apologize for my inappropriate words and thank you for your excellent work.

  1. CLAM has released the code for heatmap visualization, however it’s a multi-classification task, MCAT, PORPOISE, and SurvPath are all multimodal survival prediction problems, they have quite some differences.
  2. What should I do to perform the exquisite visualization in the paper? How to understand pathway-to-histology attention? How can I modify the code in CLAM to do this? After all, MCAT and PORPOISE have done the same work, but none of them have open their code.
  3. How to perform the integrated gradient attribution of pathways? I know maybe it is based on captum, but how? Finally, thank you very much for your contribution to the community, but many of your papers, including visualization and even some exquisite curves, are difficult for beginners, which may be very easy for you. For us, we even need to study for a few months and finally end up without any results. I sincerely hope that you can release this part of the code for everyone to learn. Thank you again

Hello, I am also trying to replicate the visualization part of this work. I would like to ask if you have made any relevant attempts and if you would be willing to share your experience. Thank you very much.