huobailiu
commented
3 weeks ago Hello, have you solved the issue? I'm also currently researching interpretable learning for 3D CNNs. Specifically, I'm referring to Class Activation Mapping (CAM) and Grad-CAM, which use heatmaps to represent the importance of specific regions in an image for predicting the output.
I came across a method on GitHub (https://github.com/yiskw713/3DClassActivationMapping) which is primarily designed for video inputs, where the visualization is done for each frame. I made some modifications to adapt it for point cloud learning tasks.
Additionally, I encountered gradient-related issues when using sparse convolutions with spconv for generating Grad-CAM. As a result, I modified the network architecture to use conv3d, which introduces some discrepancies compared to the results obtained with sparse convolutions.
I also found a paper titled "Visual Explanations From Deep 3D Convolutional Neural Networks for Alzheimer’s Disease Classification" that proposes four methods for visualizing 3D CNNs. It could be helpful to refer to and learn from. Here's the translation of the provided text: 你好，请问你解决这个问题了吗？我最近也在研究3dcnn的可解释学习内容。 主要还是参照2dcnn的CAM和Grad-CAM，通过热力图表示图像某一部分内容对于预测结果的重要程度。 我参考了github上的方法：https://github.com/yiskw713/3DClassActivationMapping 然而，这个主要是针对视频输入的，最后的可视化效果是每一帧图像，所以我针对点云学习的任务进行一点点修改。 另外，对于spconv的稀疏卷积，我在进行Grad-CAM生成时遇到了梯度上的问题，所以我修改网络结构转成conv3d，这会与稀疏卷积的结果有一定的误差。 论文《Visual Explanations From Deep 3D Convolutional Neural Networks for Alzheimer’s Disease Classification》提出了四种3dcnn可视化的方法，也可以参考学习一下
Aoko-Blue
yinheyanxian
I'm wondering how to visualise spconv's focus, there are plugins for 2D convolution that show exactly which positions the current network is focusing on, and I'm wondering if spconv has a similar tool.