Open wanghonen opened 2 months ago
试试把L103注释掉呢?https://github.com/PaddlePaddle/InterpretDL/blob/master/interpretdl/interpreter/gradient_cam.py#L103
十分感谢您的回复
OSError Traceback (most recent call last)
/tmp/ipykernel_57869/2151664618.py in
~/PaddleSeg/paddleseg/models/backbones/resnet_vd.py in ResNet101_vd(args) 387 @manager.BACKBONES.add_component 388 def ResNet101_vd(args): --> 389 model = ResNet_vd(layers=101, **args) 390 return model 391
~/PaddleSeg/paddleseg/models/backbones/resnet_vd.py in init(self, layers, output_stride, multi_grid, in_channels, pretrained, data_format) 259 stride=2, 260 act='relu', --> 261 data_format=data_format) 262 self.conv1_2 = ConvBNLayer( 263 in_channels=32,
~/PaddleSeg/paddleseg/models/backbones/resnet_vd.py in init(self, in_channels, out_channels, kernel_size, stride, dilation, groups, is_vd_mode, act, data_format) 59 groups=groups, 60 bias_attr=False, ---> 61 data_format=data_format) 62 63 self._batch_norm = layers.SyncBatchNorm(
/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/nn/layer/conv.py in init(self, in_channels, out_channels, kernel_size, stride, padding, dilation, groups, padding_mode, weight_attr, bias_attr, data_format) 655 weight_attr=weight_attr, 656 bias_attr=bias_attr, --> 657 data_format=data_format) 658 659 def forward(self, x):
/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/nn/layer/conv.py in init(self, in_channels, out_channels, kernel_size, transposed, dims, stride, padding, padding_mode, output_padding, dilation, groups, weight_attr, bias_attr, data_format) 134 shape=filter_shape, 135 attr=self._param_attr, --> 136 default_initializer=_get_default_param_initializer()) 137 self.bias = self.create_parameter( 138 attr=self._bias_attr, shape=[self._out_channels], is_bias=True)
/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/dygraph/layers.py in create_parameter(self, shape, attr, dtype, is_bias, default_initializer) 422 temp_attr = None 423 return self._helper.create_parameter(temp_attr, shape, dtype, is_bias, --> 424 default_initializer) 425 426 @deprecated(
/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layer_helper_base.py in create_parameter(self, attr, shape, dtype, is_bias, default_initializer, stop_gradient, type) 379 type=type, 380 stop_gradient=stop_gradient, --> 381 **attr._to_kwargs(with_initializer=True)) 382 else: 383 self.startup_program.global_block().create_parameter(
/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/framework.py in create_parameter(self, *args, **kwargs) 3570 pass 3571 else: -> 3572 initializer(param, self) 3573 return param 3574
/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/initializer.py in call(self, var, block) 365 out_var = _C_ops.gaussian_random( 366 'shape', var.shape, 'dtype', out_dtype, 'mean', self._mean, --> 367 'std', self._std_dev, 'seed', self._seed, 'use_mkldnn', False) 368 369 if var.dtype in [VarDesc.VarType.FP16, VarDesc.VarType.BF16]:
OSError: [operator < gaussian_random > error]
看看你那边能复现这两个jupyter里的热图吗?https://github.com/PaddlePaddle/InterpretDL/blob/master/tutorials/GradCAM_Segmentation.ipynb https://github.com/PaddlePaddle/InterpretDL/blob/master/tutorials/InputGrad_Segmentation.ipynb
我理解是单通道的图片效果不好,我这边没有单通道的模型,建议你先试试看,确认一下这几行结果的shape,调整一下适配单通道。
我这边找个单通道模型试试
PS 另外基于patch作为token的transformer模型,不确定效果
- 看看你那边能复现这两个jupyter里的热图吗?https://github.com/PaddlePaddle/InterpretDL/blob/master/tutorials/GradCAM_Segmentation.ipynb https://github.com/PaddlePaddle/InterpretDL/blob/master/tutorials/InputGrad_Segmentation.ipynb
- 我理解是单通道的图片效果不好,我这边没有单通道的模型,建议你先试试看,确认一下这几行结果的shape,调整一下适配单通道。
- 我这边找个单通道模型试试
PS 另外基于patch作为token的transformer模型,不确定效果
好的好的,我先试一下,如果你需要我这的模型和数据的话,我可以邮箱发给你,我的邮箱是985388035@qq.com, 十分感谢
- 看看你那边能复现这两个jupyter里的热图吗?https://github.com/PaddlePaddle/InterpretDL/blob/master/tutorials/GradCAM_Segmentation.ipynb https://github.com/PaddlePaddle/InterpretDL/blob/master/tutorials/InputGrad_Segmentation.ipynb
- 我理解是单通道的图片效果不好,我这边没有单通道的模型,建议你先试试看,确认一下这几行结果的shape,调整一下适配单通道。
- 我这边找个单通道模型试试
PS 另外基于patch作为token的transformer模型,不确定效果
GradCAM是能复现的,效果看着也正常,inputgrad_segmentaton复现不了,会报我上面给的那个错误
GradCAM是能复现的,效果看着也正常,inputgrad_segmentaton复现不了,会报我上面给的那个错误
这个我这边能正常跑,跑出的结果和链接中的区别不大。
我把deeplabv3改成了单通道的,然后跑了一下,可以看下这个:https://github.com/PaddlePaddle/InterpretDL/blob/master/tutorials/InputGrad_Segmentation_for_one_channel.ipynb
GradCAM是能复现的,效果看着也正常,inputgrad_segmentaton复现不了,会报我上面给的那个错误
这个我这边能正常跑,跑出的结果和链接中的区别不大。
我把deeplabv3改成了单通道的,然后跑了一下,可以看下这个:https://github.com/PaddlePaddle/InterpretDL/blob/master/tutorials/InputGrad_Segmentation_for_one_channel.ipynb 你好,非常感谢,我根据你给的链接,确实可以运行成功,不过目前关于inputgrad_segmentaton有几个问题
1.图片红框里面的两个模型路径可以都写自己训练好的模型吗 trained_model_path = '/home/aistudio/deeplabv3pmodel.pdparams' a = paddle.load("/home/aistudio/deeplabv3pmodel.pdparams")
2.我怎么知道可视化的是哪个层的呢,这个代码表示可视化的目标层吗,跟gradcam的代码不太一样, model_weights['backbone.conv1_1._conv.weight'] = a['backbone.conv1_1._conv.weight'][:, 2:3]
GradCAM是能复现的,效果看着也正常,inputgrad_segmentaton复现不了,会报我上面给的那个错误
这个我这边能正常跑,跑出的结果和链接中的区别不大。
我把deeplabv3改成了单通道的,然后跑了一下,可以看下这个:https://github.com/PaddlePaddle/InterpretDL/blob/master/tutorials/InputGrad_Segmentation_for_one_channel.ipynb
还有个问题是为什么segformer产生的gradients是三维[1, 256, 512]的,而deeplabv3+产生的gradients是四维[1, 2048, 64, 64]的呢
1.图片红框里面的两个模型路径可以都写自己训练好的模型吗 2.我怎么知道可视化的是哪个层的呢,这个代码表示可视化的目标层吗,跟gradcam的代码不太一样,
可以的,注意:那个demo只是为了把3通道的模型转成单通道用的,相关代码都是为了适配单通道(包括模型部分和输入部分的代码修改)。
input_grad的解释算法默认是对输入层做的,对transformer的模型,需要调成对embedding层,但是效果需要调,不一定好,对segmentation没做过,需要自己尝试一下。
- 如果我想保存图片的话在哪里能修改代码呢
在这里修改
- 能麻烦你再弄个gradcam的单通道的demo吗
原理都是一样的,把模型和输入改了,其它都是一样的。
PS 另外基于patch作为token的transformer模型,不确定效果
还有个问题是为什么segformer产生的gradients是三维[1, 256, 512]的,而deeplabv3+产生的gradients是四维[1, 2048, 64, 64]的呢
demo里用的deeplabv3p,gradients也是[512,512]。
我使用grad cam做语义分割的可解释性,我训练好的模型也是单通道训练的,我是用gradcam的时候也是单通道,会报下面这个错误,请问我该怎么修改呢
AssertionError Traceback (most recent call last) /tmp/ipykernel_2663/1576993860.py in
4 pixels = [(30, 30)]
5 # good layers for deeplabv3p: backbone.bb_3_2.relu, head.aspp, head.decoder.conv
----> 6 exp = gradcam.interpret(img_path, pixels, target_layer_name='backbone.norm4', resize_to=512, crop_to=512)
/tmp/ipykernel_2663/3928282195.py in interpret(self, inputs, pixels, labels, target_layer_name, gradient_of, resize_to, crop_to, visual, save_path) 217 vis_explanation = explanation_to_vis(imgs[i], 218 gradcam_explanation[i], --> 219 style='overlay_heatmap') 220 if visual: 221 show_vis_explanation(vis_explanation)
~/.data/webide/pip/lib/python3.7/site-packages/interpretdl/data_processor/visualizer.py in explanation_to_vis(batched_image, explanation, style) 51 assert len(batched_image.shape) == 3 52 ---> 53 assert len(explanation.shape) == 2, f"image shape {batched_image.shape} vs " \ 54 f"explanation {explanation.shape}" 55
AssertionError: image shape (512, 512, 3) vs explanation (512,)