Visualize grad-CAM with Inception-V3

[sakares]

Regarding the idea of Class Activation Mapping, I try to reproduce visualize_cam with Inception-V3 but It seems I cannot get a proper heat-map of grad-CAM.

As far as I debug with the example which uses VGG-16, it seems the given code visualize the only Tensor from Convolution2D, _Pooling2D. Note: I have put a workaround by debugging a penultimate layer

# file visualization.py
# under def visualize_cam()

    if penultimate_layer_idx is None:
        for idx, layer in utils.reverse_enumerate(model.layers[:layer_idx-1]):
            if isinstance(layer, (Convolution2D, _Pooling2D)):
                # debug
                print("found penultimate_layer_idx at %s:%s" % (idx, layer))
                penultimate_layer_idx = idx
                break

produced found penultimate_layer_idx at 18:<keras.layers.pooling.MaxPooling2D object at 0x120db47b8> After looking at vggnet.py it seems applying grad-CAM on block5_pool or the given line: x = MaxPooling2D((2, 2), strides=(2, 2), name='block5_pool')(x)

so, I looked at the inception_v3 from Keras InceptionV3 and perform a transfer learning method for binary classification cat-dog by following the instruction on Keras documentation "Fine-tune InceptionV3 on a new set of classes", I found only GlobalAveragePooling2D or avg_pool tensor seems to be visualized before the softmax layer. so, I have worked around on getting penultimate_layer_idx with GlobalAveragePooling2D but it didn't work.

any advice, please?

Update: I have tried to reproduce the grad-CAM result with InceptionV3 on ImageNet and it seems the visualize_cam work on the layer 299 from following logs: found penultimate_layer_idx at 299:<keras.layers.convolutional.Conv2D object at 0x1266d4748> and snapshot here

[sakares]

I got it. Actually, it was about my preprocessing code during ImageDataGenerator for fine-tuning process.

[Abhijit-2592]

hey @sakares and @raghakot , I am having a similar issue. Grad-cam isn't working properly in inceptionv3. I did the same tutorial which Raghakot gave with only difference being I used Inceptionv3 instead of vgg16 and my Grad-cam isn't working properly.

I manually set the penultimate_layer_idx = 299 ('conv2d_94'). Still the same result