Implementing Grad-Cam on a custom FCN Tensorflow Segmentation model

[varungupta31]

I wish to implement gradcam visualizations on a custom trained FCN segmentation model. I have trained my model using this FCN repo.

I have the trained .meta, .index and checkpoint files. The predictions is an array of ndarrays, as here

Is it possible to integrate this library with the above code for generating Gradcam visuals? Any help is appreciated, Thanks.

[jacobgil]

Hi, Unfortunately no, this library is only for pytorch. The only way I see would be to port the model to pytorch (with ONNX for example).

You could also work with one of the keras pytorch implementations, like this one: https://github.com/jacobgil/keras-grad-cam/blob/master/grad-cam.py#L44 And change it to work with segmentation, according do the segmentation notebook in this repo.

[varungupta31]

I was trying to find a 'shortcut' way to implement gradcam on my trained instance. Keras model too will require a keras model object to work, which again required to re-build the model in keras :/

But thank you for replying to this slightly off-topic thread :) Good Day!

[varungupta31]

Hey @jacobgil Please help me out in implementing gradcam in my semantic segmentation model based on FCN8+VGG19, trained in Keras. (I asked this earlier, and being absolutely clueless with how to implement it in a pure tensorflow implementation, I trained it again in Keras).

I have the trained model file (.h5). The keras model .predict return a tensor of the same shape as my image (unlike a dictionary in Pytorch, as demonstrated in your notebook). I'm not much familiar with Pytorch.

Requesting you to help me out with implementing gradcam from this point. Thank you.

Ps - I have also opened an issue in your keras grad-cam repo, but the dicussions there seem inactive.