search

jacobgil / keras-grad-cam

An implementation of Grad-CAM with keras
MIT License
659 stars 236 forks source link

Apply GradCam to Cnn+LSTM #32

Open gabarlacchi opened 4 years ago

gabarlacchi commented 4 years ago

Hi all, I am working with a feature extractor (Inception V3, VGG16, whatever) plus an LSTM for sequences classification (let's say 3 sec. each one). There's any way to use GradCam in order to obtain areas activation on the input sequences? Right now, I just classify one sequence without single frame classification, is still possible apply GradCam to this network? I didn't find any example code about it, but just for GradCam applied to single feature extractor. Thanks in advance for the help, Gabriele

monjurulkarim commented 3 years ago

@barloccia Did you find any solution?

gabarlacchi commented 3 years ago

Hi, I used the standard way by applying grad-cam to each frame within the sequence. Since the CNN+LSTM network internally predict the class for each frame, you can then apply grad-cam to them. In my case this represents the area within every image in which the network focused at prediction time. I applied the implementation of grad-cam tensorflow 2: https://github.com/ismailuddin/gradcam-tensorflow-2

Hope this helps, Best