search

keplr-io / quiver

Interactive convnet features visualization for Keras
https://keplr-io.github.io/quiver/
MIT License
1.75k stars 223 forks source link

Gap cam #53

Closed kretes closed 2 years ago

kretes commented 7 years ago

This PR introduces class activation mappings from global average pooling. Whole approach is described in http://cnnlocalization.csail.mit.edu/ http://arxiv.org/pdf/1512.04150.pdf

Changes consist mostly of code that will be activated only if the selected layer is Global Average Pooling (previously quiver was throwing an exception there).

There is slightly different logic as to how to generate the 'deprocess' the image in this layer, because it is needed to compare images for various classes, and so the normalization is done only once and for whole set of visualizations, not each image on its own.

kretes commented 7 years ago

screenshot from 2017-07-08 23-32-38 This is an example of how it look like.

For GAP layer it shows that many images as there are classes to predict after it. UI would benefit from indicating which image is a representation of each class - it is done in the same order as the classes exist in the model, but it is not expressed anywhere in the GUI. Furhtermore - the predictions are sorted from the highest probability, which may confuse even more.

Actually for a GAP network quiver should show predictions and Class activation mappings together , so that it is clear which is which.

jakebian commented 7 years ago

Wow, great work. Will review tomorrow.

kretes commented 7 years ago

Regarding the 'toggling' - We can do that, sure. However before that change quiver didn't show anything for a 'globalaveragepooling' layer - in the logs there was an error. Since output of GAP layer is a 1-dim vector, id didn't have a shape[2] which is expected in save_layer_outputs.

jakebian commented 7 years ago

Ah I see, makes sense. Still the vision has always been to support many different visualizations per layer, so this is a good opportunity to start implementing that in a somewhat generic way, with the subfolder/naming, and the client side changes I'm going to make.