search

raghakot / keras-vis

Neural network visualization toolkit for keras
https://raghakot.github.io/keras-vis
MIT License
2.98k stars 664 forks source link

How do we extend this to TimeDistributed Layers? #48

Open sharathadavanne opened 7 years ago

sharathadavanne commented 7 years ago

Hi, great package and examples. Thanks a lot for doing this. Currently, I am working with TimeDistributed multilabel output. How do we extend it to work on this?

raghakot commented 7 years ago

I am assuming the input is something like (batchsize, time_dims, input_dims, channels). The heatmap will simply be of the same shape which shows how the attention will vary over time_dims.

As for multi-label output. we have the ability to set multiple filter_indices. so filter_indices = [0, 5] would mean that outputs 0, 5 are on.

It would be really cool to have examples for these use-cases in examples/ folder. If your code is not confidential or proprietary, consider submitting an example PR. I am willing to work with you to get it working. Having a wide variety of examples/ is really the key rather than documentation to illustrate the various use-cases.

In either case, I can help you debug with any specific issues you might be having.

sharathadavanne commented 7 years ago

Hi @raghakot I have emailed you some test data and the model. If we can get it working, then we can post it as an example here.

raghakot commented 7 years ago

Cool. Looking at it.

raghakot commented 7 years ago

Minor update. With backprop_modifier='relu' in saliency, you do see something for weak_out. Does it make sense? This is using the deconv approach from matthew ziegler's paper.

I am still working on a solution for TimeDistributed nets.

sivabala1994 commented 6 years ago

Hi @raghakot, Great work! it was really useful for getting insights about my project results. I am working on a time series based data. I am doing a RNN classification task and It will be great if you could share the technique on how to use Keras-vis for saliency analysis in RNN(LSTM) models.