search

raghakot / keras-vis

Neural network visualization toolkit for keras
https://raghakot.github.io/keras-vis
MIT License
2.97k stars 664 forks source link

Can I generate an input that maximizes the activity of the convolution filter over time series? #163

Open konnyaku256 opened 5 years ago

konnyaku256 commented 5 years ago

I am not good at English, so please forgive me.

Thank you for a wonderful project! This project is very useful for my research.

Currently, I am challenging activation maximization of the convolution filter of recurrent neural network. Specifically, the network is PredNet. This network contains a layer of Convolutional LSTM.

I tried to maximize using the following function. visualize_activation

Here, I specify the model input shape as follows. (timesteps, image_height, image_width, channels).

The output shape of this function was the same as the model input, but the same image was output at all time steps.

However, I expect that an input image optimized in chronological order will be generated.

Can you solve this problem?

keisen commented 5 years ago

Hi @konnyaku256 .

Currently, I am challenging activation maximization of the convolution filter of recurrent neural network.

The generated image is the input value for maxmizing the convolution filer's output, because visualize_activation generate input value that any given feature will be maximized.

However, I expect that an input image optimized in chronological order will be generated.

For example, if you'd like to get the input value for generating the expecting output value, you can define new Loss class that outputs max-loss when it is the expecting output value.

But I don't know that will be working well, sorry. If you do it, will you please post here your result.

Thanks!

konnyaku256 commented 5 years ago

Hi @keisen . What I want is not a single image. Can visualize_activation generate sequential images as optimal inputs? In the case of a sequential model, I think that it is necessary to optimize it like that.

I'm sorry if I can not convey it well.

keisen commented 5 years ago

Would you please email me? The address is found in my profile.

konnyaku256 commented 5 years ago

Sorry for late reply. I'm very busy. So we will refrain from detailed discussion.

This issue is close to #165 .

I hope the problem will be resolved.

keisen commented 5 years ago

Okay, I got it. I write my idea down here. I hope it help you. (I can tell you more detail in YOUR LANGUAGE if you have a time and need it.)

I believe you've misunderstood about visualize_activation . The function's result which you looked at is CORRECT, because it generate input value that any given FEATURE WILL BE MAXMIZED. The convolution filter's weights does not change as time proceeds in predicting phase.

This issue is close to #165 .

#165 is the issue regarding visualize_cam so it is not same as this. OMG, that one too :confused:

Thank!