search

raghakot / keras-vis

Neural network visualization toolkit for keras
https://raghakot.github.io/keras-vis
MIT License
2.97k stars 664 forks source link

3D saliency implementation #124

Open kitamura-felipe opened 6 years ago

kitamura-felipe commented 6 years ago

This PR adds saliency3D.py, which implements a 3D version of visualize_cam().

Usage is described in example/3D_saliency_snippet.ipynb

A minor change in scipy/misc/pilutils.py is needed and is described in vis/visualization/follow_this.txt

keisen commented 5 years ago

Thank you for the PR.

I don't think we should merge this change. Because currently keras-vis (i.e., master branch) is able to visualize cam in 3-dims.

https://github.com/raghakot/keras-vis/blob/9821aeab1f9396dba5e9137d0c3bcf4e6dee52bd/vis/visualization/saliency.py#L187-L188

kitamura-felipe commented 5 years ago

Indeed it calculates 3dims. But there is a minor bug in the output to handle 3d grayscale images.

Em sex, 31 de ago de 2018 19:34, Keisen notifications@github.com escreveu:

Thank you for the PR.

I don't think we should merge this change. Because currently keras-vis (i.e., master branch) is able to visualize cam in 3-dims.

https://github.com/raghakot/keras-vis/blob/9821aeab1f9396dba5e9137d0c3bcf4e6dee52bd/vis/visualization/saliency.py#L187-L188

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/raghakot/keras-vis/pull/124#issuecomment-417805961, or mute the thread https://github.com/notifications/unsubscribe-auth/AX5aCcpw8r2BgeQ1hSdQuzcJRpIc4O_Sks5uWboGgaJpZM4VZetc .

keisen commented 5 years ago

Can you show us the detail of minor bug ?

kitamura-felipe commented 5 years ago

I don't have the error anymore, but if you run the code on a 3D model with 1 channel (batch_size, d1, d2, d3, channel=1), the error will appear.

The solution I proposed is just to adjust the output of as visualize_cam_with_losses folows:

[image: image.png]

[image: photo] Felipe Campos Kitamura Head, Artificial Intelligence Lab DASA Neuroradiologist at Universidade Federal de São Paulo +55 11 97963-0032 <+55+11+97963-0032> | kitamura.felipe@gmail.com http://facebook.com/felipe.kitamura? http://br.linkedin.com/in/felipe-kitamura-7a6477a1

On Sat, Sep 1, 2018 at 1:15 AM Keisen notifications@github.com wrote:

Can you show us the detail of minor bug ?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/raghakot/keras-vis/pull/124#issuecomment-417831344, or mute the thread https://github.com/notifications/unsubscribe-auth/AX5aCc5OpL1HAYwe9FpASyVwwQxknEy0ks5uWgnEgaJpZM4VZetc .

keisen commented 5 years ago

I don't have the error anymore, but if you run the code on a 3D model with 1 channel (batch_size, d1, d2, d3, channel=1), the error will appear.

I tried it but the error was not occurred. The shape of input data is (batch_size, d1, d2, d3, 1) and the shape of visualize_cam result is (d1, d2, d3), It has no problem.

Can you post the short script (or the link of Gist) here that can reproduce your issue ?

visualize_cam_with_losses folows: [image: image.png] [image: photo]

Please resend the images if you need.

raghakot commented 5 years ago

We should just fix the issue that causes his use-case to not work. The API should handle N-dim inputs.