Open taylorty opened 6 years ago
I got the same error, too.
jet_heatmap = np.uint8(cm.jet(grads)[..., :3] * 255) ax[i].imshow(overlay(jet_heatmap, img)) #this line raise an error
I changed the previous line to the following line . jet_heatmap = np.uint8(cm.jet(grads) * 255)[:, : , :, 0]
Maybe it is due to different python version's slice method is different?
Anyway, this project is very helpful.
I'm having the same error running it from a regular python script based on the ipython notebook. I'm using python3, keras 2.0.8 and matplotlib 2.0.2. In the matplotlib in their change log they mention some changes to colormaps. But I can't see why it would be breaking changes although I'm not familiar with the matplotlib internals.
The weird thing for me is the 4th dimension the function cm.jet returns.
cm.jet(grads).shape = (299,299,3,4)
I can understand that it would have shape (w,h,4) where the third dimension would be a RGBA vector of length 4. In that case @raghakot 's code [...,:3] would just remove the alpha channel, the 4th element of that vector, across the tensor. Which makes sense if i look at the overlay function (since there he's mixing the alphas himself).
Right now as a workaround I'm just displaying the grads side by side with the images. Using ax[i][1].imshow(grads, cmap='jet')
. Although weirdly i seem to be having inverse coloring as in the ipython notebook. Full code here.
@linchundan88 @taylorty What versions are you guys running? Matplotlib 2.0.2 also?(check with pip list
or conda list
)
ax[i].imshow(overlay(grads, img))
Instead of
ax[i].imshow(overlay(jet_heatmap, img))
Worked fine for me
Any news to this issue? None of the workarounds mentioned here, give me the same heatmap for the Grad-Cam Visualization as in the attention.ipynb tutorial.
Weirdly, I got the full code working on a second machine after several attempts of cloning, updating und erasing the repository on that machine but I couldn't get it to work on my primary machine. The machine, on which the code is working straight on, is using matplotlib 2.0.2 and keras-vis 0.4.1
The problem seems to be the matplotlib, since cm.jet(...) is returning 4 jet images. So I just overlayed each individually. The 4th returning image seems to be the alpha channel and is mostly only white. Additionally, I ignored the normalization step for uint 255. This gave me some reasonable results.
# Lets overlay the heatmap onto original image.
jet_grads = cm.jet(grads)
overlay_img = img
for idx in [0, 1, 2]:
jet_heatmap = jet_grads[..., idx]
overlay_img = overlay(overlay_img, jet_heatmap)
ax[i].imshow(overlay_img)
@phisad can you expand a little more how you solved it?
I'm stuck on visualize_attention.ipynb
, and I think the issue is coming from the file vis/visualization/saliency.py
. The heatmaps there, in the function visualize_saliency_with_losses
and visualize_cam_with_losses
, are returning as a black and white image, no color channels. So, overlaying that "black and white" heatmap on top of the color images is causing the issue (at least for me - with unmodified code).
@mimoralea OK, the closest I can get to the visualize_attention.ipynb
narrative is to just select the first from the cm.jet(grads).
# Lets overlay the heatmap onto original image.
jet_heatmap = cm.jet(grads)[..., 0]
ax[i].imshow(overlay(jet_heatmap, img))
The uint conversion is depending on your image scale (if [0...255] or [0...1]).
There might be a change in matplotlib behavior that introduced the problem. In the commit from fix #67 master@raghakot raghakot committed on Aug 24, 2017
there was actually before return np.uint8(cm.jet(grads)[..., :3] * 255)[0]
We could interpret [0]
as "selecting the first returning map".
But the changed notebook states only np.uint8(cm.jet(grads)[..., :3] * 255)
without [0]
.
I see. FWIW, I think my issue is slightly different. It has to do with my model being trained on grayscale images. So, when I try to overlay the heatmap (which is 1 channel as well), then the plot methods throws a similar error.
Thanks so much for the help @phisad!
Hi,
I tried to run attention.ipynb here with Tensorflow backend. But it showed an error in the last section of the code saying that "ValueError:
array1
andarray2
must have the same shapes"ValueError Traceback (most recent call last)