InvalidArgumentError: The first dimension of paddings must be the rank of inputs[4,2] [1,224,224]

[JeremyBYU]

• [ X] Check that you are up-to-date with the master branch of keras-vis. You can update with: pip install git+git://github.com/raghakot/keras-vis.git --upgrade --no-deps

• [X ] If running on TensorFlow, check that you are up-to-date with the latest version. The installation instructions can be found here.

• [ X] Provide a link to a GitHub Gist of a Python script that can reproduce your issue (or just copy the script here if it is short).

I have the latest keras, tensorflow 1.10, and the latest keras-vis cloned from master. I am running the example code for found here: https://github.com/raghakot/keras-vis/blob/master/examples/resnet/attention.ipynb

This code triggers the error:

f, ax = plt.subplots(1, 2)
for i, img in enumerate([img1, img2]):    
    # 20 is the imagenet index corresponding to `ouzel`
    grads = visualize_saliency(model, layer_idx, filter_indices=0, seed_input=img)

    # visualize grads as heatmap
    ax[i].imshow(grads, cmap='jet')

This is the error that I receive:

---------------------------------------------------------------------------
InvalidArgumentError                      Traceback (most recent call last)
<ipython-input-4-d46658b19073> in <module>()
      2 for i, img in enumerate([img1, img2]):
      3     # 20 is the imagenet index corresponding to `ouzel`
----> 4     grads = visualize_saliency(model, layer_idx, filter_indices=0, seed_input=img)
      5 
      6     # visualize grads as heatmap

~\Anaconda3\envs\tf-gpu-new\lib\site-packages\vis\visualization\saliency.py in visualize_saliency(model, layer_idx, filter_indices, seed_input, wrt_tensor, backprop_modifier, grad_modifier)
    125         (ActivationMaximization(model.layers[layer_idx], filter_indices), -1)
    126     ]
--> 127     return visualize_saliency_with_losses(model.input, losses, seed_input, wrt_tensor, grad_modifier)
    128 
    129 

~\Anaconda3\envs\tf-gpu-new\lib\site-packages\vis\visualization\saliency.py in visualize_saliency_with_losses(input_tensor, losses, seed_input, wrt_tensor, grad_modifier)
     74     """
     75     opt = Optimizer(input_tensor, losses, wrt_tensor=wrt_tensor, norm_grads=False)
---> 76     grads = opt.minimize(seed_input=seed_input, max_iter=1, grad_modifier=grad_modifier, verbose=False)[1]
     77 
     78     channel_idx = 1 if K.image_data_format() == 'channels_first' else -1

~\Anaconda3\envs\tf-gpu-new\lib\site-packages\vis\optimizer.py in minimize(self, seed_input, max_iter, input_modifiers, grad_modifier, callbacks, verbose)
    150 
    151             # 0 learning phase for 'test'
--> 152             computed_values = self.compute_fn([seed_input, 0])
    153             losses = computed_values[:len(self.loss_names)]
    154             named_losses = list(zip(self.loss_names, losses))

~\Anaconda3\envs\tf-gpu-new\lib\site-packages\keras\backend\tensorflow_backend.py in __call__(self, inputs)
   2661                 return self._legacy_call(inputs)
   2662 
-> 2663             return self._call(inputs)
   2664         else:
   2665             if py_any(is_tensor(x) for x in inputs):

~\Anaconda3\envs\tf-gpu-new\lib\site-packages\keras\backend\tensorflow_backend.py in _call(self, inputs)
   2631                                 symbol_vals,
   2632                                 session)
-> 2633         fetched = self._callable_fn(*array_vals)
   2634         return fetched[:len(self.outputs)]
   2635 

~\Anaconda3\envs\tf-gpu-new\lib\site-packages\tensorflow\python\client\session.py in __call__(self, *args, **kwargs)
   1380           ret = tf_session.TF_SessionRunCallable(
   1381               self._session._session, self._handle, args, status,
-> 1382               run_metadata_ptr)
   1383         if run_metadata:
   1384           proto_data = tf_session.TF_GetBuffer(run_metadata_ptr)

~\Anaconda3\envs\tf-gpu-new\lib\site-packages\tensorflow\python\framework\errors_impl.py in __exit__(self, type_arg, value_arg, traceback_arg)
    517             None, None,
    518             compat.as_text(c_api.TF_Message(self.status.status)),
--> 519             c_api.TF_GetCode(self.status.status))
    520     # Delete the underlying status object from memory otherwise it stays alive
    521     # as there is a reference to status from this from the traceback due to

InvalidArgumentError: The first dimension of paddings must be the rank of inputs[4,2] [1,224,224]
     [[Node: conv1_pad_1/Pad = Pad[T=DT_FLOAT, Tpaddings=DT_INT32, _class=["loc:@gradients/conv1_pad_1/Pad_grad/Slice_1"], _device="/job:localhost/replica:0/task:0/device:GPU:0"](_arg_input_1_1_0_0/_5541, conv1_pad_1/Pad/paddings)]]
     [[Node: gradients/conv1_pad_1/Pad_grad/Slice_1/_5765 = _Recv[client_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device_incarnation=1, tensor_name="edge_5348_gradients/conv1_pad_1/Pad_grad/Slice_1", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:CPU:0"]()]]

[keisen]

I could not reproduce the above error.

Do you use image what it's in examples/? (If there is any modification in the ipynb file, please show me the diff.) Can you show me the result performing pip freeze?

[JeremyBYU]

Okay, it looks like it was in image issue. I was using my own image, but it was in a .png format (it also was already 224X224). I saved the image as a JPG and now there are no errors.

Here are the example images of png and jpg. First throws error, second doesn't seem to. https://ibb.co/ixXpjp https://ibb.co/iEVLqU

Thanks!

[keisen]

Aren't you directly using Pillow API?

Your image file is 'P' mode file (See details: https://pillow.readthedocs.io/en/3.1.x/handbook/concepts.html#concept-modes). So, you have to use Keras API or to convert mode to 'RGB'.

• Using Keras API

  from keras.preprocessing import image
  img = image.load_img('/your/image/file/path')

• Using convert()

  from PIL import Image
  img = Image.open('/your/image/file/path')
  img = img.convert('RGB')

I believe this issue is resolved. I close this but please reopen this issue if I have mistaken.

[ArpithaFAU]

Hello I also have a similar issue, i am using my own image and i get the error below: "tensorflow.python.framework.errors_impl.InvalidArgumentError: The first dimension of paddings must be the rank of inputs[4,2] [1,512,512] [[{{node zero_padding2d_1_1/Pad}}]]"

I am giving the same size with which i trained the model for my image.