ValueError: operands could not be broadcast together with remapped shapes [original->remapped]: (3,2) and requested shape (4,2)

[Mahi-Mai]

I was so excited when I stumbled across this code, and I'm so grateful that its creator chose to share it with us. However, I'm struggling to use it. Here's my code...

from keras import models
import cv2
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline

from smooth_tiled_predictions import predict_img_with_smooth_windowing

import warnings
warnings.filterwarnings('ignore')

model = models.load_model(local_model_path)
im = cv2.imread(pth_to_img)
im = np.expand_dims(im,axis=0)

predictions_smooth = predict_img_with_smooth_windowing(im,
                                                       window_size=256,
                                                       subdivisions=2,  # Minimal amount of overlap for windowing. Must be an even number
                                                       nb_classes=2,
                                                       pred_func=(lambda img_batch_subdiv: model.predict(image_to_neural_input(img_batch_subdiv)))
                                                      )

print(type(predictions_smooth))
print(predictions_smooth.shape)

This results in the following stack trace...

ValueError                                Traceback (most recent call last)
<ipython-input-4-ad3aae6a71d0> in <module>
     10                                                        subdivisions=2,  # Minimal amount of overlap for windowing. Must be an even number
     11                                                        nb_classes=2,
---> 12                                                        pred_func=(lambda img_batch_subdiv: model.predict(image_to_neural_input(img_batch_subdiv)))
     13                                                       )
     14 

/repos/Kernel-Counting/U-Net/Prediction/smooth_tiled_predictions.py in predict_img_with_smooth_windowing(input_img, window_size, subdivisions, nb_classes, pred_func)
    221     http://blog.kaggle.com/2017/05/09/dstl-satellite-imagery-competition-3rd-place-winners-interview-vladimir-sergey/
    222     """
--> 223     pad = _pad_img(input_img, window_size, subdivisions)
    224     pads = _rotate_mirror_do(pad)
    225 

/repos/Kernel-Counting/U-Net/Prediction/smooth_tiled_predictions.py in _pad_img(img, window_size, subdivisions)
     77     aug = int(round(window_size * (1 - 1.0/subdivisions)))
     78     more_borders = ((aug, aug), (aug, aug), (0, 0))
---> 79     ret = np.pad(img, pad_width=more_borders, mode='reflect')
     80     # gc.collect()
     81 

<__array_function__ internals> in pad(*args, **kwargs)

/usr/local/anaconda/lib/python3.6/site-packages/numpy/lib/arraypad.py in pad(array, pad_width, mode, **kwargs)
    739 
    740     # Broadcast to shape (array.ndim, 2)
--> 741     pad_width = _as_pairs(pad_width, array.ndim, as_index=True)
    742 
    743     if callable(mode):

/usr/local/anaconda/lib/python3.6/site-packages/numpy/lib/arraypad.py in _as_pairs(x, ndim, as_index)
    514     # Converting the array with `tolist` seems to improve performance
    515     # when iterating and indexing the result (see usage in `pad`)
--> 516     return np.broadcast_to(x, (ndim, 2)).tolist()
    517 
    518 

<__array_function__ internals> in broadcast_to(*args, **kwargs)

/usr/local/anaconda/lib/python3.6/site-packages/numpy/lib/stride_tricks.py in broadcast_to(array, shape, subok)
    180            [1, 2, 3]])
    181     """
--> 182     return _broadcast_to(array, shape, subok=subok, readonly=True)
    183 
    184 

/usr/local/anaconda/lib/python3.6/site-packages/numpy/lib/stride_tricks.py in _broadcast_to(array, shape, subok, readonly)
    125     it = np.nditer(
    126         (array,), flags=['multi_index', 'refs_ok', 'zerosize_ok'] + extras,
--> 127         op_flags=['readonly'], itershape=shape, order='C')
    128     with it:
    129         # never really has writebackifcopy semantics

ValueError: operands could not be broadcast together with remapped shapes [original->remapped]: (3,2) and requested shape (4,2)

I get the same stack regardless of whether or not I implement the changes suggested in https://github.com/Vooban/Smoothly-Blend-Image-Patches/issues/1

Please help! I want to use this so badly. I even tried reworking things a bit- adjusting the prediction function and not expanding the image array, like this:

im = cv2.imread(files[0])

predictions_smooth = predict_img_with_smooth_windowing(im,
                                                       window_size=256,
                                                       subdivisions=2,  # Minimal amount of overlap for windowing. Must be an even number
                                                       nb_classes=2,
                                                       pred_func=(lambda img_batch_subdiv: model.predict(img_batch_subdiv))
                                                      )

print(type(predictions_smooth))
print(predictions_smooth.shape)

And that just gave me a different error:

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-5-96028bcce00b> in <module>
      9                                                        subdivisions=2,  # Minimal amount of overlap for windowing. Must be an even number
     10                                                        nb_classes=2,
---> 11                                                        pred_func=(lambda img_batch_subdiv: model.predict(img_batch_subdiv))
     12                                                       )
     13 

/repos/Kernel-Counting/U-Net/Prediction/smooth_tiled_predictions.py in predict_img_with_smooth_windowing(input_img, window_size, subdivisions, nb_classes, pred_func)
    251     for pad in tqdm(pads):
    252         # For every rotation:
--> 253         sd = _windowed_subdivs(pad, window_size, subdivisions, nb_classes, pred_func)
    254         one_padded_result = _recreate_from_subdivs(
    255             sd, window_size, subdivisions,

/repos/Kernel-Counting/U-Net/Prediction/smooth_tiled_predictions.py in _windowed_subdivs(padded_img, window_size, subdivisions, nb_classes, pred_func)
    191 
    192     # Such 5D array:
--> 193     subdivs = subdivs.reshape(a, b, c, d, nb_classes)
    194     gc.collect()
    195 

ValueError: cannot reshape array of size 4194304 into shape (8,8,256,256,2)

[ajinkya8484]

I was so excited when I stumbled across this code, and I'm so grateful that its creator chose to share it with us. However, I'm struggling to use it. Here's my code...

from keras import models
import cv2
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline

from smooth_tiled_predictions import predict_img_with_smooth_windowing

import warnings
warnings.filterwarnings('ignore')

model = models.load_model(local_model_path)
im = cv2.imread(pth_to_img)
im = np.expand_dims(im,axis=0)

predictions_smooth = predict_img_with_smooth_windowing(im,
                                                       window_size=256,
                                                       subdivisions=2,  # Minimal amount of overlap for windowing. Must be an even number
                                                       nb_classes=2,
                                                       pred_func=(lambda img_batch_subdiv: model.predict(image_to_neural_input(img_batch_subdiv)))
                                                      )

print(type(predictions_smooth))
print(predictions_smooth.shape)

This results in the following stack trace...

ValueError                                Traceback (most recent call last)
<ipython-input-4-ad3aae6a71d0> in <module>
     10                                                        subdivisions=2,  # Minimal amount of overlap for windowing. Must be an even number
     11                                                        nb_classes=2,
---> 12                                                        pred_func=(lambda img_batch_subdiv: model.predict(image_to_neural_input(img_batch_subdiv)))
     13                                                       )
     14 

/repos/Kernel-Counting/U-Net/Prediction/smooth_tiled_predictions.py in predict_img_with_smooth_windowing(input_img, window_size, subdivisions, nb_classes, pred_func)
    221     http://blog.kaggle.com/2017/05/09/dstl-satellite-imagery-competition-3rd-place-winners-interview-vladimir-sergey/
    222     """
--> 223     pad = _pad_img(input_img, window_size, subdivisions)
    224     pads = _rotate_mirror_do(pad)
    225 

/repos/Kernel-Counting/U-Net/Prediction/smooth_tiled_predictions.py in _pad_img(img, window_size, subdivisions)
     77     aug = int(round(window_size * (1 - 1.0/subdivisions)))
     78     more_borders = ((aug, aug), (aug, aug), (0, 0))
---> 79     ret = np.pad(img, pad_width=more_borders, mode='reflect')
     80     # gc.collect()
     81 

<__array_function__ internals> in pad(*args, **kwargs)

/usr/local/anaconda/lib/python3.6/site-packages/numpy/lib/arraypad.py in pad(array, pad_width, mode, **kwargs)
    739 
    740     # Broadcast to shape (array.ndim, 2)
--> 741     pad_width = _as_pairs(pad_width, array.ndim, as_index=True)
    742 
    743     if callable(mode):

/usr/local/anaconda/lib/python3.6/site-packages/numpy/lib/arraypad.py in _as_pairs(x, ndim, as_index)
    514     # Converting the array with `tolist` seems to improve performance
    515     # when iterating and indexing the result (see usage in `pad`)
--> 516     return np.broadcast_to(x, (ndim, 2)).tolist()
    517 
    518 

<__array_function__ internals> in broadcast_to(*args, **kwargs)

/usr/local/anaconda/lib/python3.6/site-packages/numpy/lib/stride_tricks.py in broadcast_to(array, shape, subok)
    180            [1, 2, 3]])
    181     """
--> 182     return _broadcast_to(array, shape, subok=subok, readonly=True)
    183 
    184 

/usr/local/anaconda/lib/python3.6/site-packages/numpy/lib/stride_tricks.py in _broadcast_to(array, shape, subok, readonly)
    125     it = np.nditer(
    126         (array,), flags=['multi_index', 'refs_ok', 'zerosize_ok'] + extras,
--> 127         op_flags=['readonly'], itershape=shape, order='C')
    128     with it:
    129         # never really has writebackifcopy semantics

ValueError: operands could not be broadcast together with remapped shapes [original->remapped]: (3,2) and requested shape (4,2)

I get the same stack regardless of whether or not I implement the changes suggested in #1

Please help! I want to use this so badly. I even tried reworking things a bit- adjusting the prediction function and not expanding the image array, like this:

im = cv2.imread(files[0])

predictions_smooth = predict_img_with_smooth_windowing(im,
                                                       window_size=256,
                                                       subdivisions=2,  # Minimal amount of overlap for windowing. Must be an even number
                                                       nb_classes=2,
                                                       pred_func=(lambda img_batch_subdiv: model.predict(img_batch_subdiv))
                                                      )

print(type(predictions_smooth))
print(predictions_smooth.shape)

And that just gave me a different error:

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-5-96028bcce00b> in <module>
      9                                                        subdivisions=2,  # Minimal amount of overlap for windowing. Must be an even number
     10                                                        nb_classes=2,
---> 11                                                        pred_func=(lambda img_batch_subdiv: model.predict(img_batch_subdiv))
     12                                                       )
     13 

/repos/Kernel-Counting/U-Net/Prediction/smooth_tiled_predictions.py in predict_img_with_smooth_windowing(input_img, window_size, subdivisions, nb_classes, pred_func)
    251     for pad in tqdm(pads):
    252         # For every rotation:
--> 253         sd = _windowed_subdivs(pad, window_size, subdivisions, nb_classes, pred_func)
    254         one_padded_result = _recreate_from_subdivs(
    255             sd, window_size, subdivisions,

/repos/Kernel-Counting/U-Net/Prediction/smooth_tiled_predictions.py in _windowed_subdivs(padded_img, window_size, subdivisions, nb_classes, pred_func)
    191 
    192     # Such 5D array:
--> 193     subdivs = subdivs.reshape(a, b, c, d, nb_classes)
    194     gc.collect()
    195 

ValueError: cannot reshape array of size 4194304 into shape (8,8,256,256,2)

I am not sure but it seems you are passing number of classes as 2, and I guess you are working on binary classification problem, in that case try using number of classes as 1.

[bnsreenu]

You get this error if you are supplying your input image with shape 4, where you pre=load multiple images into a numpy array. For example (100, 2048, 2048, 3) where 100 is the number of images and (2048, 2048,3) is the size of your each large image. This code expects input to be just (2048, 2048, 3) and it internally generates patches and provides input to your model as (num_patches, 2048, 2048, 3).

Also change Changed padx to pady in line 212 and 174 as suggested in other replies.