boolean index did not match indexed array along dimension 0

[sumit-mandal]

I am getting this weird error while using AnchorImage for object detection model as well as image classification model

error says boolean index did not match indexed array along dimension 0; dimension is 100 but corresponding boolean dimension is 4

[jklaise]

I believe we've seen a similar issue before, but would need to check the historical issues.

Can you provide information about how you're configuring the explainer, i.e. what shape is the model expecting (should be 4-dimensions starting with batch) and what shape of image you're putting through explain (should be 3-dimensional as there is no batch dimension).

[sumit-mandal]

Adding the complete code here

target_size = (28,28)
train_generator = train_datagen.flow_from_directory(dir_,target_size=target_size,
                                                   batch_size = 1920, class_mode='categorical',
                                                   shuffle = False, subset = 'training')
test_generator = train_datagen.flow_from_directory(dir_, target_size=target_size,
                                                  batch_size=480,class_mode = 'categorical',
                                                  shuffle = False,
                                                  subset = 'validation')
#Fetch the data and labels
x_train,y_train = next(train_generator) # next returns the next item from the iterator
x_test,y_test = next(test_generator)

# Fix the file path
test_filepath = []
for filepath in test_generator.filepaths:
    filepath = filepath.replace("\\",'/')
    test_filepath.append(filepath)

idx = 0
plt.imshow(x_train[idx])

model = Sequential()

#First conv layer
model.add(Conv2D(16,(3,3),activation='relu',input_shape=(28,28,3)))
model.add(MaxPooling2D(pool_size=(2,2)))

# Second conv layer
model.add(Conv2D(32,(3,3),activation='relu'))
model.add(MaxPooling2D(pool_size=(2,2)))

#Third conv layer
model.add(Conv2D(64,(3,3),activation='relu'))
model.add(MaxPooling2D(pool_size=(2,2)))

model.add(Flatten())

# Hidden Dense layer
model.add(Dense(512,activation='relu'))
model.add(Dropout(0.2))

# Output neuron
model.add(Dense(3,activation='softmax'))

model.compile(optimizer='adam',loss='categorical_crossentropy',metrics=['accuracy'])

history = model.fit(train_generator,
                   epochs=2,
                   verbose=1,
                   batch_size=64)

score = model.evaluate(x_test, y_test, verbose=0)
print('Test accuracy: ', score[1])

from skimage import io
from tensorflow.keras.preprocessing import image
import skimage.transform

url = 'Dog-cat-panda/dogs/dogs_00052.jpg'

def read_and_transform_img(url):

    img = skimage.io.imread(url)
    img = skimage.transform.resize(img, (28,28))

    img = image.img_to_array(img)
    img = np.expand_dims(img, axis=0)

    return img

images = read_and_transform_img(url)

preds = model.predict(images)
prediction = np.argmax(preds)
pct = np.max(preds)

predict_fn = lambda x: model.predict(images)

image_shape = x_train[0]

resized_img = np.resize(image_shape,(28,28,1))

image_shape_1 = resized_img.shape
from alibi.explainers import AnchorImage
segmentation_fn = 'slic'
kwargs = {'n_segments': 15, 'compactness': 20, 'sigma': .5}
explainer = AnchorImage(predict_fn, image_shape_1, segmentation_fn=segmentation_fn,
                        segmentation_kwargs=kwargs, images_background=None)

image = x_test[idx]
plt.imshow(image[:,:,0]);

explanation = explainer.explain(image, threshold=.95, p_sample=.8, seed=0)

[jklaise]

@sumit-mandal I think your predict_fn is incorrectly defined:

predict_fn = lambda x: model.predict(images)

should be

predict_fn = lambda x: model.predict(x)

At the moment feeding anything to predict_fn makes no difference as the existing images will be used as input instead which would break the explainer.

[sumit-mandal]

Created predict_fn as mentioned by you. It creates a new error of Predictor failed to be called on <class 'numpy.ndarray'> of shape (1, 28, 28, 1) and dtype float32. Check that the parameterimage_shapeis correctly specified.

Full code

predict_fn = lambda x: model.predict(x)
image_shape = x_train[0]
resized_img = np.resize(image_shape,(28,28,1))
image_shape_1 = resized_img.shape

from alibi.explainers import AnchorImage
segmentation_fn = 'slic'
kwargs = {'n_segments': 15, 'compactness': 20, 'sigma': .5}
explainer = AnchorImage(predict_fn, image_shape_1, segmentation_fn=segmentation_fn,
                        segmentation_kwargs=kwargs, images_background=None)

[mauicv]

I think the issue is that the CNN you've trained takes input shape (28, 28, 3) as per:

model.add(Conv2D(16,(3,3),activation='relu',input_shape=(28,28,3)))

but then later you take an image, x_train[0], resize it to (28, 28, 1) and pass it's shape as the image_shape argument to AnchorImage:

predict_fn = lambda x: model.predict(x)
image_shape = x_train[0]
resized_img = np.resize(image_shape,(28,28,1)) # Incorrect resize for CNN input
image_shape_1 = resized_img.shape

So when AnchorExplain tests the network takes the specified image shape it throws an error. If you resize the image to (28, 28, 3) the error should resolve, try:

predict_fn = lambda x: model.predict(x)
image_shape = x_train[0]
resized_img = np.resize(image_shape,(28,28,3))
image_shape_1 = resized_img.shape

from alibi.explainers import AnchorImage
segmentation_fn = 'slic'
kwargs = {'n_segments': 15, 'compactness': 20, 'sigma': .5}
explainer = AnchorImage(predict_fn, image_shape_1, segmentation_fn=segmentation_fn,
                        segmentation_kwargs=kwargs, images_background=None)

[sumit-mandal]

Wow.!! That worked like a charm. For image classification anchor image is sorted. I am getting the same error when trying it on yolo_v3 architecture.

photo_filename = './000023.jpg'
image_with_bbox, pred_boxes, pred_classes,pred_scores = prediction(photo_filename)

print("pred_classes",pred_classes)

dirname,basename = os.path.split(photo_filename)

dirname1,basename1 = basename.split(".")

print('dirname1',dirname1)
print('basename1',basename1)

def lime_classification_function(list_images_numpy):
    probabilities = []

    for i in range(len(list_images_numpy)):

        img = list_images_numpy[i]
        # the yolo prediction function you mentioned

        # class_probs = v_scores
        probabilities.append(pred_scores)
#         probabilities = np.argmax(probabilities)
    # print(probabilities)
    # print('Length of probs',len(probabilities))
    print(type(probabilities))

    return probabilities[0]

def load_image_pixels(filename, shape):
        # load the image to get its shape
        image = load_img(filename)
        width, height = image.size
        # load the image with the required size
        image = load_img(filename, target_size=shape)
        # convert to numpy array
        image = img_to_array(image)
        # scale pixel values to [0, 1]
        image = image.astype('float32')
        image /= 255.0
        # add a dimension so that we have one sample
        image = expand_dims(image, 0)
        return image, width, height

image_to_explain = load_image_pixels(photo_filename,(28,28,3))

resized_img = np.resize(image_to_explain,(28,28,3))

image_shape_1 = resized_img.shape
from alibi.explainers import AnchorImage
# explainer = lime_image.LimeImageExplainer()
segmentation_fn = 'slic'
kwargs = {'n_segments': 15, 'compactness': 20, 'sigma': .5}
explainer = AnchorImage(lime_classification_function, image_shape_1, segmentation_fn=segmentation_fn,
                        segmentation_kwargs=kwargs, images_background=None)

class_names = ["aeroplane", "bicycle", "bird", "boat", "bottle","bus", "car", "cat",
               "chair", "cow", "diningtable", "dog", "horse", "motorbike", "person",
               "pottedplant", "sheep", "sofa", "train", "tvmonitor"]

explanation = explainer.explain(image, threshold=.95, p_sample=.8, seed=0)

[jklaise]

@sumit-mandal I think it might be again an issue with your prediction function, e.g. what is the type of list_images_numpy and what is the type of the return array? As mentioned before, the type of the prediction function should always be Callable[[np.ndarray], np.ndarray] where there is only one argument np.ndarray with leading batch dimension and the return value should also be np.ndarray with leading batch dimension. For a concrete example, a 3-channel image classification use-case should have np.ndarray of shape (batch_size, height, width, n_channels) and return an np.ndarray of shape (batch_size, prob1, prob2, ...). Please have a read through https://docs.seldon.io/projects/alibi/en/latest/overview/white_box_black_box.html for more detail.