marcotcr / lime

Lime: Explaining the predictions of any machine learning classifier
BSD 2-Clause "Simplified" License
11.4k stars 1.79k forks source link

mark_boundaries failed to read the original input image as background #715

Open y-sheng opened 1 year ago

y-sheng commented 1 year ago

Hi, I am trying to use lime to explain an image regression model trained using autoKeras.

After fitting the explainer and marking boundaries it returned an image as attached below:

Capture

The original input is a detached house, I can see it does highlight some interesting boundaries but failed to render the original input image.

Below is the code I ran, it will be great if you can point out where I got it wrong :(

 `explainer = lime_image.LimeImageExplainer()
 from tensorflow.keras.models import load_model

 loaded_model = load_model(r"..\image_regressor\best_model", custom_objects=ak.CUSTOM_OBJECTS)
 explanation = explainer.explain_instance(imgs, loaded_model.predict,  
                                     top_labels=3, hide_color=0, num_samples=1000)

 from skimage.segmentation import mark_boundaries
 temp, mask = explanation.get_image_and_mask(explanation.top_labels[0], positive_only=False, num_features=10, hide_rest=False)
 plt.imshow(mark_boundaries(temp / 2 + 0.5, mask))`

and the best model from autokeras is structured like this:

`
Layer (type) Output Shape Param

input_1 (InputLayer)        [(None, 128, 128, 3)]     0                

cast_to_float32 (CastToFloa  (None, 128, 128, 3)      0         

conv2d (Conv2D)             (None, 122, 122, 32)      4736          

conv2d_1 (Conv2D)           (None, 116, 116, 32)      50208        

max_pooling2d (MaxPooling2D  (None, 19, 19, 32)       0         

conv2d_2 (Conv2D)           (None, 13, 13, 32)        50208       

conv2d_3 (Conv2D)           (None, 13, 13, 32)        50208         

max_pooling2d_1 (MaxPooling  (None, 2, 2, 32)         0         

dropout (Dropout)           (None, 2, 2, 32)          0               

flatten (Flatten)           (None, 128)               0                       

regression_head_1 (Dense)   (None, 1)                 129             

=================================================================
Total params: 155,489
Trainable params: 155,489
Non-trainable params: 0
_________________________________________________________________
None`