OcclusionSensitivity callback issue - get stucks or index error

[druzkaya]

TF 2.1, tf-explain 0.2.0

Hello,

I'm using tf-explain (callbacks) on about 70.000 images, 56563, and two labels (0: 13%, 1: 87%), in order to explain class 0 classification. Whereas GradCAM works perfectly, I systematically have issues with OcclusionSensitivity. With class_index=0, it gets stuck forever, with class_index=1 I get the following error message.

cb_gradcam = GradCAMCallback( validation_data=(X_val, y_val), class_index=1, output_dir=log_dir)

cb_occlusion = OcclusionSensitivityCallback( validation_data=(X_val, y_val), class_index=1, patch_size=4, output_dir=log_dir)

model.fit(x=X_train_sc, y=y_train, epochs=1, batch_size=batch_size, validation_split=0.2, validation_freq=1, use_multiprocessing=True, callbacks=[tensorboard_callback, cb_occlusion],
) --> with cb_gradcam I get good results in TensorBoard

Error for class_index=1 (and for class_index=0, just get stucks, no error message, but I have to kill the code) :

---

IndexError Traceback (most recent call last)

in 5 validation_freq=1, 6 use_multiprocessing=True, ----> 7 callbacks=[tensorboard_callback, cb_occlusion], 8 9 ) /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/tensorflow_core/python/keras/engine/training.py in fit(self, x, y, batch_size, epochs, verbose, callbacks, validation_split, validation_data, shuffle, class_weight, sample_weight, initial_epoch, steps_per_epoch, validation_steps, validation_freq, max_queue_size, workers, use_multiprocessing, **kwargs) 817 max_queue_size=max_queue_size, 818 workers=workers, --> 819 use_multiprocessing=use_multiprocessing) 820 821 def evaluate(self, /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/tensorflow_core/python/keras/engine/training_v2.py in fit(self, model, x, y, batch_size, epochs, verbose, callbacks, validation_split, validation_data, shuffle, class_weight, sample_weight, initial_epoch, steps_per_epoch, validation_steps, validation_freq, max_queue_size, workers, use_multiprocessing, **kwargs) 395 total_epochs=1) 396 cbks.make_logs(model, epoch_logs, eval_result, ModeKeys.TEST, --> 397 prefix='val_') 398 399 return model.history /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/contextlib.py in __exit__(self, type, value, traceback) 117 if type is None: 118 try: --> 119 next(self.gen) 120 except StopIteration: 121 return False /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/tensorflow_core/python/keras/engine/training_v2.py in on_epoch(self, epoch, mode) 769 if mode == ModeKeys.TRAIN: 770 # Epochs only apply to `fit`. --> 771 self.callbacks.on_epoch_end(epoch, epoch_logs) 772 self.progbar.on_epoch_end(epoch, epoch_logs) 773 /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/tensorflow_core/python/keras/callbacks.py in on_epoch_end(self, epoch, logs) 300 logs = logs or {} 301 for callback in self.callbacks: --> 302 callback.on_epoch_end(epoch, logs) 303 304 def on_train_batch_begin(self, batch, logs=None): /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/tf_explain/callbacks/occlusion_sensitivity.py in on_epoch_end(self, epoch, logs) 54 explainer = OcclusionSensitivity() 55 grid = explainer.explain( ---> 56 self.validation_data, self.model, self.class_index, self.patch_size 57 ) 58 /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/tf_explain/core/occlusion_sensitivity.py in explain(self, validation_data, model, class_index, patch_size, colormap) 47 [ 48 self.get_sensitivity_map(model, image, class_index, patch_size) ---> 49 for image in images 50 ] 51 ) /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/tf_explain/core/occlusion_sensitivity.py in (.0) 47 [ 48 self.get_sensitivity_map(model, image, class_index, patch_size) ---> 49 for image in images 50 ] 51 ) /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/tf_explain/core/occlusion_sensitivity.py in get_sensitivity_map(self, model, image, class_index, patch_size) 96 predictions = model.predict(np.array(patches), batch_size=self.batch_size) 97 target_class_predictions = [ ---> 98 prediction[class_index] for prediction in predictions 99 ] 100 /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/tf_explain/core/occlusion_sensitivity.py in (.0) 96 predictions = model.predict(np.array(patches), batch_size=self.batch_size) 97 target_class_predictions = [ ---> 98 prediction[class_index] for prediction in predictions 99 ] 100 IndexError: index 1 is out of bounds for axis 0 with size 1 ---- Any help would be much appreciated ! Thanks

[RaphaelMeudec]

@druzkaya I see you closed the issue, I'm still curious about what was going wrong?