Error with finding pertinent positives and negatives using the CEMExplainer

[divyat09]

I am using CEMExplainer to generate explanations for my dataset of the following shape: (91,3)

However, I face the following error:

---

IndexError Traceback (most recent call last)

in 10 11 (adv_pn, delta_pn, info_pn) = explainer.explain_instance(train_dataset, arg_mode, ae_model, arg_kappa, arg_b, ---> 12 arg_max_iter, arg_init_const, arg_beta, arg_gamma) ~/AIX360/aix360/algorithms/contrastive/CEM.py in explain_instance(self, input_X, arg_mode, AE_model, arg_kappa, arg_b, arg_max_iter, arg_init_const, arg_beta, arg_gamma) 75 target_label = orig_class 76 ---> 77 target = np.array([np.eye(self._wbmodel._nb_classes)[target_label]]) 78 79 # Hard coding batch_size=1 **IndexError: index 80 is out of bounds for axis 0 with size 1** --------------------------------------------------------------------------- The code that I use to find pertinent negative/postitive: --------------------------------------------------------------------------- mymodel = KerasClassifier(pred_model) explainer = CEMExplainer(mymodel) arg_mode = "PN" # Find pertinent negative arg_max_iter = 1000 # Maximum number of iterations to search for the optimal PN for given parameter settings arg_init_const = 10.0 # Initial coefficient value for main loss term that encourages class change arg_b = 9 # No. of updates to the coefficient of the main loss term arg_kappa = 10 # Minimum confidence gap between the PNs (changed) class probability and original class' probability arg_beta = 1e-1 # Controls sparsity of the solution (L1 loss) arg_gamma = 100 # Controls how much to adhere to a (optionally trained) autoencoder (adv_pn, delta_pn, info_pn) = explainer.explain_instance(train_dataset, arg_mode, ae_model, arg_kappa, arg_b, arg_max_iter, arg_init_const, arg_beta, arg_gamma) ---------------------------------------------------------------------------

[sadhamanus]

Hi Divyat, One obvious error I can see in your call is that you have set gamma to 100 when I presume you do not have an autoencoder that you have supplied. In such a case gamma should be 0.

Also note that a PN may not be always possible to find especially if the input is in the interior of the class distribution. You can of course play with the parameters to try to find one (viz. reduce kappa, increase iterations etc.)

[vijay-arya]

@divyat09 In addition to what @sadhamanus pointed, here are some details about code line 77 which might help you debug your example:

self._wbmodel._nb_classes should return the number of output classes in your trained model. For example, for mnist example, it returns 10. target_label should return the output class corresponding to the data instance you are trying to explain. For example, in mnist example, if we try to explain an image instance that is classified by the model as digit "3", then target_label would be set to 3. So effectively, target variable would be a [1,10] array for mnist.

Are you training a Keras model on your dataset? Just verify in your example if "self._wbmodel._nb_classes" is returns the correct number of output classes for your model or throws an error.

[divyat09]

@sadhamanus
I am using auto-encoder, I have trained an auto-encoder in Keras and pass it correctly to the PN API. Also, just to experiment, I set gamma=0 but still got the same error.

@vijay-arya I tested the output of "explainer._wbmodel._nb_classes" and the output is "1". I believe its incorrect since I am solving a binary classification problem. Yes, I am using Keras to train models. Also, are the target labels in range [1, num_total_labels] or [0, num_total_labels-1]? The Keras model which is being explained using CEM actually has labels as 0 and 1.

[vijay-arya]

@divyat09 Can you try creating a model that has 2 output classes, something along the lines of the following example: https://nbviewer.jupyter.org/github/IBM/AIX360/blob/master/examples/tutorials/HELOC.ipynb#c2 In case you've added a softmax, please also remove that as CEM requires logits as output.