Integrated Gradients with pre-trained CamembertForSequenceClassification model explainer

[clechristophe]

Hi there,

I'm trying to get explanation for a classification model (TFCamembertForSequenceClassification fine-tuned on some data) using the IntegratedGradients explainer.

I'm following this great tutorial and I have an error at the end:

NotImplementedError                       Traceback (most recent call last)
/tmp/ipykernel_19827/3591270780.py in <module>
      1 # get explanation
----> 2 explanation = ig.explain(np.array(X_test),
      3                         forward_kwargs=kwargs,
      4                          baselines=baselines,
      5                          target=predictions)

/dds/miniconda/envs/py39/lib/python3.9/site-packages/alibi/explainers/integrated_gradients.py in explain(self, X, forward_kwargs, baselines, target, attribute_to_layer_inputs)
    928                 self.model(inputs, **forward_kwargs)
    929 
--> 930             _validate_output(self.model, target)
    931 
    932             if self.layer is None:

/dds/miniconda/envs/py39/lib/python3.9/site-packages/alibi/explainers/integrated_gradients.py in _validate_output(model, target)
    696     """
    697     if not model.output_shape or not any(isinstance(model.output_shape, t) for t in _valid_output_shape_type):
--> 698         raise NotImplementedError(f"The model output_shape attribute must be in {_valid_output_shape_type}. "
    699                                   f"Found model.output_shape: {model.output_shape}")
    700 

NotImplementedError: The model output_shape attribute must be in [<class 'tuple'>, <class 'list'>]. Found model.output_shape: TFSequenceClassifierOutput(loss=None, logits=(None, 2), hidden_states=None, attentions=None)`

I feel like I have to make a wrapper for the model in order to output correct probabilities but I have trouble doing that:

class TFSeqWrapper(keras.Model):
    def __init__(self, transformer: keras.Model, **kwargs):
        """
        Constructor.

        Parameters
        ----------
        transformer:
            Transformer to be wrapped.
        """
        super().__init__()
        self.transformer = transformer

    def call(self, X_test):
        """
        Performs forward pass throguh the model.

        Parameters
        ----------
        input_ids:
            Indices of input sequence tokens in the vocabulary.
        attention_mask:
            Mask to avoid performing attention on padding token indices.

        Returns
        -------
            Classification probabilities.
        """
        out = self(X_test).logits.argmax(axis=1)
        return out

Is this a normal behavior ? Can we use IntegratedGradients with Camembert/Roberta based models ? Am I on the right track to solve this ?

[jklaise]

@clechristophe were you able to run the example notebook as-is (with the default model) without issues? What version of transformers are you running this with?

What errors are you getting when using the wrapped version? It looks correct to me.

In general you're on the right track - the first error message just asks that your model adheres to the alibi expectation that the tensorflow.keras.Model has an output_shape which looks like is not the case for the bare transformers model (it returns a TFSequenceClassifierOutput object, so we need a wrapper that returns the expected type).

[clechristophe]

Hi @jklaise,

Yes the example notebook works fine with transformers 4.16.2.

When using the wrapped version, I've got the following error when computing the predictions:

---------------------------------------------------------------------------
RecursionError                            Traceback (most recent call last)
/tmp/ipykernel_21316/1438954853.py in <module>
      1 # get predictions
----> 2 predictions = automodel(X_test)
      3 
      4 
      5 # Get the baselines. Note that the baseline contains special characters (e.g, [CLS], [SEP], [UNK] [PAD]) and

/dds/miniconda/envs/py39/lib/python3.9/site-packages/keras/utils/traceback_utils.py in error_handler(*args, **kwargs)
     65     except Exception as e:  # pylint: disable=broad-except
     66       filtered_tb = _process_traceback_frames(e.__traceback__)
---> 67       raise e.with_traceback(filtered_tb) from None
     68     finally:
     69       del filtered_tb

/tmp/ipykernel_21316/1862017203.py in call(self, X_test)
     27             Classification probabilities.
     28         """
---> 29         out = self(X_test).logits.argmax(axis=1)
     30         return out

... last 1 frames repeated, from the frame below ...

/tmp/ipykernel_21316/1862017203.py in call(self, X_test)
     27             Classification probabilities.
     28         """
---> 29         out = self(X_test).logits.argmax(axis=1)
     30         return out

RecursionError: Exception encountered when calling layer "tf_seq_wrapper" (type TFSeqWrapper).

maximum recursion depth exceeded while calling a Python object

Call arguments received:
  • X_test=[['tf.Tensor(shape=(), dtype=int32)', 'tf.Tensor(shape=(), dtype=int32)', 'tf.Tensor(shape=(), dtype=int32)', 'tf.Tensor(shape=(), dtype=int32)', 'tf.Tensor(shape=(), dtype=int32)', 'tf.Tensor(shape=(), dtype=int32)', 'tf.Tensor(shape=(), dtype=int32)', 'tf.Tensor(shape=(), dtype=int3

I feel like I'm not wrapping correctly or computing the predictions correctly...

[jklaise]

Ah I think there's a typo as you're calling self(X_test) instead of self.transformer(X_test). The former would indeed result in an infinite recursion!

[gipster]

Are you sure you should call self(X_test) and not something like self.transformer(X_test) ?