Add Primary Attribution support for BERT classification

[jalammar]

Primary attribution are currently supported for causal (GPT) models and enc-dec (T5/T0) models. It would be great to add support for MLM models. A good first implementation would target either BERT or RoBERTa. My sense is that Sequence Classification takes precedent, then token classification.

BertForSequenceClassification RobertaForSequenceClassification

Captum's Interpreting BERT Models guide is a good place to look.

As far as how to fit it into Ecco, these are initial ideas:

• We'll need a way to indicate the classification head in init.py. Likely a new head config parameter .
• We may have to add a 'predict()' or classify() method to lm.py. ("predict" to match the familiar sklearn api, "classify" to be more specific as generation is also prediction)

[guustfranssensEY]

Hey Jay,

Do I understand correctly that it is currently not possible to use Ecco for custom trained Bert models?

Or is it through the use of a custom model config or by passing it as a Pytorch model? If so, is there more documentation on this besides the following

[jalammar]

It's possible to use custom bert models for neuron activation capture/factorization and CCA. But not yet for Primary Attribution.

[guustfranssensEY]

Good to hear that it is possible! To your knowledge, is there example code that showcases how to get insights with Ecco from a custom Bert model trained from the AutoModelForSequenceClassification class?

I assume I will need to use the LM class instead of the from_pretrained()

[JoaoLages]

It would be easy to implement this for MLMs as it requires a single forward step for classification. We would just need a new visualization from the JS side :) Just wanted to point out that this package is already covering a lot of what you intend. Maybe it would be healthy to have a chat with them.