pytorch / captum

Model interpretability and understanding for PyTorch
https://captum.ai
BSD 3-Clause "New" or "Revised" License
4.96k stars 499 forks source link

Add Tutorial for Recurrent based models #1205

Open FrancescoMandru opened 1 year ago

FrancescoMandru commented 1 year ago

I wonder how should I set up my script to do some tests on a GRU / LSTM based model, with some linear layers etc. I have done some tests of your tools and they work, however I'm struggling finding which tools are well suited for these kind of networks and also given an input sequence of shape (Batch, seq_len, features), how should I interpret the output of some of these tools.

Let's take for example the IntegratedGradients tool. Given that. the ouput has 2 components, attributions and deltas. Attributions has the same shape of the input, and I wonder how would you interpret this output, moreover, how do you set your tool etc etc.

I would like to see a tutorial like the one that you have done for BERT.

jingyan-li commented 10 months ago

A tutorial regarding RNN models would be very helpful, in terms of the layer attribution, activations, and feature attribution of the model.