After recent conversation with Yifan, we need the following encodings for all the models (or for as many as possible):
The one after the embedding layer
The one after the encoder bit of the model (For the few models which don't have the encoder-decoder architecture, anything from the middle layers would work)
The final representation (If I understand correctly, this is what @noah already wrote the script for)
More specifically, we could use any numerical representation from the model, so any embedding between the initial embedding and the final representation could be investigated.
After recent conversation with Yifan, we need the following encodings for all the models (or for as many as possible): The one after the embedding layer The one after the encoder bit of the model (For the few models which don't have the encoder-decoder architecture, anything from the middle layers would work) The final representation (If I understand correctly, this is what @noah already wrote the script for)