Feature importances with protein embedders

[evabalini]

Hello,

Thank you for your work and efforts on bio_embeddings. The library is really easy to use and well documented. I was wondering if you have any insight on the explainability of an embedder. More specifically, let's say we use the embeddings from esm and try to predict a certain target. Is there a way to know which region/position(s) of the sequence is important for the prediction? This is not a question specifically for ESM but for all the embedders included in the package.

Thank you in advance! Eva Balini

[nilsimda]

Hi,

since this is also part of my research I'll share my insight on this. I am by no means an expert on this though. The embedders are typically large language models that use the transformer architecture as a basis. This means they often have hundreds of millions of parameters that all determine the prediction, which makes it harder to rank feature importance and is the reason that Deep Learning Models are often talked about as black boxes. So to answer your question: There is no straight forward way to rank feature importance after using a large protein language model as an embedder. Some things to consider to gain insight into which parts of the input sequences are relevant might be:

• Dropping parts of the input sequences and see how model perfomance is impacted
• Make use of the Attention Mechanism that is part of the Transformer Architecture (this may be difficult, especially with larger models)

Hope this helps! Nils Imdahl

[sacdallago]

Thanks for the contribution @nilsimda . @evabalini : in short, we don't exactly know what the embeddings encode, in what layers, or on what heads. It's an open research question.

For specific targets, and modifications there of (so, you are working around some WT sequence), probably some heads will fire up, that may be responsible for structural features, or even encode for binding ability of residues/pockets... but: mine is speculation! Would need extensive research. I was recently on a call with some collaborators where we were trying to find out some of these relationships (sequence-structure-embeddings), and for a single protein it takes a week to get to some level of "intepretability", which may not be 100% translatable to another sequence (and definitely not to a different embedder).