search

facebookresearch / esm

Evolutionary Scale Modeling (esm): Pretrained language models for proteins
MIT License
3.16k stars 627 forks source link

details of outerdifference #263

Closed zhenyuhe00 closed 2 years ago

zhenyuhe00 commented 2 years ago

Hi, I read your ESM-2 paper and I'm curious about the details of outerdifference. the outer product is calculated as: (c, 1) dot (1, c) → (c, c) So I wonder if outer difference is calculated as: (c, 1) - (1, c) → (c, c) by pytorch broadcast

Thanks in advance!

tomsercu commented 2 years ago

yes that sounds right. It is evolved from the "outer concatenation" described in Wang et al 2017 Accurate De Novo Prediction of Protein Contact Map by Ultra-Deep Learning Model

zhenyuhe00 commented 2 years ago

many thanks!