search

facebookresearch / esm

Evolutionary Scale Modeling (esm): Pretrained language models for proteins
MIT License
3.16k stars 627 forks source link

Supervised contact prediction network #44

Closed mhj112358 closed 3 years ago

mhj112358 commented 3 years ago

Hi guys,

Congrats for the excellent work and great results. Great to see sequence embedding works indeed.

May I ask in your MSA Transformer supervised contact prediction network, did you use the outer concat of the query sequence embedding (or with symmetries row self-attention maps) as the only input so that it demonstrates the superior information content of sequence embedding replacing traditional MSA-related features or did you still include all the RaptorX features as input to the resnet as stated in Rives et al 2020? If latter, did you conduct an ablation study like that in Rives et al 2020, to see how much does the sequence embedding contribute to the improved contact precision?

Thanks in advance.

tomsercu commented 3 years ago

Thanks for your interest! In the MSA Transformer supervised results, we have not tried feature combination. Since we're significantly outperforming an identical resnet on potts model features, we expect not much gain anymore from feature combination.