search

facebookresearch / esm

Evolutionary Scale Modeling (esm): Pretrained language models for proteins
MIT License
3.16k stars 627 forks source link

Question about ESM-1v results. #532

Closed elttaes closed 2 months ago

elttaes commented 1 year ago

Recently, I tried to reproduce the results of ESM-1v in the PABP_YEAST_Fields2013-singles dataset, but I found that when performing zero-shot inference on PABP_YEAST_Fields2013-singles according to the method of esm/examples/variant-prediction/predicts, the best spearman results of five esm-1v model were only 0.51, much lower than the zero-shot result 0.6658 reported in the paper. And ESM-1b is also only 0.60, lower than 0.68 which report in the paper.

In BLAT_ECOLX_Ranganathan2015 dataset I using esm-1b can get the result of spearman 0.70 which is similar to the result 0.72 in the paper.

Could you please tell me how to reproduce the results in the paper? Do I need to do some processing on the results of the five esm1v outputs, or calculate the spearman with the label separately? Thank you.

niu6211 commented 7 months ago

您好，请教一个问题，请问如何获取BLAT_ECOLX_Ranganathan2015.csv 文件