search

facebookresearch / esm

Evolutionary Scale Modeling (esm): Pretrained language models for proteins
MIT License
3.16k stars 627 forks source link

Unsupervised fine-tuning ESM-1v script #229

Closed NingNing-C closed 2 years ago

NingNing-C commented 2 years ago

Hi, Great work! I'm wondering if you could provide the unsupervised fine-tuning script described in zero-shot paper? Or maybe suggest how to do the unsupervised fine-tuning for a specific protein family? Many thanks!

tomsercu commented 2 years ago

Thank you! You'd just need a standard pytorch training loop and the MSA of a protein family, as described in paper Section 4.4 and Appendix B. Here's a discussion on finetuning with some further pointers: #33