search

facebookresearch / esm

Evolutionary Scale Modeling (esm): Pretrained language models for proteins
MIT License
3.26k stars 643 forks source link

Model fine tuning example #12

Closed wjs20 closed 4 years ago

wjs20 commented 4 years ago

Hi,

Thanks for making the model public! I would like to fine-tune the model for a downstream task, however in your colab model you only make inference using the pre-trained model, without any fine tuning it seems. Could you advise me on the best way to go about finetuning with this model? provide an example script?

Thanks so much.

joshim5 commented 4 years ago

Hi @wjs20, thanks for your interest! Yes, the ESM model is a subclass of nn.Module, so you can just use it as a component inside another torch model. We don't provide any examples fine-tuning the model, but there are many generic pytorch tutorials that can show how to do this, for example this one from Huggingface.

Hope this helps! I'm closing out the issue, but please let us know if you have any additional questions.

wjs20 commented 4 years ago

Thanks for the advice joshin5. I will have a go!