search

evolutionaryscale / esm

Other
1.3k stars 153 forks source link

Re-train or fine tuning ESM-3 model #97

Open anonimoustt opened 2 months ago

anonimoustt commented 2 months ago

Hi,

Is it possible to fine-tune or re-train the ESM-3 model with the protein sequence and structure or pdb file, and then apply the fine-tuned model to generate sequence or structure embeddings?

luojc712 commented 2 months ago

I have the same question. If ESM3 can be fine-tuned, how should it be operated?

AlpriElse commented 2 months ago

@anonimoustt @luojc712 Here's a comment from the older, archived Meta ESM repo from someone on the team about finetuning: https://github.com/facebookresearch/esm/discussions/33

It's an outdated comment but still relevant. The existing model does extend nn.Module and there are tutorials for other pre-trained models