search

facebookresearch / esm

Evolutionary Scale Modeling (esm): Pretrained language models for proteins
MIT License
3.16k stars 627 forks source link

ESMForMaskedLM not in transformers #158

Closed fteufel closed 2 years ago

fteufel commented 2 years ago

Hi, I saw that ESM now seems to be available in the transformers library: https://huggingface.co/facebook/esm-1b However, the imports 

from transformers import ESMForMaskedLM, ESMTokenizer
----> 1 from transformers import ESMForMaskedLM, ESMTokenizer

ImportError: cannot import name 'ESMForMaskedLM' from 'transformers' ('home/miniconda3/envs/testenv/lib/python3.9/site-packages/transformers/__init__.py)

don't seem to work in 4.15.0. Is there a fork of the library available somewhere that has them?

liujas000 commented 2 years ago

Hi @fteufel ! We have a pending pull request here that you should be able to use as a temporary solution

gianhiltbrunner commented 2 years ago

The pull request seems to be closed now. :(

ratthachat commented 2 years ago

The pull request seems to be closed now. :(

For future ref on temp solution, We can use the following command to install the temp PR directly

!git clone -b add_esm-proper --single-branch https://github.com/liujas000/transformers.git 
!pip -q install ./transformers

Note that in this branch, it’s transformers version 4.13

felixgabler commented 2 years ago

@liujas000 do you think it would be possible to also add ESM2 to your PR (or just repo for pip -q)? Would be fantastic!

tomsercu commented 1 year ago

Very happy to share that - thanks to the amazing work of @Rocketknight1 and team - this is now supported:

Classification tasks with proteins, just like BERT: https://colab.research.google.com/github/huggingface/notebooks/blob/main/examples/protein_language_modeling.ipynb

Fold proteins in Colab or your local GPU and export PDB files: https://colab.research.google.com/github/huggingface/notebooks/blob/main/examples/protein_folding.ipynb