search

cthoyt / protein-embedding-review

A list of papers and information on whether they are useful
https://cthoyt.github.io/protein-embedding-review
Creative Commons Zero v1.0 Universal
0 stars 0 forks source link

Add BERT-based models to the protein embedding review #1

Open helena-balabin opened 3 years ago

helena-balabin commented 3 years ago

I've taken a look at some of the papers that you've listed, and I suggest to add two more papers that have BERT-like architectures (rather than LSTM/ELMo-based stuff):

  1. ProteinBERT: This paper has code. However, based on quick first look, I have some concerns regarding the quality of the paper.
  2. ProtTrans: This paper includes multiple protein embedding models that are based on state-of-the-art(!) language model architectures. All of the code is available (although it looks like it's mostly examples and not actual code), and all of the models are available on huggingface as well, which makes a possible implementation much easier.
cthoyt commented 3 years ago

I actually already got ProtTrans, but it seems that you've pointed out something missing from this document - the model names!

Thanks for looking into this

helena-balabin commented 3 years ago

Now that you say it, I see it! For some reason, the title of ProtTrans is just not showing up in the overview page.

cthoyt commented 3 years ago

huh. same issue for me. the output page on this one needs some serious update either way