T5 Unsupervised Learning(pretraining)

UKPLab / sentence-transformers

Multilingual Sentence & Image Embeddings with BERT

https://www.SBERT.net

Apache License 2.0

14.37k stars 2.39k forks source link

T5 Unsupervised Learning(pretraining) #911

Open prince14322 opened 3 years ago

prince14322 commented 3 years ago

Can you please let me know how can we unsupervised training for T5 model? This link and this link gives example for Bert, XLNet, GPT models using CML or MLM pr PLM.

nreimers commented 3 years ago

Hi @prince14322 I sadly have no experiences with pre-training T5.