search

JohnGiorgi / DeCLUTR

The corresponding code from our paper "DeCLUTR: Deep Contrastive Learning for Unsupervised Textual Representations". Do not hesitate to open an issue if you run into any trouble!
https://aclanthology.org/2021.acl-long.72/
Apache License 2.0
378 stars 33 forks source link

can i finetune the model ? #249

Closed boscoj2008 closed 2 years ago

boscoj2008 commented 2 years ago

I wondering if there is a way to fine-tune on a domain task rather than training on my small data set. Thanks

JohnGiorgi commented 2 years ago

Yup. The model is accessible via Transformers. So you can load it in that library and fine-tune it on your task in PyTorch or Tensorflow. There is documentation here. Just change the pretrained model name to "johngiorgi/declutr-base" (or our other pretrained models, see here for a list).

JohnGiorgi commented 2 years ago

Closing, please feel free to re-open if your question is not answered. Note that you can also load our models in the SentenceTransformers library (I've added some documentation on that here), so you can take full advantage of that library's utilities for training.