search

UKPLab / sentence-transformers

State-of-the-Art Text Embeddings
https://www.sbert.net
Apache License 2.0
14.83k stars 2.44k forks source link

TextCNN on token_embeddings ? #996

Closed huubinh49 closed 3 years ago

huubinh49 commented 3 years ago

I am trying to finetune to extract more information from BERT model. Can I use SentenceTransformer for returning token_embbedings and apply sentence_transformers.models.CNN as TextCNN on this token embeddings?

nreimers commented 3 years ago

Yes, here is an example: https://github.com/UKPLab/sentence-transformers/blob/master/examples/training/avg_word_embeddings/training_stsbenchmark_cnn.py

However, I would not expect an improvement from this. BERT is already a powerful model, adding a CNN will not really help.