Explore CrossEncoders - Githubissues

wri-dssg-omdena / policy-data-analyzer

Building a model to recognize incentives for landscape restoration in environmental policies from Latin America, the US and India. Bringing NLP to the world of policy analysis through an extensible framework that includes scraping, preprocessing, active learning and text analysis pipelines.

Other

33 stars 10 forks source link

Explore CrossEncoders #74

Open thefirebanks opened 3 years ago

thefirebanks commented 3 years ago

As specified in this issue: https://github.com/UKPLab/sentence-transformers/issues/350#issuecomment-757687915

The authors of SBERT recommend using CrossEncoders for sentence classification.

thefirebanks commented 3 years ago

Explore CrossEncoders

thefirebanks commented 3 years ago

Relevant links:

CrossEncoder folders: https://github.com/UKPLab/sentence-transformers/tree/master/sentence_transformers/cross_encoder
CrossEncoder usage: https://github.com/UKPLab/sentence-transformers/blob/master/examples/applications/cross-encoder/cross-encoder_usage.py
Main threads explaining how/why use CrossEncoders for our example:
- https://www.sbert.net/examples/applications/cross-encoder/README.html
- https://github.com/UKPLab/sentence-transformers/issues/350
Pretrained CrossEncoders: https://www.sbert.net/docs/pretrained_cross-encoders.html
CrossEncoders and fine-tuning?: https://github.com/UKPLab/sentence-transformers/issues/615
Retrieve and rerank pipeline: https://www.sbert.net/examples/applications/retrieve_rerank/README.html
Advantages of SBERT over BERT:
- https://github.com/UKPLab/sentence-transformers/issues/354
- https://github.com/UKPLab/sentence-transformers/issues/540
Augmented SBERT:
- Paper: https://arxiv.org/pdf/2010.08240.pdf
- Clarifications on steps: https://github.com/UKPLab/sentence-transformers/issues/553
Optional: Batch hard triplet loss: https://github.com/UKPLab/sentence-transformers/blob/master/examples/training/other/training_batch_hard_trec.py