Sentence pair classification?

McGill-NLP / llm2vec

Code for 'LLM2Vec: Large Language Models Are Secretly Powerful Text Encoders'

https://mcgill-nlp.github.io/llm2vec/

MIT License

1.17k stars 88 forks source link

Sentence pair classification? #83

Closed davedgd closed 4 months ago

davedgd commented 4 months ago

I’ve perused the examples, which are fantastic, but I’m curious: Would it be possible to extend this approach to sentence pair classification?

vaibhavad commented 4 months ago

Yes, definitely, in our paper, we do evaluate on sentence pair classification using the MTEB benchmark. Unfortunately, we don't have an example file for that at the moment. Feel free to start a PR if you are interested in contributing.

davedgd commented 4 months ago

Thank you for the quick reply, Vaibhav! We're excited about potentially incorporating your approach into a couple of our papers in progress, so I'd definitely be happy to help contribute an example via a PR. I should be working on this over the next couple of days into next week.

vaibhavad commented 4 months ago

Looking forward to it :)

Feel free to re-open the issue if you have any more questions