Finetuning code for sequence classification, NLI task ?

McGill-NLP / llm2vec

Code for 'LLM2Vec: Large Language Models Are Secretly Powerful Text Encoders'

https://mcgill-nlp.github.io/llm2vec/

MIT License

1.31k stars 95 forks source link

Finetuning code for sequence classification, NLI task ? #131

Closed sandeep-krutrim closed 2 months ago

sandeep-krutrim commented 3 months ago

Please provide. a script for finetuning the supervised contrastive models for sentence classification and NLI task.

Is Lora Finetuning required or AutoModelforSequenceClassification can be used to train ?

vaibhavad commented 3 months ago

Hi @sandeep-krutrim,

We have provided an example of sentence classification here.

The example above keeps the language model frozen and just trains a linear classifier. However, it is also possible to train the LLM as well. Let me know if you need any more details.

vaibhavad commented 2 months ago

Closing as it is stale. Feel free to re-open if the issue still persists.