search

unslothai / unsloth

Finetune Llama 3.1, Mistral, Phi & Gemma LLMs 2-5x faster with 80% less memory
https://unsloth.ai
Apache License 2.0
14.57k stars 962 forks source link

please add support for deberta model #240

Open praveenhm opened 5 months ago

praveenhm commented 5 months ago

Add a support for deberta model "MoritzLaurer/deberta-v3-large-zeroshot-v1.1-all-33". This model is heavily used for text classification.

danielhanchen commented 5 months ago

Hmm an issue is Deberta is an encoder decoder BERT type model right? I might add support for other BERT type models in a future release :)

praveenhm commented 5 months ago

This is encoder only from the BERT family, like BERT -> distilBert -> deberta. This is highly used in text classification and fine tuning.