search

unslothai / unsloth

Finetune Llama 3.2, Mistral, Phi, Qwen 2.5 & Gemma LLMs 2-5x faster with 80% less memory
https://unsloth.ai
Apache License 2.0
18.77k stars 1.32k forks source link

Unsloth for models that are not currently supported #1338

Open scigeek72 opened 6 days ago

scigeek72 commented 6 days ago

Hi, Thanks for the wonderful library. People like me with limited resources are grateful since it helped me with fine tuning some of the larger models which would have otherwise remain beyond my reach.

However, I want to fine tune a model ("defog/sqlcoder-7b-2"/"defog/sqlcoder2") on my custom data using Unsloth but I don't think I can since this model is not in the list of supported models on unsloth repo. I am wondering if it is possible to add this model to the list.

A somewhat related question is: Is there a way for us to use Unsloth on models that aren't explicitly supported by Unsloth yet?

Thanks.

danielhanchen commented 4 days ago

All model support is coming soon!!

adamo1139 commented 14 hours ago

If you just load this model in the script, it should work.

https://huggingface.co/defog/sqlcoder-7b-2/blob/main/config.json

This model is llama arch and unsloth works with most if not all models with LlamaForCasualLM architecture. It's a Codellama 7b finetune.