search

unslothai / unsloth

Finetune Llama 3.1, Mistral, Phi & Gemma LLMs 2-5x faster with 80% less memory
https://unsloth.ai
Apache License 2.0
15.23k stars 1.02k forks source link

How to evaluate the performance of an Unsloth Fine-tuned model? #749

Open atmikshetty opened 1 month ago

atmikshetty commented 1 month ago

I have been trying to evalute Unsloth Fine-tuned LLAMA3 model using "evaluate" provided by HuggingFace, but while it seems to br working fine for models fine-tuned in other ways, it throws a lot of errors for UnSloth AI Fine-tuned models. Can you please make a colab notebook to check how the fine-tuned model works on new data?

danielhanchen commented 1 month ago

I'm unsure with the evaluate library - could you elaborate what exactly it does?