search

unslothai / unsloth

Finetune Llama 3.2, Mistral, Phi & Gemma LLMs 2-5x faster with 80% less memory
https://unsloth.ai
Apache License 2.0
16.27k stars 1.12k forks source link

Suggestion - FAQ w Physical System Requirements, #469

Open LaughterOnWater opened 4 months ago

LaughterOnWater commented 4 months ago

Would you consider adding a beginner's FAQ section or a 'Physical System Requirements' note that provides guidance on the recommended computer specs for optimal performance with Unsloth? This would help new users avoid frustration and ensure they have the necessary resources to run the model successfully. A simple table or paragraph mentioning the minimum/recommended CPU, GPU, RAM, and storage requirements would be very useful.

Including links to beginner-friendly tutorials, videos, or documentation on setting up Unsloth locally could make it more accessible to us noobs. Links on how to get started in a Google colab or AWS hosted solution could also lower barriers for those who don't have a capable local system.

It might be nice to get a clue on how long it takes to fine-tune, likely sweet-spot settings to mess with, etc.

This might ease the overall beginner experience and make Unsloth more approachable for a broader audience.

danielhanchen commented 4 months ago

That is a fantastic idea!

Approximately:

Mistral 7b: Min 6-8GB VRAM Llama-3 8b: Min 8-12GB VRAM Gemma 2b: Min 8GB VRAM Gemma 7b: Min 12GB VRAM Llama-2 7b: Min 8GB VRAM Llama-2 13b: Min 16GB VRAM Llama-2 34b: Min 24GB VRAM Llama-3 70b: Min 48GB VRAM

I'll colate them in a table