What are the GPU requirements for fine-tuning the WizardCoder-Python 7/13/34B models?

nlpxucan / WizardLM

LLMs build upon Evol Insturct: WizardLM, WizardCoder, WizardMath

9.06k stars 709 forks source link

What are the GPU requirements for fine-tuning the WizardCoder-Python 7/13/34B models? #216

Open matt-sharp opened 9 months ago

matt-sharp commented 9 months ago

I might need to fine-tune Wizardcoder on my source code and would like to know which model to start with. I have a single V100 with 32Gb RAM. Which size of model is it possible to fine-tune with this hardware?